I never had a llot of success using a blind mandrel. I turn mine between centers then chuck the insert up and drill my tone channel. There are a ton of tutorials here and if you have the time to run thru old threads you will find a wealth of info.. Good luck
Your problem is you are letting the end grain of the insert slightly bend the blind mandrel. Instead of staying on center the live center point is seeking softer wood. I turn my blank round first, that gives me a
solid center on both ends, then drill and then use my blind mandrel, result is rarely off center.
I really do not think there is a perfect way, every method I have tried has faults, even my current one. By next week or next month I may be doing it different. Even the wood can dictate what method is best.
Marvin