First what type of drill bit do you use? My recommendation is to stay away from spade bits, they follow the path of least resistance, and the hole is not straight nor round. Next you cannot judge a an unfinished hole as to the size, when you put a finish in the inside it will be smaller, not much but it will be smaller.
I use 1/2" jobber drill bits, with 2 or 3 coats of finish I haven't had a problem with the size. Another factor to consider, when someone starts using the call the humidity from their breath will cause the wood to swell, and if the guts are to tight it might crack the wood.
Next, 1/64" is so minor when considering the natural movement of the wood, the thickness of the finish, the accuracy of a straight drilled hole and finally the accuracy of your drilling method it won't even come into play. Look at the shape of the wedge on the guts, it will account for almost all of the errors of above, but trying to drill the hole out larger is a nightmare.
Marvin