To help control the movement of cutting circles on the cutter arm, I installed a drill bit stop collar on its tail end. After making a rough adjustment, I slide the stop collar up against the arbor and tighten it down.
Make a simple adjustable fence and a right angle drilling jig to make your drill press work quicker and more accurately. Store them nearby so they are handy when you need them.
See how a woodworker built a piston stop block to their shop-built drill press fence.