I have experienced the same
I select a powder charge that proiduces good speed with good crono numbers and then tune it in using seating depth. I use .003 as my incremental change and I too usually find a round hole within the first 5 increments. There is sometimes the odd barrel that will like a .015 jam but usually the hole comes in the first .009. I have wondered about but never felt I had to adjust powder a bit after finding the single hole, to make it smaller. Logically one would think that would be the thing to do.
For those who don't think precise seating depths and exact OAL's, measured where the bullet will engage the lands matters, this process will show one very quickly.