Simply put, metal removal and surface finish requires force, as it's really rubbing off the chip, not cutting it. If not enough force is applied, ie, via speed, feed and depth, you end up scratching off the metal which leaves a rough surface and gouges in the part. It's not unusual to use a .020-.050 finish pass on high quality steel to get a fine finish and hold tolerance.If you find yourself needing to remove another .002, you missed the finish depth. That's the point where the files and sand paper come out of the drawer. A .002 cut is doable but you will need extreme rpm to get away with it. You try to avoid ending up .002 short at all costs.
Of course that's hardly practical machining barrels, but the cutting tool does not know the difference.
Tool grinding is a skill all it's own. There is a work around though and that is HS inserts that are available that allow one to have precision quality cutting tools without the 20 year learning curve. Tools that are better suited to the operations a gunsmith employes. BTW, most inserts designed specifically for stainless are actually HS and not Carbide. FWIW most exotic metals are not machinable with carbide.