Yes, you are doing way too much
screwing around. Use the lighter scope, get rid of the weights and shoot a LV all the way thru. Or else get another action, stock, etc., and build a dedicated HV and have a HV and a LV that you do not have to make changes in.