Make the offhand snap to the gun, instead of gun snapping to the hand.
It currently creates this weird "teleport" and your aim gets thrown off. I suggest letting the offhand teleport to the weapon instead of the other way around, and keep the gun pointed where it was.

vid attached: https://i.imgur.com/aDCcBiN.mp4