I've had this problem on one J-3 and on two Zeiss Jena 5cm F1.5's original in LTM before. On one Zeiss lens, the rear group was loose and had to be tightened. On the J-3, There was too much slop in the helical, and I used a thicker grease to steady it. The 2nd Zeiss lens had "lead a rough life", but at $170 I did not have any complaints. The focus mount was loose in the lens mount. It wobbled almost 0.1mm, enough top throw the focus off.
Try This: Focus the lens on the top of a lamp, or someting else that is very easy to see. Then pull on the focus ring. Push down on it, try to make it wobble. You should see the RF move off of the target. That is what my Zeiss lens did, and enough to throw things off at F2.8. I made a paper liner to keep the lens from wobbling in the mount, corrected the problem. If yours wobbles, I would have a service shop try to tighten things up.