Today I rigged up the collimation set-up, I put the J12 on a Kiev 4; same camera that produced the shots above. I used a ground glass with a mark on the ground surface and made sure the ground surface was facing the lens. Surprise, surprise, with both rings in place the focus is absolutely perfect! With only the thickest ring in place I had to shift the focus on the 135mm Nikkor to halfway between infinity and 20m, with only the thinnest ring in place much worse.
Then I put the J12 on a Contax II with both rings in place, ever so slightly out of focus as if it could have done with perhaps 0,1mm extra shims.
I also tried the method TenEleven suggested, it looks OK though I found it a bit difficult to execute.
I think I need to shoot a test roll with the thinnest ring removed to see what happens.