Setting up a lens test .... advise.

[raid]

But Ben and Raid, under these uncontrolled conditions, (especially the handheld shots at low speeds), perhaps the $40 Jupiter 8 lens will have faired even better, if at the moment of exposure, your hands happened to hold the camera slightly more still during that exposure.

Coming from a science background as I do, I have to take exception to your basic premise of lens testing. If you are going to bother testing lenses against each other, you HAVE to eliminate all variables other than the lens being tested otherwise you cannot come to a conclusion about the performance of a lens where many variables have affected the outcome.

The idea of "real world shooting conditions" just doesn't work if you are testing lenses. Either you understand that or you don't, but that is the way it is, and I stand behind that statement 100%, with all of my integrity on display for ridicule if it is incorrect.

Raid, as a statistician, I am somewhat surprised that this is not obvious to you.

Frank,
If you take many photos at the same setting with one camera, then even handheld photos may give you an insight of its performance. Don't you agree with this? Being a statistician, my professional live has been all about quality control and experiments, so I would not do this with a "scientifically planned experiement". I agree with you on this issue.

 

[raid]

I certainly will have a controlled experiment in my test, and this includes all factors I can think of that I can fix for all lenses used. I guess, the hand-held in the field testing of lenses is more a personal matter. In my last test, I had three replications, and I am quite confident that my test results overall hold up. My lenses were either recently CLA'd or looked very clean and did well when used recently. The only lens I took to a camera repair person to check out for internal haze was the Summicron (rigid). Even that one may actually be quite clean internally, but I still have the camera repairman clean it anyways.

 

[Simon Larbalestier]

That's very true Frank, in fact that's exactly why i purchased a Nikkor 3.5/2.5 for my S3 and am looking for the early 35 1.4 lux. One day i will have to stop.
OT i find myself searching for vintage bags like old fishing bags that are well worn......

Simon

 

[FrankS]

In my opinion, the greatest and limiting factor in lens performance is camera movement during exposure. To test and compare the performance of different lenses without controlling this variable, again in my opinion, is senseless. Simple as that.

 

[raid]

I used a heavy tipod and also a cable release in order to reduce vibration and camera movement. This also allowed me to focus more accurately.