Setting up a lens test .... advise.

raid

raid

Dad Photographer
Local time
3:38 PM
Joined
Nov 2, 2005
Messages
36,578
I want to do another test for the eight or nine 50mm rangefinder lenses I own. In my last (flare) test, I used Ilford XP2 Super film, and I used window light and indoor lights for illumination of the model. When setting ASA at 200, I had exposure setting f 2.0 @ 1/60 and f 4.0 @ 1/15. It was suggested that slower film would have been better for such a test. I now want to test the overall quality of each lens for portraits without shooting into the window light. Flare has now been tested and I want to move on to test something else. If I use slower film indoors, I must use a flash. Would this be OK for a lens test? Is color film also an alternative here (with flash)? The last option I see is taking the photos outdoor with slower film and natural light. This may be a little more complicated due to having 8-9 lenses and the need to keep my little model sitting without much moving around in the garden.
Your suggestion are requested.
 
Try using your wife as a model. She's apt to keep more still so you can be more pecise in your focus although 9 lenses would probably try her patience without the promise of jewelry. Another altenative would be a manaquin or large doll if you want to be critical in your testing.
 
What are you testing for? I would confine the test to give results that can be compared so you can make clear judgements.
 
Finder said:
What are you testing for? I would confine the test to give results that can be compared so you can make clear judgements.
Click to expand...


I want to see which lens gives a better overall result. This could include sharpness and contrast and overall look.
 
awilder said:
Try using your wife as a model. She's apt to keep more still so you can be more pecise in your focus although 9 lenses would probably try her patience without the promise of jewelry. Another altenative would be a manaquin or large doll if you want to be critical in your testing.
Click to expand...


Oh boy ... more jewelry is needed! I will see what I can do here. I don't like using a large doll or similar as I like to send some of the resulting photos to family and friends. I'm nor sure they would understand my use of an inflateable doll, as Joe is suggesting!
 
No flash please...

No flash please...

...unless you do a ceiling bounce.

Else you'll get hotspots, uneven illumination, etc.

Shoot a still subject (like a doll as suggested or a Barney doll or perhaps a flower or plant), use B mode to get the right exposure (remember reciprocity failure), bracket as necessary.

Remember that right focus and exposure are critical for evaluating resolution, contrast, look, etc. You may even want to measure the distance to the subject to confirm that your rangefinder is accurate.

Wai Leong
===
raid amin said:
I want to do another test for the eight or nine 50mm rangefinder lenses I own. In my last (flare) test, I used Ilford XP2 Super film, and I used window light and indoor lights for illumination of the model. When setting ASA at 200, I had exposure setting f 2.0 @ 1/60 and f 4.0 @ 1/15. It was suggested that slower film would have been better for such a test. I now want to test the overall quality of each lens for portraits without shooting into the window light. Flare has now been tested and I want to move on to test something else. If I use slower film indoors, I must use a flash. Would this be OK for a lens test? Is color film also an alternative here (with flash)? The last option I see is taking the photos outdoor with slower film and natural light. This may be a little more complicated due to having 8-9 lenses and the need to keep my little model sitting without much moving around in the garden.
Your suggestion are requested.
Click to expand...
 
In order for the test to actually have any significant meaning you need to eliminate as many variable from the system as possible. This would require a solid tripod and fixed subject. The slightest bit of movement from the camera or subject, especially in a portrait that is focused up close, and all of a sudden a sharp lens will be pronounced "soft." If you want some usable nice portraits, go for it, but it would be dubious to then claim the exercise a fair lens test if you are examining sharpness. With that many lenses to test you also run the risk of natural light actually changing during the process (clouds, time of day, shifting shadows...). The more controlled the environment the better. Make sure you post any results - not all of us have 9 50mm lenses to choose from and we would like to see a comparison!
 
I would run a test chart series beside your model shots to help you evaluate the results.
I always like the test setup DPreview uses, he only uses live models for testing flash results and inanimate subjects for the rest.
 
I would shoot slide film for the test. It will take away the variable of bad focus on the printers side of it and test for the best colors too.
 
A lens' color signature is as importment as its resolution. So unless you only shoot B & W, color is a must.

One mistake I've noticed in these homegrown tests is that people tend to shoot a human face fairly centered in the field. Our minds are inately tuned to latch onto to facial images (esp. the eyes) which in turn means in such shots were apt to subconciously give less weight to details on the periphery. This leads to a bias in results toward lenses which resolve better in the center but may do poorly on the edges over ones which may do a better job over the whole field. In other words, shoot something of interest over the whole frame of your negative.
 
I think the method all depends on your reasons for testing. If you're looking to publish the results on the internet and that they should have relevance for others, you need to be as universal as possible since everybody has different photographic needs. Then again, from what I've observed, no matter what you do, the people who own a particular lens that scores lower on your test will belittle your test methodology and perhaps even challenge your intelligence. OTOH if the test is to have significance for your own photography then I'd say to use subject, film, lighting, aperture etc. as you anticipate shooting in real life, because the results will be more relevant to you.

As an example, I had a chance to test the new 50 Summilux-ASPH against my pre-ASPH. There's only one reason I would pay extra and carry the weight of a 50/1.4 vs a Cron, and that's to shoot in very low light, handheld, on fast b&w film. So my side-by side shots were made in a jazz bar, at about 15 ft, at f/1.4 and 1/8 sec, on ISO 400 b&w, and there was absolutely no detectable difference in sharpness or contrast anywhere in the frame. Had I shot these lenses together on a tripod using slow slide film with a planar subject with fine detail throughout, I might have seen where the improvements were made to the ASPH, but since I would never shoot such a lens of that kind of subject (I'd use a Hasselblad if I wanted maximum fine detail), such a test might have been useful to justify my having spent $2500 on that lens, but would have had no relevance to how I would be using it.
 
It lis much easier for me to replace a live model by some object that is not moving at all, so this advice by Weileong certainly will be followed.

I also will follow Nick's advice in using both color and B&W film. This is not a problem at all. I shoot a soften color as I shoot with B&W. I will try to remember not to center the object of interest and set it slightly to one side of the frame.

As for Japp's suggestion to use a test chart, I must confess that I don't have one and never checked out before where online to get one. My own "test chart" was a peice of white paper with many letters on it, similar to an eye sight test but more home made.

I often use slide film, but scanning all the slides will be time consuming, and I rather use a method in which I can ask a commercial lab to provide me with scans.

I agree with Ben's comment above. In my recent test for eight 50mm lenses in which I tested for flare, the results of the Summicron (rigid first version) were below what is expected of a Summicron lens. My camera repair person told me yesterday that the amount of haze he found was not much, and he still has to clean it. Most people commenting on the test result were surprised by the flare and they suggested that I check the lens out for internal haze.

I rarely see a test for 8-9 similar lenses, and this is my main gaol here. I want to see if I can find real differences between some of the lenses used in the test, and I also enjoy reading your comments since different photographers may have different preferences. Sometimes certain small things are pointed out by some people that I did not even notice.
 
Ben that one of the best written posts about practical lens testing i've read on the internet forums.
Thank you.

Simon
 
Yes, practicality is the most important factor when it comes to lens testing. How practical is the test and how realistic the findings? If I shoot mostly handheld then the test should be also this way even though I may induce some variation due to handshake and related issues, but the test results would reflect more accurately how the lenses will be used. Replcation in such a case is very important to obtain more validity.
 
Ben Z said:
I think the method all depends on your reasons for testing. If you're looking to publish the results on the internet and that they should have relevance for others, you need to be as universal as possible since everybody has different photographic needs. Then again, from what I've observed, no matter what you do, the people who own a particular lens that scores lower on your test will belittle your test methodology and perhaps even challenge your intelligence. OTOH if the test is to have significance for your own photography then I'd say to use subject, film, lighting, aperture etc. as you anticipate shooting in real life, because the results will be more relevant to you.

As an example, I had a chance to test the new 50 Summilux-ASPH against my pre-ASPH. There's only one reason I would pay extra and carry the weight of a 50/1.4 vs a Cron, and that's to shoot in very low light, handheld, on fast b&w film. So my side-by side shots were made in a jazz bar, at about 15 ft, at f/1.4 and 1/8 sec, on ISO 400 b&w, and there was absolutely no detectable difference in sharpness or contrast anywhere in the frame. Had I shot these lenses together on a tripod using slow slide film with a planar subject with fine detail throughout, I might have seen where the improvements were made to the ASPH, but since I would never shoot such a lens of that kind of subject (I'd use a Hasselblad if I wanted maximum fine detail), such a test might have been useful to justify my having spent $2500 on that lens, but would have had no relevance to how I would be using it.
Click to expand...

But Ben and Raid, under these uncontrolled conditions, (especially the handheld shots at low speeds), perhaps the $40 Jupiter 8 lens will have faired even better, if at the moment of exposure, your hands happened to hold the camera slightly more still during that exposure.

Coming from a science background as I do, I have to take exception to your basic premise of lens testing. If you are going to bother testing lenses against each other, you HAVE to eliminate all variables other than the lens being tested otherwise you cannot come to a conclusion about the performance of a lens where many variables have affected the outcome.

The idea of "real world shooting conditions" just doesn't work if you are testing lenses. Either you understand that or you don't, but that is the way it is, and I stand behind that statement 100%, with all of my integrity on display for ridicule if it is incorrect.

Raid, as a statistician, I am somewhat surprised that this is not obvious to you.
 
Last edited:
I think there's also a subjective more emotional issue here to do with the cost of certain lenses.
I posted a WTB ad on another site and it fueled a very interesting debate about the pros and cons of pre ASAP and ASAP 35 luxes. I was only looking for someone selling one! I like many people, i'm sure, read a lot of reviews and comments posted by users on many forums and although i agee with Frank's scientific approach and others like Erwin's lens reviews when it comes to evaluating lenses using established benchmarks to show different lenses characteristics,i'm also fascinated by Dante Stella's reviews of lenses on his very imformative site which seems to base evaluations based on the way they perform when used in the field ( he certainly illustrates it with great examples). I'm ot really sure if "emotional" is exactly the right word here but it certainly a kick in the lower regions when you've shelled out a large amount for the latest Leica ASAP (for example) and then discovered when you're printing your negs you can't tell the difference! Sure it feels good to know the lens you just invested in is worth the expense and hits the mark in latest technological lens developments but often when the negs are all printed and evaluated its the very moment captured in the photograph that makes it a winner and as is often the case with me!
What counts then is whatever's on the camera. I think i've rambled on a bit and time to get back to scanning my negs........

Simon
 
Last edited by a moderator:
There's a world of difference between a lens's potential and the perfection of a moment captured by the photographer. You can demonstrate one under controlled conditions, the other is totally up to the photographer's skill.
 
Hi Frank
yes you're right - i think i was just trying to say that what sometimes fuels my GAS attacks are reviews which make you want to go and buy the latest lens version - then i have to control myself and think how i print my images and delete my saved searches on Ebay!
Simon
 
Simon, and then the issue becomes more confused if, as I do, one searches for classic, vintage results in lenses as opposed to simply sharper.
 
You must log in or register to reply here.
Back
Top Bottom