Testing

[Bill Pierce]

Do you ever test your lenses? I don’t mean the kind of problem specific optical testing that goes on in a testing lab. I’m talking about taking pictures (which is already testing a camera and lens, not a lens) that can give you an idea of the general level of performance.. We tend to think of a high price and good internet reviews as a guarantee of top performance and a lesser price of a lens as an indication that it isn't as good. Actually, there’s enough variation in production of a lens and our needs that it makes sense to look at your specific lens. Beyond that, the real value of “home testing” is to know where a lens is at its best and worst.

Mirrorless cameras using the image sensor itself to determine focus have made looking at your lenses fairly straight forward compared to rangefinders and DLSRs which could produce focus variation from body to body. Still, there are precautions I would recommend, I would turn off image stabilization even though that means much testing will have to be done with the camera on a tripod. If you are going to test using autofocus rather than say a magnified manual focus, realize that you are testing the autofocus system, too. That’s not necessarily a bad thing if that’s the way you work.

Take some pictures at a medium distance. I take pictures of my wall of bookshelves, The various size of title type provide a good barometer of the rendering of fine detail. Take pictures at a distance. Street signs or a buildings exterior window frames are good sharpness barometers. If you can look down on a grassy field or any other large textured surface and then run the images through a program like FastRaw Viewer’s fine detail display you will see if there is any curvature of field at large apertures. Take pictures at all marked apertures, and, with a zoom, at the widest, longest and an in between focal length (yes, it’s a real pain).

The temptation is to take the test frames and view them at 100% on your computer screen. Actually, that is such overkill that I’m not sure it’s a meaningful examination. All of my testing is comparative. Any new lens is tested against an old lens that I know well and comes the closest to matching the new lens focal length. Then I look at a variety of more modest magnifications. This gives me a good view of the relative ability to capture fine detail and deal with vignetting, distortion and chromatic aberrations. Probably more important, some turn out to be soft wide open and are immediately put to work taking portraits. Some wide angles show a real lack of corner sharpness, and it makes me glad that I use them on the street and no one hires me to do architecture. And sadly, some are so sharp, so good that they make me feel inadequate.

Your thoughts?

 

[xayraa33]

Most photographers have field tested their lenses even before the advent of digital cameras...sometimes their favourite lenses turn out to be the less than optically perfect ones.

 

[peterm1]

Bill, I do not test in any formal or structured way. I am just not that interested in it.
The only thing that matters to me is whether I like a len's images. If I do, that is good enough.
I mainly use classic lenses and consequently I am only too aware of such len's shortcomings - potential and real. And I am aware that in some situations, such lenses have compromised performance. But in general, what I am looking for is a "character" that suits my purposes and my style of photography. As far as I am concerned, that is never determined by pixel peeping on a computer screen. I absolutely agree with you when you say that ".......the real value of “home testing” is to know where a lens is at its best and worst." But I kind of do this informally based on experiencing the lens rather than overt testing.

 

[Bill Blackwell]

As a Leica M shooter, all of my lenses are primes. It's never been a good idea for me to get into testing/comparing lenses because I get sucked into minute details which are ultimately unimportant.

However, when I do test, I don't have a specific method; I've tested by shooting bookshelves and scenes with equal frequency. When testing, I've always compared one lens against another in order to establish their differences. For example, color renditions might be different and one lens might be sharper than the other, but that's not to say the lesser isn't sharp at all. Generally speaking these differences can often only be realized by a direct comparison.

It's simply this: if I don't like the way a lens renders I get rid of it. If I like the way a lens renders - regardless of how it might compare to another - I keep it. And these factors rarely, if ever, require "testing".

Only some of the on-line testing done by the usual suspects holds much meaning [to me]. Others, IMHO, are completely worthless.

 

[Richard G]

I exhaustively tested my C Sonnar Zeiss ZM 50 to see exactly what my focus shift profile was. Curious about my 1970/80s 50 Summicron with which I've taken lots of my favourite photos I found it had been back focussing by a couple of cm close up for the thirty plus years I’d owned it. Never tested closely any others.

The Summaron M 28 f5.6 attracted me. The talk of its softness etc I fortunately ignored. In use it’s much sharper and more interesting than even Leica’s own advertising would suggest.

Back to the Mandler lenses. I bought a second hand 90 Elmarit M 2.8 with a slight wobble. It’s such a wonderful lens. Hard to take a bad photo with it. There is a bite it has. Lovely out of focus rendering too.

Finally one you’d have had, a 35 Summilux bought only in the last couple of years. Perfect hood, virtually new in box lens with original caps bought locally in Melbourne. Didn’t need it of course. Alarmingly soft at f1.4 and full of Leica ‘glow’. But it’s tiny and sharp and the right contrast stopped down, and very straight and I’ve wanted this one since the late ‘70s after reading your chapter in the Leica Manual on available light photography. Not a lens for wide open on digital at midday.

 

[Richard G]

The universal testing I add is seeing whether the hood is necessary. I’ve gone from always using a hood to only if necessary, just for the compactness.

 

[Doug]

I started to answer, no, I never test my lenses. But that's not so. True that I don't test for sharpness, but I do take shots intended to reveal geometric distortions.

 

[kbb]

After acquiring a lens, assuming I have time, I shoot several aperture series from a tripod at infinity and a few other distances. If I see no grotesque flaws, I move on to trying to make interesting photos. It can take a long time to fully ascertain the characteristics and usefulness of a given lens.

 

[Oscuro]

Caro Bill,

No. I am finding that I use some more than others in certain situations. It is an unconscious process. And the process is "testing" for I'm not sure what. Simplicity is several times the point. Any 28, any 35, any 300. Make picture. Wonder. Forget. Work. Look back and maybe analyze or maybe not. And then the phone rings and I go to the work.

Here is the thing: Fuji X100F has a beautiful 23mm lens which looks like 35mm in 135. It's fixed. Looks as good as anything the Leica had on the front. I cannot change. It has 50 and 75 digital zoom which is very, very good.

The GR III has a 28 (equiv) in front of APS-C and it looks very good. I cannot change. It has 35mm and 50mm digital zoom which is also very, very good.

Please to me quantify the difference between brushstrokes of the best realists against any photograph process. This old lady can make a good picture with 2 mp. (Just don't blow up too big, okay?) And even so, one can use Genuine Fractals to make it big or other systems.

So, lens testing? Not so much. I love a 50mm on the big ugly Nikon. But I am using it not very much. Always it has the zoom. Practical. Did I test it? No.

I tested the images. Without respect to the lenses. Only later did I find that I was using some lenses more than others. And with all the cameras, other things made me like to use the camera and lens combination than just the lens. Or just the camera. Or just the film. Or....

XOXO
Mme. O.

 

[Dogman]

In my backyard is a shed built on the back of a storage room. I keep the grills and smokers and various garden tools under it. But the walls are covered in colorful metal signs I've collected over the years along hand tools and random ephemera hanging on nails or tucked into spaces between signs. It's a glorious mess! It's also a great place to test lenses for sharpness, distortion and the way they reproduce color (even though I shoot B&W the vast majority of the time).

Every time I get a new/used lens or even a new-to-me camera I take it to The Wall. I don't do anything formal, I don't use a tripod or evenly light anything and I always use the standard color and large fine JPEG settings. I just lean my elbows on the cover of a Weber gas grill and shoot at all apertures from wide open to ƒ/8 or so. With zooms, I do this at all marked focal lengths. Then I take pictures of things in the backyard at various distances and finally I shoot some photos of the distant trees and buildings. Finally I download the files into Lightroom and inspect them after I add a standard amount of sharpening. This is necessary because I turn off or minimize all camera's sharpening.

I get a pretty good idea of how the lens is gonna perform by doing this. I've weeded out a new lens or two with obvious decentering, returned a couple of cameras due to poor consistency in exposures and found the limitations of various lenses that otherwise perform decently. It's down and dirty but it works for me.

 

[Ko.Fe.]

I'm not after exotics. Most of the lenses I'm dealing with were tested, reported.
Distortions, CA and such. Known already.
With non RF, AF lenses I'm not really checking anything rather than focus is working.
With RF lenses I'm checking for focus accuracy. I used to check for focus shift. This is how I got rid of all 50 Crons. 🙂
They are worse lenses for it comparing to ZM, VM.

I forgot who from greatest has mentioned it. Lens must be in possession and in regular use for year. To be able to understand this particular lens. Using it on film and on digital. I'm using this testing method now.

 

[Godfrey]

I test all the time. In fact, I consider any shooting I do when NOT specifically out for a planned shooting session to be a test. A lot of satisfying work has come out of testing like this, and I'm always aware of any developing problems with the gear, particularly older gear.

When I am out on a planned shooting session, I am not testing and just work at the photography of the moment. 🙂

G

 

[Doug A]

The first picture I take with any new lens is a portrait of my wife. Depending on the lens focal length the framing ranges from a tight head photo to waist up framing. The film is always a fast emulsion (Tri-X these days) in a high acutance developer (Rodinal for at least the last 20 years). The lens is always one stop closed from wide open and the focus is on her nearest eye. We have been doing this since we met in 1964, using heaven knows how many lenses. I make an 8 x 10 or thereabout print. If she likes it the lens is a keeper. If not, it eventually finds a new home.

The latest addition to my little LTM lens collection is a 35/2.5 Nikkor. I was concerned that she might not like it given its reputation as a very sharp lens. The verdict? "You know I usually don't like sharp lenses but this one is lovely."

 

[charjohncarter]

I only tripod a lens for a test when I've read a report that one of my lenses is inferior (that is on the ever correct internet). I usually get a results that is better than the IN tester.

For instance a common statement is that wide open it is soft. I shoot my lens at wide open (5 feet) focus on the center then refocus on the edge and take another shot. I guess they never have heard of focus plane curvature.

 

[jsrockit]

I’ve only tested after thinking I see a problem when out doing my regular photographs. This pretty much stopped after getting rid of my Leica M.

 

[raid]

I sometimes try certain lens settings to see if the resulting images look a certain way or not.
I found out that the Canon 50/1.4 ltm does not do well in dim light when using an old m 4/3 camera. The Panasonic Lux 25/1.4 goes very well in such conditions. The Canon does very well in daylight with aperture settings 5.6-11.

Many years ago I used to compare many lenses via portraits and stills. It was never a real test as we see done by some.

 

[willie_901]

Jim Kasson's blog The Last Word offers an objective approach to lens testing. The protocol is relatively simple and does not require specialized equipment or calculations.

o "The exposures can be made in a five minutes, once the target and camera are set up. Postprocessing for a set of images takes another ten minutes, using Lightroom."

o Kasson also discusses the problems associated with conventional lens testing methods.

 

[PKR]

I test everything.

I start indoors with sheets of news print on a wall, lit at 45°. The camera on a tripod. Then proceed outdoors.

I do my best to test several copies of the same lens when possible. I've found great variation between identical copies on rare instances. I keep the winners.

Once, years back, I took my gear into Nikon SF for yearly cleaning and adjustment. When picking up my gear a couple of weeks later, I was asked to wait as a factory tech wanted to speak with me. These folks are/were rotated on a two year term. The Japanese tech, in broken English, asked me where I bought my 24mm f2.8. He had the lens in his hand. I told him I wasn't sure without checking but likely, B&H. I asked why. He said, it was the sharpest example of that lens he'd ever tested. I got lucky.

I used a 55 micro as a standard 50. I must have tested 5 before settling on one, when replacing the lens. The newer 60 micro D, that I have, is superior to my 55.

I always figured with a small format, sharp lenses are important. The newer zooms are pretty good. But, testing is a must. I had a Nikkor 17-35 F2.8 that was really sharp at the 35 setting. At other focal lengths it was a dog. I didn't test others, the lens wasn't replaced.

I don't know if the issue is QC or maybe lenses get bounced around too much in shipping? But, there is a difference between examples of the same lens, sometimes very noticeable.

I find the testing process really boring. I try to break the time into 1 hour or so units. I keep extensive notes on the tests. I'm always looking for the sharpest f stop.