Online Lens Reviews -- Why They're Usually Useless: Use Case and Weighting

NickTrop

NickTrop

Veteran
Local time
3:04 AM
Joined
Feb 19, 2006
Messages
3,077
Like most folks into photography I obsess on gear. I agonize over each purchase, even relatively small ones. I pour over all the review sites -- Rockwell, OpticalLimits!, DxO etc. and several YouTube channels comparing and contrasting. This is especially true with lenses since I more frequently purchase lenses than camera bodies. And here's what I've concluded:

-- They're mostly useless. And they're flawed in two areas:

1. Use Case
They don't take into consideration HOW a lens is to be used and by whom? So, for example, I own a Tokina 17-35/4. It gets mediocre reviews in the "online press". The main knock, pairing it down, is "softness in the corners wide open especially at 35mm..." On MTF charts it achieves well into "excellent" resolution range by F8, corners and edges not far behind. By F11 there is excellent sharpness across the image plane with a slight drop-off in the center (but still excellent resolution).

This lens is a wide-angle zoom. I have used it exclusively outdoors. I shoot this lens almost always at its widest or near widest angle of view. I amost never shoot below f5.6. This "use case" for such a lens for architecture, landscape, and interiors on a tripod and/or using an external flash. That is, this lens is used when there is ample light to shoot at proper working apertures for its intended use.

Its "use case" is different, say, from a 35mm prime, or a fast "nifty fifty". Undervalued in this lens is that it has the lowest amount of barrel distortion of any ultrawide zoom I'm aware of. For such a lens, an ultrawide, this is important to users -- especially a 3rd-party lens where the camera's processing engine may not correct for it.

Finally, this lens is significantly less expensive than many/most/all other options. Yet, it does not skimp on build quality.

So, while the online press rates this as "mediocre" due to performance "in the extremes" for a lens in this category, the specs for its intended use/use case are not properly "weighted" and the metrics that diminish this lens in their view are less important than lenses designed for other types of shooting. Yet, online review sites completely ignore this (for the most part) and rate/review lenses "one size fits all" comparing each on the same criteria, equally weighted, as if a "lens is a lens" and there is no difference in use case -- they're all used for taking pictures. (Another issue is they tend to ignore price and value. Is it really worth it to pay $1,000 more for a lens to achieve marginal performance gains that are too often invisible? But that's a topic for another post.)


This is just one example -- I could go on with many.

2. Weighted (Not Straight) Averages Based on Use Case

Since I have limited time and have already written a (par for the course) tome, tied into use case is "weighting". We all know of weighted averages, right?

So, corner and edge sharpness might be very highly rated if you're shooting art, text, or other technical types of photography. However, for a 50mm aren't corners and edges blurred when shooting candids in natural light and subject is invariably at/near the center of the frame? Isn't the size and weight (not average) of a lens more important if the use case for this lens in natural light candids so as to be able to carry the lens easily and not intimidate subjects?

Thus, from my perspective, many/most/all online reviews whether text or video are largely useless because they seem to weight all categories equally and disregard use case.

FIN
 
When I read a camera or lens review online, I usually just go to the "Pros" and "Cons" section at the end of the review. I consider whether the bad points and good points have any relevance to my uses. For instance, if it has great video features I just think "So what" and ignore it since I don't do video. If it's ISO at 100,000 is not so good that's another "So what" to me.

Lenstip.com and Opticallimits.com do some pretty fair lens reviews in my opinion. But I ignore their final evaluations because a lot of the time they are based on how much the lens costs. Price is not as important as performance to me. I already know how much I can spend before I ever start thinking about a specific lens...I just wanna know how it sharp it is, how much distortion is has, etc.
 
I too look at review sites until I have a dizzy head from it. I seem to have more trust in looking for user reviews on various forums like RFF and others. These reviews seem to be more real world type of information for making a decision.
 
For me, I usually go by other folks' images on 'net and forums (but I'm usually buying an older/vintage lenses).
 
I agree.

What I based my buying of lenses is what I wanted to accomplish. My favorite lens that I used was a Canon 24-70 f2.8. It worked for about 95% of photographs I made.
 
Lenstests aren't liable to a standardized test method. Lenstests are generally not scientific and replicable. So beware of lenstests - except some case-by-case exdeptions.

Just my 2 cents
 
There's other oddities in lens tests; they don't seem to know that 12x8's are usually printed as 10x8's and that knocks off the edges. Secondly, someone who prints 4x6 and nothing else won't even notice the difference. The tests seem designed for the 24"x36" and bigger crowd...


Regards, David
 
One ingredient I always considered, talking to others who were in the TCPPA, was reliability. When I was on the job, the tools had to work. No exceptions. Excuses don’t matter to a client.

And a very important event can never be replicated with the majority of people. Only Hollywood actors can do that. What I mean by that statement are the emotions, the spontaneity of them on display, showing in all parts of the human form, especially on the faces during an event such as a wedding, sporting event, to name two.
 
For my film work, I've been using older lenses a lot recently. Some, like the 50mm Summar and 35mm Elmar, don't do well with reviewers unless they use them to shoot black and white and then make prints. Those who do like the older lenses. Those who obsess over the reviews lenses generate rather than the photographs they help produce probably won't get it. Lens reviews are fun to read as long as you understand most are there to get your attention rather than provide information about how they help a photographer in the real world.
Thus endeth the rant.
 
presspass said:
For my film work, I've been using older lenses a lot recently. Some, like the 50mm Summar
Click to expand...

I did an entire thread on ther Summar here many moons ago ("Summar of Love"). That lens would get ravaged if judged by today's standards. But my it will render an image like no other under the right conditions and settings. You can't Photoshop it, you can't get it with any other lens.
 
Requin said:
Lenstests aren't liable to a standardized test method. Lenstests are generally not scientific and replicable. So beware of lenstests - except some case-by-case exdeptions.

Just my 2 cents
Click to expand...

I don't know if I agree with this. What are the critria for testing a lens in a particular focal length given the use case for that focal length? You can list them. Most do. Then how important is each criteria given the use for that particular type of lens? Weight them. Come up with a score.

For example. To -me- in a fast 50, portability is important. Widest aperture is not that important. I don't buy a 1.4 to shoot at 1.4 all the time. The DoF is too narrow. I buy a 1.4 to shoot decent at 2.0, 2.8. So performance at those apertures is important. I'm not overly concerned with edge and corner sharpness. Or even vignetting or purple fringing, which is easily removed.

Currently, the Sigma Art 50/1.4 is the toast of the photographic town mainly due to sharpness at 1.4. Things even out as you stop down. But the thing is big and heavy by 50mm standards. I, given the "use case" for a standard prime in this range would weight size and weight over performance wide open because I'm always going to "feel the weight" and the lens might be too obtrusive for candids. I'm hardly ever going to shoot at 1.4 because if issues keeping things in focus -- especially moving subjects.

Also, I rate t-stop value very high on the list. No sense springing for a 1.4 if only 1.8 is coming out the rear. This is something that is completely ignored by every site except DxO (because it's hard to test).

But the point is, that although the criteria for evaluating a lens will be largely the same from lens to lens, how each criteria is weighted should vary based on use case for each focal length. The Tokina I cited might be judged on the same criteria as the 50/1.4 but each criteria might be weighted differently when averaged (so long at the end it adds to 100).
 
One thing I look for in a lens review (or from a lens reviewer) is consistency. Not so much how they rate the lens, but how they evaluate it based on previous lens reviews. Very rarely do I look at example photos, but take note of the type of camera used (e.g. film vs digital).

Two of my favorite Nikon MF lenses for film are the Nikkor-H 2.8cm f/3.5 and the Nikkor 50mm f/1.2 AI-s. Both used on a F3P... if I had followed the review conclusions I might not have purchased these lenses.
 
I read reviews and look at image samples to get an idea of what a lens is capable of. Just more data when considering a purchase. Whether or not I keep (or keep using, for those of us who tend to accumulate gear) a lens is borne out through my own actual shooting and testing. So, not useless, but not the be all end all.
 
ptpdprinter said:
If you are regularly shooting at f8 almost any lens will do You don’t need a lens review.
Click to expand...

Right. However, not every lens, based on use case, should be evaluated on its performance at wide apertures or such metrics should not be weighted as heavily in the overall evaluation of a lens. An ultrawide zoom is not a lens you go sneaking around bars at night, typically. This kind of lens is used mainly for landscapes, architecture, and interiors on a tripod with a flash. where light is not an issue
 
You must log in or register to reply here.
Back
Top Bottom