Reviewing Reviews

[Bill Pierce]

When a new camera arrives on the market, it’s interesting to read various camera reviews on the web. In many cases there’s a rush to get the report out. One of the great reviewers of the past was Geoffrey Crawley writing in the British Journal of Photography. It usually took Geoffrey about 6 months to put a camera through its paces, but the report was thorough, detailed, filled with accurate test measurements and, if such a thing is possible, objective. If a “test report” is going to appear on the web shortly after the camera is announced, either the tester is going to have to work relatively quickly and often superficially or have the kind of relationship with the manufacturer that gives them access to a prerelease model, an often cozy relationship which tends to soften public criticism.

And when it comes to image quality, often a major part of any report, often nothing is said about which image processing programs were used and at which of their settings. (And if your camera does not use the mainstream Bayer array, it can be very important.) And while that obviously effects “sharpness” so does focusing accuracy. Did the tester rely on the autofocus. magnified and wide open manual focus or a bracketed focus range? We should be told.

Just for fun, I tracked down every review I could on a recent release, a camera that won’t actually hit store shelves until early fall. Reviewers placed the camera in every category from awful to superb, but mostly in the “I liked it.” category. Of course, being a selfish and egotistical pig, I really don’t care about their feelings. I would like a really good test report to see if I might like it. Fortunately, there are sites that deliver that. My favorite is a pay site

http://www.reidreviews.com/

If you know of other sites that you have come to trust and respect, I would appreciate your listing them here so we could all benefit from them.

 

[Graham Line]

Poor Geoffrey wouldn't have a chance these days. Six months is longer than the life cycle of many products. DPreview can be good, but the reviews (not the previews or the I-opened-the-box-story, or the follow-up) are overstuffed with factory-supplied data, and the reader responses are generally useless.

 

[Brooklynguy]

Two reviewers/photographers whose opinions I respect:

-bythom/Thom Hogan
-Ming Thein

Most others are just shills for the camera companies, looking for referral fees, afraid to bite the hand that feeds them.

 

[shimokita]

Buy a new camera at ~USD 12,000 including one lens based on a review, I would venture that all a review can do is encourage / discourage the buyer to look at the camera hands-on. The focus is different... while a working photographer would surely value trusted comments... a consumer might value the current hot web personality.

The "Corvair" model of consumer protection doesn't really apply unless the camera has a tendency to explode on impact, but the essence of Nader-ism lives on. An "important" aspect of reviews seem to be the negatives, understanding the downside.

+1 for reidreviews... The smart money tends toward not to buying something only seen written on a T-shirt.

 

[YYV_146]

Two reviewers/photographers whose opinions I respect:

-bythom/Thom Hogan
-Ming Thein

Most others are just shills for the camera companies, looking for referral fees, afraid to bite the hand that feeds them.

Ming Thein lost credibility to me for regularly bashing adapters yet using an E-mount body for personal and professional work with adapted Otus lenses. I also don't buy his adapter focus plane argument because, if anything, digital CFV backs paired with film-era bodies fares worse.

I also feel that his assessment of median format digital is generally severely inflated. I shoot with a 645z. It's a nice sensor. Is it better than the best 135 sensors? Yes for printing, no for whatever 'transition' or 'tonality' is.

I get the fact that companies frown on mixing systems, but I still think that people should put their mouth where their money is...but I digress. I still read his blog, but with a grain of salt.

Sean Reid is very good. Diglloyd mostly reasonable. Kai can be brutally honest when he's not fooling around. DPR's recent reviews are getting better and better - it's a shame they don't post more about Leica.

 

[Ko.Fe.]

I'll just follow title of this thread.
Few days ago I googled Leica M9 in 2016. One blog came. I read it and get funny feeling after it. Like it is written not by the person to whom it actually assigned to...

As support and ex-sales person who deal with electronics, I have stopped buying new digital gear. For my personal needs and confidence I'm buying something at least three years old. Because it will have enough reviews from real users and photogs, not just from Huff and DPreview.
I'm not so rich&stupid to buy new, just released gear.

 

[icebear]

Reviews can only help in your pre-screening. Get some facts to determine, if there is a no-go factor in the potential camera.
After the screening still left the contender with a positive "yes maybe" ... you go and take the camera for a test ride, either from a brick and mortar store and you buy it there, if you like it. Or you rent it when there is no other way to test it. Yes, this can mean that you don't have the latest and greatest after the first two reviews are out but you are getting a camera, not a cell phone.😀

I read quite a few reviews before signing up for a Leica Monochrom workshop. The minute I opened those files back home on my PC my decision was done I called my pusher😉 the next day.

 

[Roger Hicks]

Well, I have reviewed cameras and lenses professionally, so I know quite a lot about this subject from the other side of the counter. Here are some guidelines for readers:

1 Only a fool would rely on a single review(er)

2 Factory data is readily available from the manufacturers and from bad reviews. Flowing directly from (1) above, there's little point in reciting it all in detail

3 Some reviewers are a lot more honest and experienced than others. It is often quite easy to tell the good ones from the bad ones

4 Niche products (Leicas, Nikon Df, Zeiss lenses) are often handed to specialist reviewers rather than done by staff writers

5 "Happy-snap" cameras require a different reviewing mind-set from niche products. If you're used to full manual control it can be difficult and frustrating to review cameras designed for people who don't really care about photography

6 More than once I have declined to review dull, "me-too" cameras, or cameras about which I could find nothing good to say. I am sure I am not the only reviewer to have done this

7 Read a review to get a "feel" for the camera. If you can't get how the reviewer reacted to it, then it was probably because there was nothing remarkably good or bad about the camera, or the reviewer is incompetent

8 Read between the lines. Few reviewers are likely to say "the video mode is a complete waste of time" but they may say "the video is of limited use"

9 As a general but far from absolute rule you'll get better reviews (= more comprehensive, better informed, by better reviewers) if you pay for them in print rather than look for them free on line

10 The main reason to read on-line reviews is if you are already acquainted with the reviewer's style, prejudices, photographic interests and preconceptions. Flailing around to try to get different viewpoints on line is generally a waste of time.

Cheers,

R.

 

[Dogman]

One of the benefits of being poor (is there such a thing...really?) is that you can't afford the "New Big Thing". That is: A new camera comes to market at $1500 initial price and has everything I want. Except I don't have enough money to buy it so I pass. After 18 months, another "New Big Thing" comes along and that first model ain't so New or Big anymore. But that first model doesn't cost $1500 either. Prices start to drop, slowly at first and then the bottom falls out as it is near discontinuation. Now you can get it for $500. It still has everything I want and now I can afford it. If I still want it.

That camera that had everything I could want has now been around long enough for people to have actually used it. And, this being the age where everyone with an ego has a blog, there is now a whole wealth of late-forming opinions available online to review and compare. By this time, I probably know what features are important to me and which features I will never use so I can judge the reviews more reliably. It's also interesting to see the photos these reviewer/bloggers display on their websites. While all of them speak with Great Authority, some of them can't take pictures for Jack Schnitt. Why would I put any faith in their reviews?

(I recently bought a lot of new but recently discontinued equipment. I've spent a lot of time online reading older reviews. The rush to be first with a review is obviously more important than reviewers using the item under review. Many just copy other reviews. Makes one wonder if some of them have ever actually handled the items they review.)

 

[jsrockit]

I would venture that all a review can do is encourage / discourage the buyer to look at the camera hands-on.

Agreed... even from a proper review.

 

[Roger Hicks]

. . . Makes one wonder if some of them have ever actually handled the items they review.

Some don't. I met one who boasted -- boasted! -- that for reviews of low end products he didn't even bother to open the box.

Cheers,

R.

 

[BlackXList]

There are definitely reviewers who I enjoy and those I don't.

I regularly check DPReview, they're not without problems, but the fact that all the camera reviews are structured the same, creates an easy way to compare. I often skip to the conclusion to read the pros and cons, and then decide if the whole thing is worth a read, one person's pros and cons may be irrelevant to another.

as far as Ming is concerned, I really like reading his reviews on stuff like the Ricoh GR, older film compacts, Sigma DP etc, the Digital MF gear I start rolling my eyes a bit too much at his evangelism for his printing method or some other item which he's decided is the most important thing ever.

Kai, I feel he occasionally makes a good point, but he often misses massive points or misreports in order to try and be funny, I do appreciate that he actually uses the equipment during the review though, because that's where equipment's foibles actually show themselves.

I really like The Camera Store reviewers, (both of them) they're not afraid to be critical of something they think is bad, even though it might cost them sales. I appreciate the honesty, and as a Canon user, this is my favourite camera review ever:

https://www.youtube.com/watch?v=baHqjUrykTU&index=5&list=PLiXUulBBkNvFE5T82WRAStBGDlxkFKFRF


I have to admit I do check Mr Rockwell's lens reviews, taken with a hefty serving of salt, there's some decent information in there sometimes, and I know he hasn't copied and pasted the press release.

 

[willie_901]

..
I regularly check DPReview, they're not without problems, but the fact that all the camera reviews are structured the same, creates an easy way to compare. I often skip to the conclusion to read the pros and cons, and then decide if the whole thing is worth a read, one person's pros and cons may be irrelevant to another.

...

I too rely on DPReview more than any other source.

I don't find the blogger useful, although I do pay to Thom Hogan's reviews.

In my view pay photography review sites are a waste of money because these sources don't benefit from peer review or editorial supervision.