hofrench@mac.co
Established
Several months after the uproar over the initial burst of essentially uncritical reviews of the Leica M8, despite what we all know by now to be serious design and Q/C issues with this $5000 camera, Michael Reichmann of Luminous Landscape has revisited the issue of the responsibilities of product reviewers.
His new post can be read here: http://www.luminous-landscape.com/essays/reviewers.shtml
I sincerely respect Michael and have benefited from regularly following his web site.
Having said that, I found the Leica review debacle to be a serious eye opener, and wonder what others make of Michael's explanations?
Call me naive, but what troubled me most was to learn that reviewers consult with manufacturers about whether or how to publicly discuss problems with their products. In this instance, Leica's advice was not to mention some of the problems which we've all become familiar with in the M8, with the result that many buyers went into a very weighty purchasing decision blindly.
Now, Michael says flatly that it is "nonsense" to hold product reviewers to the same standards, ethical or otherwise, as journalists.
Since there is no opportunity for feedback on Michael's site, at least not on the main pages, I thought RF would be a good place to open a discussion on this subject.
His new post can be read here: http://www.luminous-landscape.com/essays/reviewers.shtml
I sincerely respect Michael and have benefited from regularly following his web site.
Having said that, I found the Leica review debacle to be a serious eye opener, and wonder what others make of Michael's explanations?
Call me naive, but what troubled me most was to learn that reviewers consult with manufacturers about whether or how to publicly discuss problems with their products. In this instance, Leica's advice was not to mention some of the problems which we've all become familiar with in the M8, with the result that many buyers went into a very weighty purchasing decision blindly.
Now, Michael says flatly that it is "nonsense" to hold product reviewers to the same standards, ethical or otherwise, as journalists.
Since there is no opportunity for feedback on Michael's site, at least not on the main pages, I thought RF would be a good place to open a discussion on this subject.
Last edited:
V
varjag
Guest
I guess the lesson here is we can't rely on opinion from just one source, no matter how credible and popular. And not just with camera reviews 
Paul T.
Veteran
Didn't the 'nonsense' comment refer to the suggestion that he shouldn't check anomalies with manufacturers?
I think in this case you get what you pay for. Reichmann's service is free, and he does a very good job. There is a huge amount of information on the web, but as with wikipedia and other sites, you can't rely on it exclusively. I think he made a judgement call and happened to be wrong, but I would guess he's learned from his mistake. As have Leica.
For all the sound and fury, the M8 launch has obviously gone well, Leica acknowledged the problems with the M8 head on and by offering free IR filters have pretty much got back to the position they should have been in when they launched the camera - ie acknowledged the camera's IR sensitivity and explained why certain design decisions were made.
Ironically, it would seem that used values of the M8 - going by one recent sale on rff - seem proportionately higher than those of the M7, once it had been out a few months. Which is perhaps the ultimate vindication of the camera, given how how much some Leica owners obsess about resale values!
I think in this case you get what you pay for. Reichmann's service is free, and he does a very good job. There is a huge amount of information on the web, but as with wikipedia and other sites, you can't rely on it exclusively. I think he made a judgement call and happened to be wrong, but I would guess he's learned from his mistake. As have Leica.
For all the sound and fury, the M8 launch has obviously gone well, Leica acknowledged the problems with the M8 head on and by offering free IR filters have pretty much got back to the position they should have been in when they launched the camera - ie acknowledged the camera's IR sensitivity and explained why certain design decisions were made.
Ironically, it would seem that used values of the M8 - going by one recent sale on rff - seem proportionately higher than those of the M7, once it had been out a few months. Which is perhaps the ultimate vindication of the camera, given how how much some Leica owners obsess about resale values!
pfogle
Well-known
It's a tricky one, this. While I can see that there are no objective constraints on reviewers to be straight, usually we expect to see a disclaimer to say that the reviewer is not beholden to the manufacturer.
Now, usually that would mean the reviewer is not being paid to write a positive slant, but in my view, withholding information at the request of the manufacturer IS being beholden, and makes the claim to be independent (if, indeed, it was ever made in this case) to be disingenuous at the least....
... which doesn't weaken others' arguments that you shouldn't rely on only one source - it's just that LL is now one of the sources I'd take with a pinch of healthy skepticism.
Now, usually that would mean the reviewer is not being paid to write a positive slant, but in my view, withholding information at the request of the manufacturer IS being beholden, and makes the claim to be independent (if, indeed, it was ever made in this case) to be disingenuous at the least....
... which doesn't weaken others' arguments that you shouldn't rely on only one source - it's just that LL is now one of the sources I'd take with a pinch of healthy skepticism.
Last edited:
steamer
Well-known
He's having fun and his site is free, but in my book doing checking with the manufacturer and not mentioning any bad details means he has zero credibility.
I'm sure the M-8 is the most amazing 5,000 dollar rangefinder camera ever made and all but he needs to post a disclaimer on his reviews to let readers know he is in bed with the maker of the cameras he reviews.
I'm sure the M-8 is the most amazing 5,000 dollar rangefinder camera ever made and all but he needs to post a disclaimer on his reviews to let readers know he is in bed with the maker of the cameras he reviews.
R
RML
Guest
I hope MR takes to heart what he wrote in the first paragraph:
"That was my mistake. With hindsight, I should have said that I did see some anomalies, don't understand what they were, and neither did Leica, but when I (we) did there would be a follow-up."
The rest of the article seems a bit of finding excuses instead of an apology.
For me, it's no deal breaker. I didn't read his reviews before and never relied on them for a purchase anyway.
"That was my mistake. With hindsight, I should have said that I did see some anomalies, don't understand what they were, and neither did Leica, but when I (we) did there would be a follow-up."
The rest of the article seems a bit of finding excuses instead of an apology.
For me, it's no deal breaker. I didn't read his reviews before and never relied on them for a purchase anyway.
hofrench@mac.co
Established
Phil Fogle and "Steamer" have hit the nail on the head as far as I am concerned.
Allowing the manufacturer any say over the content or judgments in a review in my book does a disservice to the reader and is a pretty serious disqualification for consideration as an independent source of information.
Michael deserves praise for his honesty in acknowledging the nature of his communications with Leica. A lot of the consumer oriented commercial press operates on a distinctly less honest basis.
He dodges, however, what seems to me to be the obvious conclusion that serving the manufacturer and serving the reader are by definition distinct and pretty much incompatible approaches to this business.
On the M8 itself, I am largely happy with the camera, even though I'll never be fully reconciled with the compromises I have to make in using it, some of which I had only dimly imagined at the time of purchase, thanks in part to the gushing and indulgent reviews.
I had no idea, when buying the $5,000 camera, that using filters would be the best workaround I could hope for for serious design flaws or compromises. I've still not received my filters from Leica, either.
Fortunately, my work is heavily oriented toward black and white.
Allowing the manufacturer any say over the content or judgments in a review in my book does a disservice to the reader and is a pretty serious disqualification for consideration as an independent source of information.
Michael deserves praise for his honesty in acknowledging the nature of his communications with Leica. A lot of the consumer oriented commercial press operates on a distinctly less honest basis.
He dodges, however, what seems to me to be the obvious conclusion that serving the manufacturer and serving the reader are by definition distinct and pretty much incompatible approaches to this business.
On the M8 itself, I am largely happy with the camera, even though I'll never be fully reconciled with the compromises I have to make in using it, some of which I had only dimly imagined at the time of purchase, thanks in part to the gushing and indulgent reviews.
I had no idea, when buying the $5,000 camera, that using filters would be the best workaround I could hope for for serious design flaws or compromises. I've still not received my filters from Leica, either.
Fortunately, my work is heavily oriented toward black and white.
Last edited:
nrj
Member
MR's piece makes some pertinent observations about the process of reviewing and in particular the reviewers relationship to the manufacturer. I don't expect a reviewer to be free from their own critical standpoint or not to discuss their observations with a manufacturer or peers. However his piece does seem to downplay the most important point of a review: to inform readers before they make purchase decisions. The review therefore should be as informative as possible to be useful, deliberately covering up flaws to please the manufacturer means the credibility of the review is shot and its impossible then to tell what else has been massaged in the process.
An interesting point for me is MR's suggestion that we learn over time about a reviewers opinions and biases so we can use this to inform our assessment of a review. He uses the analogy of a film reviewer where you learn their taste matches your own because you can go see the recommended films and over time come to trust the reviewers judgement. With cameras I think this analogy breaks down to the point where its unhelpful. Cameras cost a lot of money and most people cannot afford to buy the latest to check whether the reviewer's taste matches theirs over time. I think a camera reviewer has a far greater responsibility to acknowledge their biases and explain their critical perspective in any review to help the reader judge whether what is said should carry weight for them. If this was done more often and more thoroughly then reading several such reviews from different reviewers would give the reader more insight and information on which to base a decision about whether the camera (or lens etc) was right for them.
Nik
An interesting point for me is MR's suggestion that we learn over time about a reviewers opinions and biases so we can use this to inform our assessment of a review. He uses the analogy of a film reviewer where you learn their taste matches your own because you can go see the recommended films and over time come to trust the reviewers judgement. With cameras I think this analogy breaks down to the point where its unhelpful. Cameras cost a lot of money and most people cannot afford to buy the latest to check whether the reviewer's taste matches theirs over time. I think a camera reviewer has a far greater responsibility to acknowledge their biases and explain their critical perspective in any review to help the reader judge whether what is said should carry weight for them. If this was done more often and more thoroughly then reading several such reviews from different reviewers would give the reader more insight and information on which to base a decision about whether the camera (or lens etc) was right for them.
Nik
ywenz
Veteran
Reichman is a sub par landscape photographer at best so I certainly don't go there for the photography... Now that his reviews are in the gutter, what other reason is there to go to his site? I'm done with LL
Ben Z
Veteran
I take published reviews for entertainment value, especially of pre-production items. I wouldn't buy a car without driving it, or a camera without shooting it. That's the Purchaser's Responsibility
jaapv
RFF Sponsoring Member.
ywenz said:
Isn't that a bit harsh?
K
Ken Tanaka
Guest
ywenz said:
I wonder if you realize how revealing such a comment is regarding the quality of your character.
jlw
Rangefinder camera pedant
hofrench@mac.co said:
I haven't yet read the original article, and don't want to go too far in responding to a quote out of context.
But as someone with a journalism degree and upwards of three decades of professional publication experience, I can tell you that my editors have always expected me to adhere to the same ethical and professional standards in a review-type article as in a news story (and I've written a lot of both.)
Those standards don't necessarily preclude contacting the story subject -- whether it's giving the mayor a chance to respond to his opponent's accusations, or asking the director whether he really intended to have the diva fall into the orchestra pit during her big aria in Act II.
But that information is supposed to be used to expand the content of the story, not suppress it. For example, it would be perfectly legitimate to write in a product review: "I had a lot of problems with feature X, but Scumco's product manager assured me that the engineers are aware of them and that a fix will be in place by the time the product goes on sale." This may not be definitive, but at least it lets readers know there's a potential issue and lets them make up their own minds about how seriously to take it.
If the argument is that it's nonsense to hold online journalists, especially self-employed ones, to the same standards as those who get paid salaries by major media organizations, I don't necessarily argue with that either. But the point is that the reader needs to be fully informed about the situation, which is why professional media organizations almost always have "full disclosure" and conflict-of-interest policies that you can find if you dig hard enough.
These policies give the reader a chance to decide for him/herself what amount of "windage" might need to be cranked into a review. For example, I subscribe to a couple of vintage-sports-car magazines; one has a policy of allowing vendors to "sponsor" (i.e., pay for the parts and work on) the magazine's project cars, while the other's parent company adheres to a policy that everything has to be paid for at fair market value, period. Naturally, I think the second magazine's policy is more confidence-inspiring... but I still find value in the first magazine's stories, although I keep in mind when reading its product evaluations that the editors and vendors are scratching each other's backs to some extent.
It would be nice if self-employed online journalists likewise spelled out their policies on how closely they cooperate with news sources -- but in the absence of such a spelling-out, the only sensible thing (as someone else has already suggested) is to treat their writings as if they were labeled "for entertainment only."
hofrench@mac.co
Established
Perhaps I should have made clearer what my problem is with MR's stance on meeting the standards of journalism.
The author did not go back to the manufacturer in order to allow it to defend itself against some kind of accusation, which in good journalism is, as you note, a standard practice.
MR, by his own admission, sought and took Leica's advice about how to handle the issue of flaws in its product in his review. Furthermore, I get the sense from things Michael has written that this is routine practice with Leica and perhaps with certain other manufacturers.
I would take this one step further by responding to Michael's movie reviewer analogy, which others in this thread have already criticized with great pertinence. Let me just ask what movie reviewer worth his salt would call the studio after seeing a film that was a clunker, or had serious plot problems, and ask the studio for advice about how to write about the film?
My very strong feeling, and it is based in part on a lifelong career in journalism, is that this is a disservice to the reader, and I am not impressed by the distinction some might make in an instance like this between an online publication like Luminous Landscape and more mainstream journalism.
Some have written dismissively of this whole matter saying things like "you get what you pay for." I think they forget, or do no sufficiently appreciate that MR seems to make a substantial part of his living from his site, and whether you are reading an essay about technique, downloading a tutorial about Lightroom, or reading his appraisal of a new camera, MR's stock in trade is his supposed expertise and implied independence.
Not to go on too long here, but there are other un-addressed questions that relate to the issue of the reviewers' obligations to the manufacturer.
One would like to know, for example, to what extent, if any, Leica or others award or punish online reviewers in any way based on their willingness to play their game, meaning allowing them to screen reviews prior to publication, and to otherwise shape coverage of their products?
Finally, let me repeat, that although I don't know Michael personally, I have gained in many ways from following his site. I wish him and it all the best. I also hope, though, that he will reconsider some of these issues in order to make LL an even better and more credible source of information for all of us.
The author did not go back to the manufacturer in order to allow it to defend itself against some kind of accusation, which in good journalism is, as you note, a standard practice.
MR, by his own admission, sought and took Leica's advice about how to handle the issue of flaws in its product in his review. Furthermore, I get the sense from things Michael has written that this is routine practice with Leica and perhaps with certain other manufacturers.
I would take this one step further by responding to Michael's movie reviewer analogy, which others in this thread have already criticized with great pertinence. Let me just ask what movie reviewer worth his salt would call the studio after seeing a film that was a clunker, or had serious plot problems, and ask the studio for advice about how to write about the film?
My very strong feeling, and it is based in part on a lifelong career in journalism, is that this is a disservice to the reader, and I am not impressed by the distinction some might make in an instance like this between an online publication like Luminous Landscape and more mainstream journalism.
Some have written dismissively of this whole matter saying things like "you get what you pay for." I think they forget, or do no sufficiently appreciate that MR seems to make a substantial part of his living from his site, and whether you are reading an essay about technique, downloading a tutorial about Lightroom, or reading his appraisal of a new camera, MR's stock in trade is his supposed expertise and implied independence.
Not to go on too long here, but there are other un-addressed questions that relate to the issue of the reviewers' obligations to the manufacturer.
One would like to know, for example, to what extent, if any, Leica or others award or punish online reviewers in any way based on their willingness to play their game, meaning allowing them to screen reviews prior to publication, and to otherwise shape coverage of their products?
Finally, let me repeat, that although I don't know Michael personally, I have gained in many ways from following his site. I wish him and it all the best. I also hope, though, that he will reconsider some of these issues in order to make LL an even better and more credible source of information for all of us.
Last edited:
Steve Bellayr
Veteran
This is not new. For decades magazines & newspapers refrain from giving bad reviews to the products that advertise with them. This started way back in the 1970's. A key word in any review if "fun." It's not enjoyable, or effective, or worthwhile or efficient but "fun." A "fun" movie, a "fun" car to drive, etc. "Fun" is a euphemism for trash. When we buy a magazine or read a newspaper we should expect the truth not the copy that the company puts out. I did not read his article and do not know whether or not he wrote the truth. I read the reviews on RFF which were realistic from honest users who were not paid to shoot with the camera.
Trius
Waiting on Maitani
Paul T. said:
This doesn't make any sense to me at all. Leica have always stood for the pinnacle of quality, the quest for perfection. They have striven to manufacture a superior product and charged accordingly. So how does this standard not carry over to the ethics of how they handle a product launch review? It boggles my mind. It's a flat-out admission that they had problems, knew they had problems, and didn't want them revealed. They used Reichman (and presumably others), and Reichman (and presumably others) willingly allowed themselves to be used. Not acceptable.
As far as getting back to where they should have been by way of the IR filters ... hell, they knew about the problem before launch but waited until there was an uproar? C'mon, how is that top shelf?
I am not denying the very real nature of the design and engineering issues. But it is clear the marketing department was driving Leica's ship, not the engineering section. And the real photographers who have made Leica what it is buy on engineering and manufacturing, not marketing. It's the Red Dot lapel pin crowd how following the marketing hype.
willie_901
Veteran
Reichmann is correct. He certainly is not a journalist.
Of course it is silly to hold him to the same standards as a professional journalist.
Journalists have editors who judge and oversee their work. Journalists are mentored (and protected by) my more experienced peers who have made, or observed others who have made, mistakes in judgment and thoroughness.
We – the readers – should not confuse the LL's articles with work produced by journalists whose primary goal is, in principle, to inform the reader.
Reichmann actually is an advertising copy writer. He does not write advertising copy in the traditional sense, i.e. to sell products for clients. The primary reason Reichmann writes to attract visitors to the LL web site. All Reichmann's free content (including reviews) is published to attract readers. Some of these readers will eventually purchase merchandise sold on LL. While Reichmann and his writers work hard to make LL's free content interesting and useful, it's purpose is not solely to inform and entertain. It's primary purpose is to increase the number of people who visit, and revisit, the LL site. If you don't visit the LL site, it's impossible to buy LL's merchandise. The more photographers who visit, the more likely LL will sell it's merchandise. This is not journalism. But it is a good business model.
At least Reichmann does not charge to read his non-journalistic, self-published writings. For this reason I rank him above those who charge to read their self-published, non-peer reviewed, articles.
Ironically Reichmann's M8 review contained a color-space map that described how the the M8 reproduces color. While I was too inexperienced to realize it at the time, the data Reichmann published told the M8's IR contamination story plain as day.
I respect Reichmann for knowing enough to publish credible data. At least Reichmann published a professional test and was aware from the beginning that IR contamination is an issue for digital image sensors.
Of course it is silly to hold him to the same standards as a professional journalist.
Journalists have editors who judge and oversee their work. Journalists are mentored (and protected by) my more experienced peers who have made, or observed others who have made, mistakes in judgment and thoroughness.
We – the readers – should not confuse the LL's articles with work produced by journalists whose primary goal is, in principle, to inform the reader.
Reichmann actually is an advertising copy writer. He does not write advertising copy in the traditional sense, i.e. to sell products for clients. The primary reason Reichmann writes to attract visitors to the LL web site. All Reichmann's free content (including reviews) is published to attract readers. Some of these readers will eventually purchase merchandise sold on LL. While Reichmann and his writers work hard to make LL's free content interesting and useful, it's purpose is not solely to inform and entertain. It's primary purpose is to increase the number of people who visit, and revisit, the LL site. If you don't visit the LL site, it's impossible to buy LL's merchandise. The more photographers who visit, the more likely LL will sell it's merchandise. This is not journalism. But it is a good business model.
At least Reichmann does not charge to read his non-journalistic, self-published writings. For this reason I rank him above those who charge to read their self-published, non-peer reviewed, articles.
Ironically Reichmann's M8 review contained a color-space map that described how the the M8 reproduces color. While I was too inexperienced to realize it at the time, the data Reichmann published told the M8's IR contamination story plain as day.
I respect Reichmann for knowing enough to publish credible data. At least Reichmann published a professional test and was aware from the beginning that IR contamination is an issue for digital image sensors.
Last edited:
ywenz
Veteran
Ken Tanaka said:
Straight shooter of the finest quality? Yes :angel:
willie_901
Veteran
jaapv said:Isn't that a bit harsh?
Sometimes the truth is harsh.
jaapv
RFF Sponsoring Member.
HTML:
I'll elaborate:
I tend to agree with those that find that MR -but not only MR!!- got sucked into what I regard as Leica's marketing blunder. And whilst it is appreciated that he more or less apologizes now, the argument :"you guys should have known I like Leicas" is not one of the strongest. And question marks regarding the journalistic validity of this kind of review a justified too - some more disclosure should have been made. But to call him a "sub-par photographer" -surely a matter of personal taste- in this context and to deny the many high-level and useful articles on his site is at least unjust and bad manners to put it mildly in my book.
Last edited: