This is a bit of a tenuious link, but I'm beginning to think that some photographers work is only appreciated by other photographers, street photos for me are a classic example. I myself have found this as others have, with Winogrands work.
Your average man in the street's idea of a good photo is usually (round here anyway) totally different to that of a "photographer" who may look past the initial image for a deeper meaning.
When we look at a famous photographer's work and initially dont like it, are we acting like Joe Public, and then, later, either trying to convince / force ourselves that we like it because they are famous.