People who say that, well, I somewhat feel sad for them. The arts, photography, etc., is about one's experience in life. No one can duplicate that. It's how you feel about yourself. It's how you feel about your place in the world.
I agree with you, and I think this is why so much 'street' photography we see is so emotionally and intellectually empty. There are a lot of hobbyist photographers, generally men who buy a Leica and feel that they have to be the next Cartier-Bresson.
The problem is, they're not HCB, and more importantly, they feel no connection to or interest in the subject matter. They go snap random photos of people on the street, but they feel no connection to the people they 'shoot', and it shows in their photos.
The people who were truly great at street photography were people who were genuinely interested in the people and places they photographed. That is the key, not just in street photography, but ANY genre of photography.
Look at my work; it is about the places I have lived. I have added, in the last few years, several hundred photos to my web portfolio. I can count maybe 50 that were not made in either the city of Fort Wayne, or the county where the city is located (Allen County, Indiana).
In Fort Wayne, a large percentage were done in Waynedale, a part of Fort Wayne that was once an independent small town before being annexed by the city in 1957. I grew up in Waynedale, and though I lived in some other parts of the city for a time (and in New Mexico for a couple of years), I ended up back in Waynedale. A couple months ago, I came even closer to 'home' when I left my apartment in Waynedale and moved into the house that had been my grandparents' house. The house my mom grew up in. My parents live around the block in the same neighborhood.
My work is compelling because I am not photographing 'strangers'. Even if I have never met them before, my subjects are people who share my city, my neighborhood, and their culture. There's a connection that I have with the people and places I document, and I share that connection with YOU, the viewer of my work.
Find that connection in YOUR work, and you'll speak to your audience in a way that makes them WANT to listen.