I'm young and generally pretty friendly looking so in my experience I can get away with photographing a lot. When I'm out, I'm out with a big DSLR and nice clothes and I make sure I don't look suspect or sneaky. I never try to sneak shots in fact - thats one of the worst things you can do. I'd be pretty confident photographing around sensitive areas, however there are certain things you just shouldn't do - for instance kids at a playground or in public with a big long lens is a NO NO, people sunbaking is a NO NO.
Nearly all the time Im working in the 17-50mm range so i need to be close to people, and I've found that when you're confident, deliberate, well dressed and outwardly pleasant people aren't bothered by you/don't notice you.
I have this vision of a lot of photographers walking round the streets with their Ricoh GR or some sort of scale focus cam with a long cloak on sneaking shots. if you do that sort of thing, trying to pretend like you're not taking photos, of course people are going to be suspicious of you.
