Frankly, if one was supposed to stop and ask before taking a street shot, some of the best photographs in the historic archive wouldn't exist.
Personally, I have no problem with walking into a crowded area and taking pictures of strangers. I use a small rangefinder, but don't hide the fact that I am photographing. If someone wants to talk to me about it, I'm happy to chat. I will sometimes start conversations with interesting-looking individuals, then ask them if I can take their picture. Sometimes I don't ask. I mix both approaches.
As a rule, I don't photograph the homeless, the marginalized, drug addicts or people in mental distress. I have relatives who fit into those categories, they are not exotic to me. Such individuals play out many intimate aspects of their lives in public, and the line between their public and private selves is blurred, or in some cases, virtually non-existent. Photographing them in the street is practically akin to someone walking into your house and photographing you in your living room or bedroom. You'd be pretty p*ssed too if someone did that. As has been pointed out in this thread, it can also be unsafe. Furthermore, I find it a photographic cliche - the "meaning" of such pictures is easily reduced to simplistic categories (pathos, desperation, humility, the "beautifully ugly," etc). We have all seen a million such pictures.
What motivates me when it comes to shooting in public places is trying to discover who is really there when you look close enough. Who are the people who cross this intersection, utilize this beach or park, ride this bus, etc.? What do their faces - or body language - tell me about how these usually private individuals are feeling at a particular moment in the midst of a crowd, and what emotions they bring with them into a public place? In the middle of all this hustle and bustle, do they look bored? Do they do something unguarded like scratch an itch in spite of being surrounded by other people? I don't mind if they look at the camera - I'm part of the environment they are reacting to. If they give me a look of hostility, annoyance, etc., it tells me how they feel about having a stranger point a camera in their direction. That's fine.
I am also interested in how people change mundane locations into something different through their actions. Beach-goers who seem to transform Lake Ontario into the Ganges through the way they relate to the water; or who use the point where land and water meet for religious gatherings. I wouldn't describe this as street photography, but it does involve me taking pictures of people without asking for their permission.
As an aside, I too once found myself surrounded by people after taking some shots of people playing chess in the park, wanting to know what I was doing. Some of these folks were obviously up to more than playing chess. I told them the truth: I was interested in the people that come to play chess and the environment, the scene, I like chess myself, I wasn't photographing the specific individuals who were accosting me, and I'm not taking pictures for anyone other than myself. I was calm and genial. Within a few minutes we were talking about other things, and they actually thanked me for taking the time to talk to them, as opposed to snapping photos and running off (which I guess is what a lot of people do there). Perhaps it was because I had spent about half-an-hour before hand watching and listening, being part of the scene, making my presence obvious, before I started snapping away. I didn't just barge in there loaded down with camera gear and a fanny pack and start firing away like I owned the place.
Anyway, I fully expect to get accosted again, or even to get a fist in the face one of these days. I try to stay away from people who look like they might throw urine. But as long as it is nothing more serious than that, or even a broken camera, I will probably survive. I might even post my experiences here, and we can debate about whether or not I deserved it.
To the OP, it sounds like the experiences you described have caused you to re-evaluate your approach, which is a reasonable response. If you want to photograph people on the margins for reasons of photo-journalistic interest, then undertake a true photojournalist approach. Research, get to know them, their lives, their environment, build trust, etc. Or you may choose to photograph the world you are more familiar with, and think about what you feel and have to say about that. In either case, I believe that if you can see yourself in the people you photograph (as opposed to them being an alien species), you are less likely to have trouble, and more likely to walk away with a good picture (and dryer to boot!).
