I know it's been done to death, but I'm not comfortable taking someone's photograph (as the point of interest) without their consent. A street scene with incidental people is different, and generally impersonal. Dunno how I'd act if cameras were being stuck in my face all the time. I'd probably NOT be happy...
JM2C.
Don