I could totally care less, if people take my picture. Anything in public is free game.
What I'm concerned about is corporations and governments taking my picture.
I was taking a picture in the South of France recently with a 90mm lens "through/across somebody's lunch" from the adjacent table. The lady jerked back when she saw my camera. I said, "Don't worry, it doesn't bother me." She replied back, "Oh, but it bothers me." I said, "Mais, Madame, je ne prends pas une foto de vous..." I said it in such a kind sweet way that it was really funny. Did she really think she was so interesting that I would want to take a picture of her?
In the end, the finger is also a reflection of the person's personality. I just act normal around people with cameras.
JP