As far as the camera itself, it has more to do with size and shape. Just the sight of a lens popping out of a rectangular box these days is unusual enough to grab anyone's attention if it's in their field of vision. If you're a middle aged or older fellow with a retro camera they won't think much of it, but if you're young there's a good chance people will think you're a hipster, especially if you're wearing a trustfundium outfit. I like to dress nice, and I like to take pictures, but I don't do both at the same time. When in Rome.
The other big thing is body language. Don't stoop. Walk at a slow, deliberate pace. Keep your head high. Don't be stiff and jerky. Don't whip out your camera (unless you have to). If you look like you belong, people are less likely to bother you. You should be comfortable anyway, regardless. Don't be tense.
It occurred to me that it's probably a very big advantage to be a woman (yes, seriously!) while shooting on the street. People are much less likely to assign sinister or otherwise bad motives to some girl or a lady with her camera.
The compacts like the Olympus XA and the Yashica T4 are good for quick, discrete shooting. They could pass for some kind of cell phone.
People have become so acclimated to the cell phone camera that getting anything out of a different form factor is always going to draw attention, whether it's black or pink. I have a harder time here in the midwest because nobody exactly comes here to visit aside from coming to watch our sports teams lose, except the Buckeyes. In which case, they come to watch their non-conference team get bulldozed to provide the pavement to a bowl game or championship.
http://www.youtube.com/watch?v=AN0WqSeCKW8
