Certain streets of the Tenderloin and Mission are wall to wall homeless and junkies, but during the day they seem to be on their best behavior and aren't a threat to pedestrians, except asking for money. Again, it's parts of these areas; not all of it. Not sure if I would make a habit of wandering through the sketchy areas at night though.
Dogpatch is supposedly pretty bad, but I haven't made it over there, yet.
Obviously certain parts of Oakland should be avoided at all cost.
I don't like the BART, because it travels through some pretty bad areas and you always hear about problems on the news.
Lots of homeless and druggies on Market Street, right in the middle of the busiest area but again, not really a threat during the day, but still not pleasant.
A lot of streets smell like a urinal at times, especially in the summer. There's feces on the streets; probably human since most people seem to clean up after their dogs. Personally I would never wear flip flops in the city, because of the feces and needles, but then again I've always been a leather shoe / boot kind of guy. Just keep an eye out, where you are stepping. You will occasionally see people smoking crack, meth, pot and shooting up etc right in the open. The smell of pot is common. It's not New York, where NYPD would shut that down in an instant. Overall SFPD is very restrained and underfunded, which isn't a surprise given the political leanings of SF.
Overall the city is walkable and shootable. The tech industry has sucked a lot of the character out of the city, but then again it hasn't been the elegant and sophisticated city it was until just after the war (WW2) in a very long time. But there is a lot to see and do in SF. There are tons of places, bars, restaurants etc to explore.
Little Italy, China town, The Embarcadero, Fisherman's Wharf, large stretches of the Mission, the Presidio, Nob Hill, Russian Hill etc are all good. Personally I don't feel unsafe and walk around with my Leicas visible and if things get squirrelly I just stick it in my jacket or bag.
Never ever, ever leave anything visible in your car. It will get stolen and you'll have your window busted. It's not just the druggies, but organized groups doing the work.
As always just keep your eyes peeled and be situationally aware.
There still is a lot to like about SF. Its not London or Paris, but those places are a whole different story. Personally I love the weather. The fog and weather make for some great pictures.