I think this is a broder concern you have for what people are thinking of you when shooting in public. I know what you mean about being sensitive to people's suspicions, but that's just part of working on the street the way I see it. I've been detained by police, questioned by any number of people, and nearly beat senseless a couple of times were it not for quick feet - people just don't like us out there sometimes.
"Who are you with?"
"Myself."
"What are you shooting for."
"I'm an artist."
That's my story - most people can't wrap their head around an "artist" meandering around with a camera anyway, and you're off the hook.
For me and my photography, children are one of my favorite subjects. Some of my best images have been of children. I have four kids, so I see a lot of meaning that other children bring into the world and by extenasion, my work. They are a truly wonderful subject.
Unfortunately any creep that wants to take pictures of children to satisfy a sick fetish is going to do it, and there's really no stopping it. And it's hard to explain to someone that we're not up to something like that. But, I would just approach it like I do any time I work on the street. If I see an image I TAKE it. It's mine. I don't act shifty or try to "sneak" an image. Granted, I don't go out of my way to be beligerant or obtrusive, but I'll definately stare you down with the lens, take your picture and walk off without explanation.
I read a long time ago and employ it routinely the words of Alfred Eisenstadt, who used to get in to all kinds of closed-door type events by dressing up, and just going like he owned the place. If you act sneaky, people assume you're up to something. If you act up front, people assume you are there and shooting for a purpose. Occasionally, people will still take exception, and that's just too bad.
So I guess it all comes down to: are photos of children what you're after? For me, the answer is that children who enter my frame are fair game, and avoiding them would be to edit myself while shooting, and I just won't do that. If you are missing photos because of a fear of what other people are thinking, that's a shame, but believe me I understand it.