Let's think about this: I think it's safe to say that Arbus gained the confidence of her subjects, but often violated their trust with the images she chose to make public. A perfect example is the picture of the boy with a hand grenade. Her contacts mostly show multiple pictures of a sweet little boy mugging for the camera. But the one she printed for public viewing was the one image where he looked possessed.
Was Arbus able to gain his confidence because she was a diminutive, "unthreatening" woman? Or was it at least easier for her to do so? As boojum points out, being female might give a photographer greater latitude that can be exploited. I agree, but that greater freedom doesn't absolve a photographer from ethical responsibilities.
My hope is that, as more women continue to enter the medium as artists, they will use their experience of women's continued marginalization to offer a broader, better rounded, and more compassionate view of the human condition. I say this not from the position of viewing women as the stereotypical nurturing, compassionate caregivers. I hope that anyone who is marginalized by our society can make their perspective more visible, both as critique and as a vision of a way forward to a more compassionate and inclusive world.