Your situation sounds very much like mine!
Most of my free time are spent with my family, 4 kids, with one carried in front with those Baby Bjorn carriers and sometimes even one behind in a back pack. Rest are in tow. But I always make it a point to carry a camera with me, and like you, I had more than once bashed them with my camera when I bend down to pick them up!
Once I bend down to pick my daughter up, my leica slipped off my shoulder and knocked her head real bad. It proceeded to drop onto the hard floor...my natural reaction was to check my camera first 🙂... My wife wasn't very pleased. The cam went back to Germany for 6 months.
Anyway, most of my photos are of my family. They are the most meaningful photos to me. Sometimes I may have a chance to snap something nice, maybe at the beach while they are playing, or at gardens etc. But not easy to get nice photos as the mood is not right.
I do hone my photography skills this way though, and when I take leave to do "proper" photography, my skills will not be rusty.
Think of it as photojournalism, long term documentation of family life....and wala! It is now "proper" photography! But my darkroom peers (community darkroom) do find me wierd though, spending so much effort to get family snapshots look right.