Opting for small size for a EDC brings its share of disadvantages; I've tried half-frame and 110 cameras, too... But, giving up covert carry, these days I just sling over my shoulder whatever rig I'm currently using and carry it everywhere. Tucked under my elbow, it's pretty inconspicuous. Just one body, one lens, only rarely an extra roll of film, no bag.
The Minolta CLE is a great one for this. But just now I returned from a walk to pick up the mail a couple blocks away but diverted a few extra blocks to pass by some local photo targets... with a Bronica RF645 over one shoulder. Easy, and certainly my most common choice to have along, anywhere.
Oh, yeah, and I have to say I did try the small route this past year with an APS-film Canon ELPH Jr, even picking the fixed non-zoom 26mm f/2.8 lens for its speed and in hopes of better optical quality. I got the Canon leather belt pouch for it, and this sure makes a handy little package. As it turns out, I'm just not satisfied with the quality of its output. And so I just have little respect when using it. Too bad...