I have to say that I have the EM1 and also shoot the X100S (and owned the original X100 as well). I had an XE2 rig to go with my X100S, but was too often frustrated with the focusing speed, which I found to be about the same as the X100S. I shot with the original GF1, then the GF3, then left m4/3 before returning to the EM5, then left the system again, before recently returning to the EM1.
I will say that I went (back to) m4/3 for the same reasons you seem to be: autofocus primarily, but also smaller sized bodies/lenses and the high-quality Olympus primes. And for all of those I haven't been disappointed. I've always wanted to love it, but then I get sensor-envy and get seduced into APS-C or FF. However, the number of times I've gone back to m4/3 tells me there's something there. IMO, it's a great compromise as you get smaller sized bodies and lenses, and faster performance. The APS-C cameras seem to be the same size as the FF bodies anymore (the Sony cameras I've used, anyway). But, I digress.
I'll say that I love shooting the X100S and can shoot ISO 3200 without issue. I rarely go beyond that since I can handhold it incredibly still, and I'd rather not deal with TOO much noise. 6400 is doable, but 3200 is where i'm comfortable going unless I have no option. Honestly, if it's that dark, I generally stop shooting unless it's really important. I'm not a working pro and I don't shoot concerts.
Shooting the EM1 at 3200 is usable. I'd say, to my amateur eyes, it's not a full stop behind the Fuji bodies; it's more 0.5 stops. I don't have any evidence--I haven't wasted my time with comparison shots of bottles of ketchup in my dark kitchen and zoomed in to 100%--but it's the general sense I get. I capped the auto-ISO (which I use often) at 3200 for both the X100S and the EM1, allowing that I'll have a bit more noise in the EM1 files at the same ISO.
But the autofocus blows the X100S away. Every. Single. Time. And to me, I'm often chasing my kid around and so the AF and general speed is more important than high ISO. I have medium format or 35mm when I have more time, or the X100S. But that's just me, your opinion may vary. Hope that helps.