ok - as noted above, I have both.
The X100s was on the shelf for a while, and then due to a repair on the OM-D E-M1, I took the X100s on a vacation as my only camera.
The X100s is ergonomically among the best - the controls are basically just what you need, no more, no less. it is easy to operate and hard to forget the menus/control function
The OVF is gorgeous. The IQ is spectacular. I was shooting at 3200 in near complete darkness in churches, handheld at slow shutter speeds, and the results were way beyond my expectations in terms of sharpness and saturation/exposure/noise.
The silent mode on the X100s is really silent, this is the quietest camera I've ever used.
the major limitation on the X100s is the fixed lens - sometimes the 35mm equiv is not what you need and then theres not much to do.
The E-M1 is a different beast, rugged, small yet not too small, great size for travel.
focus is great in most situations, the EVF is spectacular , IBIS to die for, lens selection almost to die for.
the weatherproofing of the E-M1 make it a wonderful little package to use in harsh environments. IQ is great, I've printed up to 16x24" and have no hesitations.
OOC jpegs are rich, almost never need to use RAW.
the downside of the E-M1 is the complicated menu/control/button system - the simple controls are easy to use and figure out - but the customization of buttons and the enable/disable of many of the wonderful options is buried in the menu system.
downside of the E-M1 - few really wide primes, IQ at high ISO's not quite as good as DSLR's
If I had to choose 1 camera - it would be the E-M1, it's more of a do anything camera.
The X100s is good for street, in home, travel etc, but not as flexible as a system camera for obvious reasons