if you don't mind the slower AF speed of the X-series (x100, x-e1, x-pro1) and the parallax effect of the viewfinder and the battery life then i don't see any reason why you shouldn't switch to the X-series.
And since you're comfortable with the 35mm focal length all the more reason to dump the weight and go for a lightweight system. Your shoulder will thank you for that.