I've used both, and they are both great cameras in their own way. For ease of use and "pocketability" go for the XA. The 35mm lens, extremely low weight, and auto-exposure make it great for carrying around all the time, without needing anything else but film to take pictures in most any situation.
Now, the IIa is also a great camera, but it's much heavier than the XA. You'll know it when you carry it in your pocket! However, I like the lens on the Retina IIa more than the Zuiko on the XA. The only downside is that sometimes the 50mm focal length can sometimes feel a little too "tight" when composing. Still, it's a great lens! If you are using it in a wide variety of lighting conditions, you might carry a light meter around, as it has no built-in meter whatsoever. If you are good at "sunny-16" it may not be a problem. The plus side to the IIa in practical use is the more accurate rangefinder, lack of battery dependence, faster lens, and ability to easily use filters and hoods. The construction and quality are beautiful, but I don't know if I'd say the Olympus XA is junk by comparison. They were both built to be quality products.
All in all, I still feel the XA is the best pocketable rangefinder available. Tiny size, decent 35mm lens, apeture priority automatic exposure, built-in and accurate light meter, simple to use, quiet, and extremely affordable. It just performs so well in so many different situations, and does it all so easily.