With Contax T2, you not only get a p&s, but also control over exposure, you even have the option for manual focus, all that you just won't find in Yashica T4/T5 or Olympus Mju I/II. So I don't really think it's just the posh look & feel that makes T2 more expensive. It really is a more sophisticated machine.
Anyway, when I want keep things simple, my favourite would be Yashica T5. Got it used for approx. $100 USD-- a bargain for a camera with a Zeiss T* lens. Problem is that nowadays a Yashica T5 with box & manual can easily go for $200... then itself becomes overpriced, IMO.
I use Olympus XA and Hexar AF, too. Love them as well.