Everyone will have a different view on this; here's mine. Bigger is better, but you have to trade size off against - well, size. When I want the best possible picture I can get (regardless of convenience) I shoot LF. When I want a good compromise between portability and image quality I shoot Hasselblad 6x6. When portability is the overriding consideration - eg on a work trip where a Lowepro backpack isn't an option - I take my Bronica 645. If it has to be REALLY small then I take a 35mm RF.
Yes, if you have only shot 35mm before, 645 will look pretty good. My Bronica kit is similar in bulk to my 35mm kit, but I hardly use the latter these days. But if I have room I will take a bigger camera than 645.
As for rectangular vs square format, when I got my old Hasselblad 500CM I was quite worried about the square format and spent some time looking for a 645 back and paid quite a bit for one when I found it. I seldom use it - following the logic above, if I am going to take my Hasselblad somewhere, I want to use the biggest image size of which it is capable. And since I now have access to every format from 35mm through 645, 6x6, 6x9, 4x5", 6x12 and 6x17, the answer is that they are all good and I find that I frame the shot to take advantage of the aspect ratio. I find no intrinsic superiority between 6x6, 6x17, or anything in between; it is how you place the image within that space that matters.
The good news is that these days all of these options are cheaper than they were, thanks to the flight to digital by many commercial photographers.
In summary, if portability is not a deciding factor - go bigger than 645.