I never owned a M4 but I would say while there are small differences in practical use, it boils down more or less to the question which styling you prefer. The M4s are a bit younger for sure, but if a couple of years makes much of a difference on cameras that are 35+ years old, is debatable. It all depends on the condition of the individual sample you are looking at. If I am informed right, the biggest differences are in the design of the film transport lever, the loading mechanism (quick loading vs. traditional loading) and the frame counter (the M2 frame counter is automatic, but has to be reset by hand, while the M4 counter resets and counts automatically) and the rewind knob. The early versions of the M2 didn't have a self timer, if that is of any importance for you. The viewfinders are more or less the same with the exception of the (rarely used) 135mm frame line on the M4 that is paired with the 35mm frame line. The M2 shows either the 35mm frame line, the 50mm frame line or the 90mm frame line, but never two frame line sets at the same time. You can find more info on the various models in condensed form
here.
On the Summicron vs. Summarit topic: The Summicrons are all fine lenses, you cannot go wrong with any of them. The early Summicron versions (v.1 "collapsible" succeeded by v.2 "rigid" and "dual range" with improved optics over v.1) are preferred by many shooters for black and white. They are prone to haze and scracthes/cleanings marks (soft glasses), so you have to pick your copy carefully. The later versions of the Summicron (v.3 and v.4) give more contrast which helps if you prefer to shoot color, but they also tend to be more expensive, especially version 4. I think the latter one would be outside your budget. The classic Summarit lens (f1.5, as opposed to the modern f2.5 Summarit) is a lens over which people are very much divided. It's basically a coated version of the 1930s Xenon (if I'm right) and as such it is often said to be not the sharpest lens, but some poeple still love it for it's character. It has very soft glass and is difficult to find in clean condition. The classic M-mount Elmar (f3.5, late f2.8; 1950s to 1974) can be had usually cheaper than any of the Summicron versions, but it is one to one and a half stop slower and optically inferior at larger apertures. Stopped down to f5.6 or f8 there isn't much of a difference, I would assume. It is a compact (if not tiny) lens collapsed, but as with the collapsible Summicron, the ergonomics of collapsible lenses are not for everyone. More condensed information on the various lens options
here
