They say that American men quit wearing hats as normal business attire when JFK took office - he notoriously took the Oath of Office sans chapeau.
The same was said about men wearing undershirts (as undershirts and not 't-shirts') - noticed to have dropped in popularity after Clark Gable removed his dress shirt in 1934's movie "It Happened One Night," and was revealed to be wearing nothing underneath.
Since the 1960's, beards on men outside of religious or academic circles has variously been seen as eccentric, revolutionary, or outlandish. Most polls indicate that people who wear beards are 'trusted less' by the general public. The British have a movement known as the "Beard Liberation Front." Of course.
Interestingly, the last time men commonly wore hats with suits as daily business attire, the most common hat style was the Trilby. These are hats not commonly seen today - the fedora is, but that wasn't popular in the '50s and '60s in the USA.