Call it calibrating your brain or judging light quality, the facts are that you CAN figure it out without a lot of trouble if you just pay attention.
The human eye/brain combination are designed to set your 'exposure' if you will, using features similar to a camera lens - the iris, which expands and contracts to let more or less light in, as well as features that no camera lens has, but digital sensors are starting to have, which sets 'ISO' if you will, in terms of brightness level.
The human eye/brain combination are far superior to any camera ever made in terms of ability to record dynamic range or latitude.
But the brain does not report to the conscious mind what 'f-stop' the iris was set at, nor what 'ISO' the brain choose to impose in order that you not be overpowered by strong light or left in the dark by dim light.
In other words, the human brain is a very poor judge of absolute light levels.
Can you 'train' your memory to recognize similar situations of lighting, and to remember what settings were effective at that time? Yes, I'm sure, and I'm sure that's what you were doing. Did it work? Sure, I believe you when you say it did.
But it is a guess, and highly prone to error. If the day had not be pretty much just as you recalled, your guess would have failed. Perhaps the error would fall within the ability of the film or sensor to cover by exposure latitude, and perhaps not.
You cannot train your brain to recognize 'this is EV 14, this is EV 10' and so on. It just isn't possible.
So your eye cannot be a meter. You can only be a relatively good rememberer/guesser. If the scene you wish to photograph is outside your frame of reference, you have nothing to rely upon.
From the earliest days of photography, photographers knew this and strove to invent mechanisms to measure light.
We don't say
'I just guess at shutter speed, why actually set the speed?' We never guess at aperture or focus, we want control over those things.
Why would we not want control over exposure?