Not a dumb question at all.
The most quantitative answer I can give is: consult the MTF charts.
Usually, MTF charts (e.g. at photodo) will list the numbers wide open and stopped down to f/8. What you will see is that the performance wide open can be
very different between two different lenses, but by f/8 the performance of those different lenses is already starting to converge. Compare the
noctilux at f/8 to the
summi at f/8. Not much difference really, at least not until the very edge of the frame.
When stopping down as far as f/16 in small format, I doubt that there will be many meaningful differences left, in terms of lp/mm detail. At f/16, diffraction softening is already in play. Peak performance is going to be more around f/8, typically (though with some RF lenses you can get fantastic performance very close to wide open).
Now, setting MTF aside... what about real world performance? Here there can be a big difference between a fast lens and a slow lens, even if both of them are equally stopped down. There is something you said which can be very important:
"all that extra glass." Indeed the faster lens, with wider max aperture, will generally be more prone to certain issues like flare and ghosting. So you should always use a hood when possible, of course. But even so, some fast lenses really don't deal well with front lighting. A hood helps but it's not a cure-all.