Nick, the entire shutter mechanism on the 35SP is located in the lens housing. So, there is not much between you and the shutter to muffle the noise. It's an old design, held over from the mid-1960's.
The 35RC and later 35RD have their escapements fitted within the body of the camera. They are therefore noticeably quieter in operation.
Another noticeable trait with this shutter is changing shutter speeds feels about as smooth as shifting gears on a 50 year old Minneapolis-Moline tractor, especially when engaging the slow speed gear train.
That said, unless you're addicted to an auto mode, the 35SP will outperform the 35RD. It's seven element G. Zuiko is simply superb.