I have two Nikon N65s (aka F65s),both purchased on a whim at a time when 1 they were new on the market and 2 my interest in film had not quite been taken over by the yet-to-be-released range of Nikon DSLRs, which came a few years later. Not rangefinders, I know, but there they were in the shop, there I was with a little money to burn, and I gave in to temptation. As we all do when new gear comes along.
Those 65s went through batteries like nobody's business. I could not get more than a few rolls of film out of a set of two when it was again time to spend for another two. I would buy and install them, take one or two rolls of film, put the cameras away for a few weeks, and zip! the batteries were drained.
A few years later I went to a photo fair (the Box Hill, Victoria annual shindig, well known to and popular with many Australian readers here) and by chance lucked into a seller who had two battery packs for the cameras. These packs took standard AA batteries and I could buy these from the supermarket or any Reject Shop. They wanted $40 each but I bargained the two down to $50 and took them home. The rest, as they say, is history.
So yes, there are times when new batteries are worth while investing, and other times when they are not.
Other than this last comment, I will say only this. If I were you, I would risk a fiver for a new battery/new batteries and take the chance. It's after all, only the price of a coffee nowadays. Let's face it, the film for your camera will cost you more than that anyway, but that's another matter entirely.
BTW those F65s/N65s are marvellous small cameras. All my D lenses work on them, and the results are as sharp as anything I could ever want. So win-win for me.
Let us know how it all works out, please. We all enjoy happy/positive endings to these threads.