If the 35/1.2 is too big for you, then it's just too big. End of story.
I can understand that. I have one. I think it takes really superb images, but I do hate carrying it around my neck. In fact, I will no longer carry it is a single lens single body outfit - it is so big and heavy that it defeats the purpose of such a minimalist kit. But if I have a camera bag with me, it's always there. Amazing lens.
I did look long and hard at the CV35/1.4 before buying the 1.2, but I saw too many images from the former that had very choppy distracting bokeh. Yes, and distortion. I knew there would be plenty of times when these characteristics would not bother me, but I compromised on weight to elliminate those "wasted" photo opportunities. I can well understand how someone else might choose differently because they find backgrounds less distracting and/or their primary goal is a teeny tiny camera kit. In which case, it makes a great choice for a fast standard lens as all other members of the kit are likely to pretty slow lenses to keep size down.
As an alternative small, light, fast lens, I can recommend a 50/1.5 sonnar (current or legacy) if you are lucky enough to get one at a decent price.
But if you really want a fast 35mm that is also small... well, you have no choice in that price range - you have to get the CV35/1.4.
Just to be safe, don't sell the 1.2 to pay for it.