The last time I had cold dead hands I was using a 200-600 and photographing owls in Canada in the winter. Minus 10 degrees. Fortunately I have my fully functioning hands back now. And my images are to keep.
Hard to choose, but if I could only have two prime lenses to cover all my needs (family, scenery, buildings), I could use Loxia 21 and Voigtlander 35APO.
Any variety of 70-200 or 100-400 would cover most of my needs. If forced to commit to a specific lens it would have to the Sony FE 70-200mm F4 Macro G OSS II as long as I can have a 1.4x TC too.
Thought about this quite a bit. For E mount, I'm choosing a lens that is near my favorite focal length. It's discontinued, so it's harder and harder to come by new, and it took me a lot of work to find a very well centered copy. So I'm going with the....
Zeiss Loxia 25mm f/2.4
On the Fuji side, I only have 2 lenses, so it's quite a bit easier. Going with the...