Do you handload? If not, 357 magnum, full stop.
A 357 magnum will chamber, 357 magnum, 38 special and 38 short colt. A 44 magnum will chamber 44 magnum, 44 special and 44 Russian.
The real question is, do you want to hunt deer at over 50 yards? If not, you don't really need a 44 magnum. With that said, if hand loading, 44 Magnum vs 357 Magnum is like 12 gauge vs 20 gauge. Sure, the factory ammo is of certain specs, but 44 magnum can be loaded down to 357 magnum or below, while 357 will never be as powerful as 44 magnum.
I prefer the recoil impulse of a giant bullet going slow. I went with 44 magnum.
ETA: For my purposes, the real question was 41 magnum vs 44 magnum. In my calculation, if hand loading, a 44 magnum can be reduced to do exactly the same thing 41 magnum can do, but the opposite is not true.