Buy or make a starter pin punch.
This is just a standard punch cut off to about 1/2 inch working length. This prevents the punch from bending or flexing.
Often a nail set with the end stoned flat will work.
When you get a really tight pin don't just keep pounding on it, that can deform the pin and lock it even tighter in place.
Get the part in a FIRM no bounce-no move setup.
Normal benches will allow bouncing, so I've often done pin removal on a concrete floor with a pad of newspaper to prevent scratching.
Have a buddy help hold it in place if necessary.
Use the starter pin punch to get the pin moving out, then go to a longer punch.
If needed use a bench block with a hole to pass the pin.