Brownells sells the tool for $17.00 and that is worth it's price for getting the job done RIGHT.
Now, you CAN tighten it without the correct tool if that's what you choose (not my choice tho).
Secure the barrel in a strong padded vice and apply Kroil, etc. just to loosen things up. Next, loosen
the locking screw that secures the castle nut so it allows the nut to turn. Take a brass punch and place
the tip at an angle on one of the cut-outs where the correct tool would be inserted. Tap with a hammer
to turn the castle nut one way or another. Personally, I would remove everything, clean and inspect,
then re-install. Next, reinstall the flash hider and screw the castle nut as tight as possible with fingers occasionally
tapping it rearward with a block of wood, etc. to seat it. Now you can use the brass punch again to tighten
the nut until sufficiently tight....you'll have to be the judge of that. The castle nut will need to be timed so
the lock nut will engage one of the cut-outs at the front of the castle nut. A little coat of grease on the barrel
splines is good as is BLUE Loctite on the locking screw....not the castle nut its self.