Thanks guys, little behind on updating this.
Ended up moving forward with my plan pretty close to how I originally described it. I just used 1/2" plywood, made it 18" high. Figure that's higher than I need, but gives me a little safety buffer as I will be using this on the road. Instead of running bolts through the uprights, I bolted 2x4 pieces onto the plywood that fit in my stake pockets. This way it doesn't take any tools to install or remove. I just pull out the sides, unfold, and drop in. The piece on the front just slides in also, blocked by the rail on the front and a couple of thin blocks that I screwed to the plywood on the inside. It all looks like it would be unwieldy, and to be honest even I kind of thought I might need a second person to help install and remove. But there's enough flex, I'm easily able to put it in and take it out by myself in no time at all.
Hope I didn't screw up with the type of plywood I bought, but if I did it's not a super expensive lesson. I hit it with deck stain, hoping moisture and/or rain doesn't destroy it immediately. But as I said, I'll only be using these sides when hauling firewood or other materials where they would come in handy. Any time I'm not hauling, I intend on removing them and putting them inside. I won't even have them set in the stake pockets when traveling empty to where I'm going to get firewood. I'll fold it all up, strap it down, and install when I get there. I know nothing's going anywhere once there's a load of wood inside the walls.
I actually just used it for the first time today. I'll get some pics of it up tomorrow night. But for now, here's some pics of when I was putting it together.
Folded up and ready to be put away: