I've been thinking it over more and my tentative conclusion is that something's wrong with the Surefire 18650 cell that came with the light.
The light initially worked with 123's and with the included 18650 and clicky cap. It worked with the DS cap and 123's.
It worked with the DS cap and 18650 when the head was partway on, and quit with the final turn or so to install the head.
The other 18650's (Surefire and Panasonic) worked fine with both caps.
I put the same 18650 cell in the light with the DS cap and clicky cap. You can see in the photos that the cell sits lower in the body with the clicky cap.
My theory is that, due to the different battery position/spring force, the cell sees more end to end force with the DS cap than the clicky cap and the 18650 protection circuit was somehow malfunctioning due to the higher end to end force on the cell. (Which would also explain working when partially threaded, then abruptly turning off when the head was fully installed)
I'm not sure why it would have started working after all my early attempts. Maybe exercising the cell got it loosened up?
I emailed Surefire about this and explained what tests I did.
Unfortunately, their guy is either ESL or didn't read my email.
To paraphrase their response: no, you need to thread everything on all the way. The scout lights work with DS00 caps.
I'm disappointed with Surefire. Chinese 18650 cells, poor tech support, and whatever is or was happening with my light. I'll try giving them a call and see if they'll exchange the suspect cell. Hopefully I won't need to speak Hindi to get the point across.