Work days: I put my shoes on just before I leave for work. Before that, I'm wearing Crocs. When I get home, shoes come off, put on Crocs.
Off Days: Wake up, put on Crocs. If I go outside to do anything, I put on shoes and don't take them off until I get ready to take a shower. Then I put on Crocs.
It's my house, I'll wear whatever the hell I want. And I don't ask anyone else to take their shoes off, either. I don't have carpet, so if the floor gets some dirt on it, I can clean it up in a matter of seconds. No worries.