I carry a small vegetable scrubbing brush, obtained in a dollar store, for a dollar (doh!). Of course, it is since only ever used for my boots.
Using a toothbrush is a similarly good idea, but the veggie brush has longer stiffer bristles, and is actually designed and intended to remove soil from locally or self-grown veggies.
When walking, if my boots are muddy and soiled near the end of the day, I seek a puddle, ditch or suitable trough in which to 'swish' my booted feet around to remove excess mud and soil. Failing a convenient, standing pool of water, long grass or any available non/cosmetic foliage works as well.
The veggie brush is the last line of external boot hygiene. Be sure to wash the brush afterwards...
I hope this helps.