The unit does not have to stay there for the whole turn to blow up a depot. You can issue a move order along with the order to destroy the depot - the unit will move after 5 days, so any movement longer than 10 days won't be completed by the end of that turn.
EDIT: Ethan you have impeccable timing
EDIT #2: As for advice, if you want to move the bulk of your forces away faster, you can always detach a regiment or brigade to stay behind and destroy the depot - you only need one combat element. Ditto if you have an inactive leader - just detach Mr. Slowpoke from the stack and give the order to your troops. Just take care that the smaller force you leave behind doesn't get overwhelmed by the enemy reaction.