The garrisons that pop up are "calculated" based on city size, fort levels etc...it's dynamic...so the bigger/stronger the city, the better the garrison. Now it's safe to build up garrisons with your own units, like guns, troops and supplies will help, and sending a good defending commande...