I think this is an engine issue, across all AGE games. It happens if two structures in the region are called by the same name, here the town and the fortress.
"Umbrellas will not be opened in the presence of the enemy." Duke of Wellington before the Battle of Waterloo, 1815.
"Top hats will not be worn in the Eighth Army" Field-Marshal Viscount Montgomery of Alamein K.G.