Well in my view I have always thought prisons were made hard to get out of, and not that hard to get in. I think they are marginaly better as a defensive post than a small village, maybe worse in some cases.
Prisons tend to be isolated, which could make it hard for an enemy to attack, but it also makes it hard to keep it maintained with supplies. Kinda like an old Castle.
And unless they are like the old time prisons with tall stone or concrete walls which would be vulnerable to explosives, you have marginal defense capability.
Most prisons now have tall chainlink and barbed wire fenses. <a good number at least> their plus is a good survelance system and guard towers, but this can also be targets, and if you were to come under attack, a machinegun or rocket launcher would make short work of them, or sustained fire could keep the guards from either reaching or leaving the towers. How long could a man sit in a tower without food or water, and taking fire? It wouldn't be long before he made a break for it.
For me, it would depend on the prison, and of course the status of the inmates. Imagine being attacked by marauders. And you have inmates. That would suck, having basicaly a two front fight.