As someone who is now in kind of a dysfunctional organization, I'm very thankful for people who give honest feedback to management. If everybody who leaves just says "It's not you, it's me!", how can management ever make informed decisions about changing processes and decision-making? OTOH I understand people's and my own urge to just play it nice, so people who will soon become just nothing more than entries in a list of contacts do not despise you. Especially if the scene of the city you live in is kinda small.
There's probably a middle ground here: give constructive feedback withouth dragging others through the mud. But it's hard to hit this middle ground in an exit interview that is scheduled on short notice. So what I'm doing right now, is prepping myself.