As I interpret it, the pressure of the daily grind is literally tearing people apart. It breaks buildings, it breaks families (likely speaking of divorce, here - the pressure getting so great that it chips away at the family until they can't take it and they divorce, or otherwise become estranged from each other), and it breaks people.
Basically, this song is speaking about the extreme pressures of modern life - the daily grind and this whole idea of having to have the most and be the best and richest and work the hardest, which can take a great toll on people, to the point where it can cause people to fall into poverty, and thus lose their homes and end up living on the streets.
The chorus emphasizes this idea of the pressure of the world and of life, and how no one wants to be a part of it, but there is no escaping it:
"It's the terror of knowing what the world is about
Watching some good friends screaming
'Let me out'"
Once people realize the stress that just trying to keep your ahead above water in modem society entails, it becomes this self serving endless circle, where you can't get ahead and the pressure of knowing that makes it harder to get ahead, so eventually you are so far behind that you can potentially end up on the streets in extreme poverty.