The holy "Hallelujah" (literally "God be praised") would be then calling God in prayer, the broken one - calling him in vain.
In one of the version of the songs, there is this part:
And remember when I moved in you
the holy dove was moving too
And every breath we drew was Hallelujah
So the singer was praising the Lord with every breath. Now if you look at the lyrics of the part in OP, you'll notice that the singer is being chided for "taking the name in vain".
You say I took the Name in vain
I don't even know the Name
This is of course against the commandment "Thou shalt not take the name of the Lord thy God in vain". So in another words, the singer apparently called God while not addressing him in prayer - judging by the rest of the song it was something akin to "Oh God, she is so beautiful!".
You saw her bathing on the roof
Her beauty and the moonlight overthrew you [...]
And from your lips she drew the hallelujah
This was of course a blasphemy, but then the singer argues:
It doesn't matter which you heard
The holy or the broken Hallelujah
Similarly, you can take another part of the song :
Maybe there's a God above
All I've ever learned from love
Was how to shoot somebody who outdrew you
And it's not a cry that you hear at night
It's not somebody who's seen the light
It's a cold and it's a broken Hallelujah
Or in different version
It's no complaint you hear tonight
It's not some pilgrim who's seen the light
it's a cold and it's a lonely(/broken )Hallelujah
Someone (who lost the loved one) is not wordlessly crying or praying (seeing the light) at night - they instead keep sobbing and calling God (as in "Oh God, Oh God, why?" ), in such a "broken", blasphemous hallelujah.