Addressing your question from a somewhat literal perspective, the "holy" Hallelujah is likely that of the first verse, "The baffled king composing hallelujah" where as the "broken" Hallelujah is that that the female of the song (Bathsheba based on the bathing reference) drew from him, "She broke your throne, and she cut your hair / And from your lips she drew the hallelujah". My reasoning is essentially that the first versus associates the Hallelujah with God, which fits the definition of "holy" whereas the second is paired up with imagery of broken thrones and cut hair.
Now, I've heard there was a secret chord
That David played, and it pleased the Lord
But you don't really care for music, do you?
It goes like this, the fourth, the fifth
The minor fall, the major lift
The baffled king composing hallelujah
Your faith was strong but you needed proof
You saw her bathing on the roof
Her beauty and the moonlight overthrew ya
She tied you to a kitchen chair
She broke your throne, and she cut your hair
And from your lips she drew the hallelujah
From that context, the third verse seems like it would be contrasting the sacred and the profane joys in life.
You say I took the name in vain
I don't even know the name
But if I did, well really, what's it to you?
There's a blaze of light in every word
It doesn't matter which you heard
The holy or the broken hallelujah
That said, I've yet to find a statement by Leonard Cohen on the subject, only what others have stated. Complicating matters, Leonard Cohen initially wrote 80 different verses according to Wikipedia, and changed them for different performances, only keeping the final verse the same.
I did my best, it wasn't much
I couldn't feel, so I tried to touch
I've told the truth, I didn't come to fool you
And even though it all went wrong
I'll stand before the lord of song
With nothing on my tongue but hallelujah