Strictly speaking, this answer duplicates one that @Chris Sunami has already given: the "you" is a woman the singer addresses. But when I came and read his answer, I felt there was still some interest in mine. Like him, I wonder if my answer will strike readers as sacrilegious. I apologize if it is offensive.
The flow of thought in Cohen's lyrics is not logical; it is subjective and psychological. The poet is associating events in the lives of biblical David and Samson with events in his own life.
Passages referring specifically to David include (1) Now, I've heard there was a secret chord / That David played, and it pleased the Lord… The baffled king composing hallelujah; (2) You saw her bathing on the roof / Her beauty and the moonlight overthrew ya; and (3) the title and refrain.
(1) The first passage (Now, I've heard there was a secret chord / That David played, and it pleased the Lord… The baffled king composing hallelujah) refers to the David as a “musician credited for composing many of the psalms contained in the Book of Psalms” (Wikipedia) which does contain the oft-repeated title word.

David is often pictured with a harp or lyre, as in the picture David Playing the Harp by Jan de Bray, 1670.
(2) The second passage (You saw her bathing on the roof / Her beauty and the moonlight overthrew ya) refers to the story of David and Bathsheba, which appears in 2 Samuel 11:
2 One evening David got up from his bed and walked around on the roof
of the palace. From the roof he saw a woman bathing. The woman was
very beautiful, 3 and David sent someone to find out about her. The
man said, “She is Bathsheba, the daughter of Eliam and the wife of
Uriah the Hittite.” 4 Then David sent messengers to get her. She came
to him, and he slept with her. (Now she was purifying herself from her
monthly uncleanness.) Then she went back home. 5 The woman conceived
and sent word to David, saying, “I am pregnant.”
(3) And the third, the title and refrain, are a word used many times through the book of Psalms and often left untranslated in English versions. (It sometimes appears consecutively, but not in as many consecutive statements as in Cohen’s song. That’s more typical of spirituals.)
Passages referring specifically to Samson include (4) She tied you to a kitchen chair / She broke your throne, and she cut your hair. This relates to the way that Delilah betrayed Samson in Judges 16:
4 Sometime later, he [Samson] fell in love with… Delilah. 5 The rulers
of the Philistines went to her and said, “See if you can lure him into
showing you the secret of his great strength and how we can overpower
him so we may tie him up and subdue him. Each one of us will give you
eleven hundred shekels of silver.” 6 So Delilah said to Samson, “Tell
me the secret of your great strength and how you can be tied up and
subdued.” 7 Samson answered her, “If anyone ties me with seven fresh
bowstrings that have not been dried, I’ll become as weak as any other
man.” 8 Then the rulers of the Philistines brought her seven fresh
bowstrings that had not been dried, and she tied him with them. 9 With
men hidden in the room, she called to him, “Samson, the Philistines
are upon you!” But he snapped the bowstrings as easily as a piece of
string snaps when it comes close to a flame. So the secret of his
strength was not discovered… 15 Then she said to him, “How can you
say, ‘I love you,’ when you won’t confide in me? This is the third
time you have made a fool of me and haven’t told me the secret of your
great strength.” 16 With such nagging she prodded him day after day
until he was sick to death of it. 17 So he told her everything. “No
razor has ever been used on my head,” he said, “because I have been a
Nazirite dedicated to God from my mother’s womb. If my head were
shaved, my strength would leave me, and I would become as weak as any
other man.” 18 When Delilah saw that he had told her everything, she
sent word to the rulers of the Philistines, “Come back once more; he
has told me everything.” So the rulers of the Philistines returned
with the silver in their hands. 19 After putting him to sleep on her
lap, she called for someone to shave off the seven braids of his hair,
and so began to subdue him. And his strength left him. 20 Then she
called, “Samson, the Philistines are upon you!” He awoke from his
sleep and thought, “I’ll go out as before and shake myself free.” But
he did not know that the Lord had left him. 21 Then the Philistines
seized him, gouged out his eyes and took him down to Gaza. Binding him
with bronze shackles, they set him to grinding grain in the prison. 22
But the hair on his head began to grow again after it had been shaved.
The biblical figures loved G-d and wished to serve Him; but they were merely men, with weakness and failure in their nature, and they failed in crucial ways suggested in the lyrics -- David with Bathsheba, Samson with Delilah. Yet through it all (in Cohen's version) they repeated the title of the song, which is the Hebrew word for "praise G-d." Serving G-d remained the meaning of their lives, and in that sense they remain loyal despite their failings. They remain hopeful that G-d will accept that which is good in them.
Similarly the poet loves the woman he addresses and wishes to serve her; but he is merely a man, with weakness and failure in his nature, and he failed in crucial ways. (The nature of this failure is perhaps suggested by the fact that David's and Samson's failures were with women.) Yet through it all the poet repeated the title of the song. Serving the woman he loves remained the meaning of his life, and in that way he remain loyal despite his failings and hopeful that she will accept that which is good in him.
The line you ask about is one of the points of transition in the song. The poet says that while David was able to serve his G-d with music, the poet is not able to serve his love with music because she does not like music. This is of course ironic if we think of the poet as Cohen himself, whose occupation is music and who is even at this moment addressing the woman with a song.
I did my best, it wasn't much
I couldn't feel, so I tried to touch
I've told the truth, I didn't come to fool you
And even though it all went wrong
I'll stand before the lord of song
With nothing on my tongue but hallelujah