It means his laugh was a hearty, good-natured laugh. There are really two ways you could've pieced this together: one, realize that the letter "k" is a common part of inherently funny words, which are simply words that can make people laugh without any other context. According to Wikipedia, (emphasis mine)
the concept that some words, especially those with a k sound, are inherently funny is a common trope stated in many fictional works. In the Neil Simon play The Sunshine Boys, for example, a character says, "Words with a k in it are funny. Alka-Seltzer is funny. Chicken is funny. Pickle is funny. All with a k. L's are not funny. M's are not funny". Similarly, the Star Trek: The Next Generation episode "The Outrageous Okona" features Joe Piscopo as a comedian who, in attempting to teach the android Data the concept of humor, refers to words ending in a k as funny.
The second clue that gave it away was the context of the poem. In it, the narrator talks about how her grandfather used to spend time with her and have a deep, personal relationship with her. This is when the author brings the laugh sounding like the letter "k". Now, however, he is distant and so tired all the time, so much so that it feels as if he's not even there.