The unspooling thread is used as a metaphor for the line of Orville's life. In this passage, he is returning to his hometown by train (and then by walking along the tracks, after the train's air conditioning breaks down), and the smell of creosote brings back a childhood memory to him:
Creosote. All at once he saw himself as a six-year-old, one summer's day, lying on his back in a neglected grassy field down the street from his house on Ten Broek Lane. The scent of creosote was strong from the railroad tracks running nearby. Alone, he stared up at the clouds passing across the sky and suddenly had the sense that the world as he was seeing it was only a part of something else. For the first time in his life he saw himself as part of some whole, some whole world to which his own being was seamlessly connected. He felt lighter, more alive, as if something else had clicked on—or in. He leapt to his feet, making his legs go as fast as they could, and ran home to tell his mother. He burst into the kitchen and blurted out his discovery as the screen door slammed—bam!—behind him.
Selma Ariel Fleischer Rose, a large, aproned shape looming over the stove, didn't respond.
He persisted, dragging a chair over, climbing up, and telling her again, slowly and loudly.
"Something else! Mom, I'm part of something else!"
Selma stared at him. He saw a cloud pass across her gaze. She sighed. "Orville-doll, there's nothing else but this."
The boy felt a rough, twisting pain in his chest. He clenched down on it, trying to make it go away. He fought back tears.
"What's wrong, honey-bunny?"
Dread was rising, the pain was going. He felt him-self numbing up, like his mouth did when he was at the dentist's. He broke eye contact. Feeling her fearful con-cern, he said, "Nothing." He turned and ran back out the door.
This passage immediately precedes the paragraph that you quote in your question. That moment from his childhood began the pathway of his life, which he likens to a thread unspooling, and which consists of running and being restless with questions for years.
Unspooling means a thread coming off a roll, starting from a fixed point (the spool of thread or, in the metaphor, that moment in Orville's childhood) and extending outwards (in the metaphor, into the future of Orville's life). I found an illustrative gif of unspooling thread: