In this excerpt from Heart of darkness, by Joseph Conrad, that you can download from Project Gutenberg website, Marlow, the sailor protagonist of the story, is describing his feelings as going up the river (emphasis mine):
Going up that river was like traveling back to the earliest beginnings of the world, when vegetation rioted on the earth and the big trees were kings. An empty stream, a great silence, an impenetrable forest. The air was warm, thick, heavy, sluggish. There was no joy in the brilliance of sunshine. The long stretches of the waterway ran on, deserted, into the gloom of overshadowed distances. On silvery sand-banks hippos and alligators sunned themselves side by side. The broadening waters flowed through a mob of wooded islands; you lost your way on that river as you would in a desert, and butted all day long against shoals, trying to find the channel, till you thought yourself bewitched and cut off for ever from everything you had known once—somewhere—far away—in another existence perhaps. There were moments when one’s past came back to one, as it will sometimes when you have not a moment to spare for yourself; but it came in the shape of an unrestful and noisy dream, remembered with wonder amongst the overwhelming realities of this strange world of plants, and water, and silence.
I don't understand what the author means when he writes that "one’s past came back to one, as it will sometimes when you have not a moment to spare for yourself". Can someone explain it?