Taking a class on existentialism and literature and I'm curious how far back the idea has been referenced in literature. What is the first work of literature to express the ideas of existentialism?
I would say that it is very hard to nail down the precise first work or thinker to reference Existentialist ideas. The Encyclopedia Britannica makes references to 16th and 17th century thinkers including Pascal and De Montaigne as precursors to Existentialism, making references to their application of the Socratic method.
The problem of what humans are in themselves can be discerned in the Socratic imperative “know thyself,” as well as in the work of the 16th-century French essayist Michel de Montaigne and Blaise Pascal, a 17th-century French religious philosopher and mathematician. Montaigne had said: “If my mind could gain a foothold, I would not write essays, I would make decisions; but it is always in apprenticeship and on trial.” And Pascal had insisted on the precarious position of humans situated between Being and Nothingness: “We burn with the desire to find solid ground and an ultimate sure foundation whereon to build a tower reaching to the Infinite. But our whole groundwork cracks, and the earth opens to abysses.” 1
In general, both the Encyclopedia Britannica and Wikipedia agree that the most direct antecedents to Existentialism were Nietzsche and Kierkegaard
Søren Kierkegaard and Friedrich Nietzsche were two of the first philosophers considered fundamental to the existentialist movement, though neither used the term "existentialism" and it is unclear whether they would have supported the existentialism of the 20th century. They focused on subjective human experience rather than the objective truths of mathematics and science, which they believed were too detached or observational to truly get at the human experience. Like Pascal, they were interested in people's quiet struggle with the apparent meaninglessness of life and the use of diversion to escape from boredom.2
From a literary perspective, the first and most significant proponents of an Existentialist philosophy were Dostoevsky and Sartre.
The first important literary author also important to existentialism was the Russian Fyodor Dostoyevsky. Dostoyevsky's Notes from Underground portrays a man unable to fit into society and unhappy with the identities he creates for himself. Jean-Paul Sartre, in his book on existentialism Existentialism is a Humanism, quoted Dostoyevsky's The Brothers Karamazov as an example of existential crisis. Sartre attributes Ivan Karamazov's claim, "If God did not exist, everything would be permitted" to Dostoyevsky himself, though this quote does not appear in the novel. 3
According to the Encyclopedia Britannica, the father of Existentialism is Dane Søren Kierkegaard.
The theme of the irreducibility of existence to reason, common to many existentialists, was also defended by the German idealist F.W.J. von Schelling as he argued against G.W.F. Hegel in the last phase of his philosophy; Schelling’s polemic, in turn, inspired the thinker usually cited as the father of existentialism, the religious Dane Søren Kierkegaard.