Fitzgerald's story "'O Russet Witch!'" appears in his collection Tales of the Jazz Age. The quotation marks are part of the story's title; it's the only story in the collection to sport this typographical feature.
This suggests to me that maybe the phrase "O Russet Witch!" is a quotation from some other work, maybe a poem or a play. However, I've been unable to find any source for this phrase beside the Fitzgerald story.
Of course, Fitzgerald did like to play games. The poem-epigraph of The Great Gatsby is attributed to the fictitious poet Thomas Park D'Invilliers, a character in This Side of Paradise.