The story is set in Baltimore.
Just for reference, here's the text of the short story, entitled "Esquire" (from the Afternoon of an Author collection) and dated to August 1936.
The biggest chunk of information about the story's setting comes near the beginning:
"Yes, I certainly need to get out," he thought. "I'd like to drive down the Shenandoah Valley, or go to Norfolk on the boat."
But both of these ideas were impractical -- they took time and energy and he had not much of either -- what there was must be conserved for work.
The Shenandoah Valley is in the northeast of West Virginia state and the northwest of Virginia state. Norfolk is a coastal city in the east of Virginia state. So it seems like the setting of the story must be either Virginia or another nearby state. After consulting a map of the US states, I concluded that the most likely possibilities are Virginia, West Virginia, and Maryland.
Another big clue comes, as you mentioned, from the statue of Lafayette:
The bus was a long time coming but he didn't like taxis and he still hoped that something would occur to him on that upper-deck passing through the green leaves of the boulevard. When it came finally he had some trouble climbing the steps but it was worth it for the first thing he saw was a pair of high school kids, a boy and a girl, sitting without any self-consciousness on the high pedestal of the Lafayette statue, their attention fast upon each other.
There are a lot of Lafayette statues in the US, but only a few in the right part of the country. We can exclude the ones in Yorktown, Virginia (2005) and Havre de Grace, Maryland (1976) as being too recent, but the one in Baltimore, Maryland (1924) seems to fit perfectly. It existed when this story was written, and it has a high pedestal on which we could easily imagine kids sitting:
Finally, let's look at the passage you quoted in the question. I think this was the main red herring which made this question hard to solve:
The bus went past a pale Athenian railroad station brought to life by the blue shirted redcaps out in front.
This doesn't mean the place is a town called Athens in the US - it means the railroad station itself looks "Athenian", like something out of the original Athens! Just look at the main railroad station of Baltimore, Penn Station. Looks like it's built in the style of an ancient Greek temple, right? (Actually it was built in Beaux-Arts style, drawing upon French neoclassicism, but whatever.)
Going back to the first passage quoted above: according to Google it takes around 2 hours today to get from Baltimore to Shenandoah Valley by car, and in the 1930s there were regular steamships from Baltimore to Norfolk. Everything seems to be coming together for Baltimore as the location.
Finally, how does this fit in to the biographical history of the author? Well, it turns out that F. Scott Fitzgerald spent some unhappy years in Baltimore during the early 1930s. His wife was hospitalised there for a recurring mental illness, and he too was hospitalised many times as well as suffering financially while working on a semi-autobiographical novel. There's even an article "F. Scott Fitzgerald in Baltimore" published in Baltimore Style. All of this seems to fit perfectly with the idea that he wrote a short story in 1936 about an unhappy and struggling author living in Baltimore.
