The "neon god" is obviously the sign pictured earlier in the song. But why is it a god?
The sign is a god because people made it a god ("the neon god they made"). In praying and bowing to the sign, they made it into a god. What the sign represents, though, is harder to answer. Many interpretations I've found have said it represented advertising and TV.
According to Garfunkel (this quote is from Wikipedia, but it's directly quoted from a book on Paul Simon's life):
Garfunkel once summed up the song's meaning as "the inability of people to communicate with each other, not particularly internationally but especially emotionally, so what you see around you are people unable to love each other."
It seems the song is about people's inability to communicate with all of the television watching and stuff; people who don't have any kind of original thought. I haven't found any direct quotes on the meaning other than Garfunkel's above, so you're just going to have to take what you can get.
However, this site cites Simon as saying this in an interview with NPR:
It's not a sophisticated thought, but a thought that I gathered from some college reading material or something. It wasn't something that I was experiencing at some deep, profound level - nobody's listening to me, nobody's listening to anyone - it was a post-adolescent angst, but it had some level of truth to it and it resonated with millions of people. Largely because it had a simple and singable melody."
It sounds like he just wrote the lyrics, then realized that they fit nicely with this theme.