To avoid this question being overly broad, I will connect it to a specific book. In Catcher in the Rye, Holden frequently wonders where the ducks in Central Park go for the winter. When I read the book I felt that this idea made him a more rounded character. It showed he was naive and uninformed, but also that he cares for the well-being of others, especially more vulnerable creatures. Various online sources suggest that the ducks symbolize his desire to protect innocence and/or his fear of change.
But this led me to wonder about symbols in a larger sense. How do we ever know if an author truly intended something as a symbol? And how do we know we are interpreting it as they intended? Couldn't an overly zealous reader point to a bunch of random items in a book like this and call them symbols, to the point of it being ridiculous? Like what if I said that taxicabs symbolize his free will, and the carbonation bubbles in Mr. Antolini's highballs symbolize Holden's ability to keep rising to the surface after being knocked down? And maybe the girl's skates symbolize the cyclical nature of the world? How are these rather absurd "symbols" to be shown less valid than the ducks?
TL;DR Without the author specifically saying that a certain item symbolizes another, what makes some symbolic interpretations more valid than others?