It's worth noting that Tolkien based all of his riddles in style, and some of them in actual wording, on classical "old literary" riddles. As mentioned in the answer to Who wrote Tolkien's riddles?:
As for the Riddles: they are 'all my own work' except for 'Thirty White Horses' which is traditional, and 'No-legs'. The remainder, though their style and method is that of old literary (but not 'folk-lore') riddles, have no models as far as I am aware, save only the egg-riddle which is a reduction to a couplet (my own) of a longer literary riddle which appears in some 'Nursery Rhyme' books, notably American ones.
-- Tolkien's Letters, Letter #110
So the theme is very much that of old, classical riddles, and thus it seems unlikely that there was any hard-and-fast ruleset as such. In ancient times people were generally less concerned with restricting themselves with multiple rules (we can see this also in the use of language, e.g. the way there were no standard English spelling rules until several hundred years ago, but I digress). To make the game fair and fun, it was enough for the riddle to be uniquely solvable - i.e. there's a solution which is deducible from the riddle as stated and which is the only word fitting it.
These are basically the same criteria as described by Mithrandir, fi12, and TML in their answers, but I can back them up as 'the' criteria for a solid classical riddle with citations from some of the internet's top experts on riddling, namely those at Puzzling SE:
-
a "solid" Riddle should have one and only one interpretation which coherently addresses each line/clue contained in the Riddle.
-
However hard the riddle a supermajority of fluent English (or whatever your target language may be) speakers must, when presented with the solution, consider it the only correct one.
-
The idea is to use as many different figurative or metaphoric descriptions as are necessary to ensure that the intersection of their interpretations is a single word or phrase.
The idea is always the same. Stylistic additions such as rhyme, rhythm, twists of meaning, and even metaphor are optional. The only real rule - insofar as there are any "rules" at all - is that the riddle should be uniquely solvable from the information provided, i.e. from the statement of the riddle itself. (And if they aren't, they'll be closed on Puzzling :-) )
Wrong," said Bilbo, who had luckily just taken his hand out again.
:P (I like pedantry, don't take it personally.)