This appears to be a frequently asked question for Keep the Aspidistra Flying, and while there's no fully confirmed answer, there is at least a prominent theory. I found a few old Usenet (Google Groups) discussions about it, putting forward and supporting some theories. The most popular theory is:
In Horace's Ars Poetica comes the line Parturient montes, nascetur
ridiculus mus which means "The mountains will go into labour and will
give birth to -- a ridiculous mouse."
This is in a section where Horace is talking of the difficulties for a
poet of being original. Translation: "...A theme that is familiar can be
made your own as long as you do not waste your time on a hackneyed
treatment....do not in imitating another writer plunge yourself into
difficulties from which shame, or the rules you have laid down for
yourself, prevent you from extricating yourself...What will emerge that
can live up to your extravagant promise? The mountains will go into
There are several friendly refs to Horace in Orwell's letters and
writing (thanks for info, Martha) so this might be a relevant meaning,
bearing in mind Gordon's poetic self-disgust. It [would] presumably also
please him that the moneyed young beasts from Cambridge didn't recognise
a fairly well-worn classical allusion. The book-title [would] refer to
Gordon's sense that his poems are merely portentous miscarriages of his
This is from one of two threads mentioned here/here which supposedly give the solution to the Mice question. The second one is a dead link, unfortunately.
The same Horace interpretation is given in the endnotes to the 2021 edition of Keep the Aspidistra Flying, in which the "Editorial material" is copyrighted to Benjamin Kohlmann, a professor in British Studies at the University of Hamburg:
the title of Gordon's remaindered poetry collection hints at its artistic insignificance. It is also possible that the title contains a more specific allusion to a passage from the Latin poet Horace's Ars Poetica, a treatise on the art of poetry: 'Quid dignum tanto feret hic promissor hiatu? | Parturiunt montes, nascetur ridiculus mus' ('What might he produce that could match his early promise? | The mountains go into labour and will give birth to a ludicrous mouse'). Horace's proverbial lines reflect on the possibility of artistic failure and the dangers of poetic pretentiousness. Orwell, who had received training in Classics at St Cyprian's and Eton, was certainly familiar with these lines.