I think there were several factors here, though of course we don't know all of them.
Not really niche
Firstly, I'd like to address the point of audience. While you say
The audience of his brilliant books are Jews, old Jews, Judeophiles and maybe 1% just randoms
In fact, he also wrote English translations of a lot of his works which were of course originally published in Yiddish, meaning the general public could access them. According to Wikipedia
English translations of dozens of his stories were frequently published in popular magazines such as Playboy and Esquire
meaning that they did have a wider audience than just those who could read Yiddish or who sought out his works. Singer was also well known because he was one of the leaders of the Yiddish literary movement.
Doesn't have to be popular
As Time Magazine puts it:
the Swedish Academy has always seemed to swing between wildly popular writers (William Golding, Gabriel García Márquez and Toni Morrison) and those who are more niche (Eyvind Johnson and Harry Martinson)
Even if Singer's work wasn't very well known, the popularity of the author's work isn't a requirement for being picked.
What the committee says
Finally, I might point out that the Nobel committee itself issues a statement explaining its reasoning:
for his impassioned narrative art which, with roots in a Polish-Jewish cultural tradition, brings universal human conditions to life.
Finally, from Singer's own acceptance speech:
The high honor bestowed upon me by the Swedish Academy is also a recognition of the Yiddish language
and indeed, the prize helped propel the Yiddish literary movement.