According to the book Retrouvailles, by Claudette Combes and Patrick Marcadet, found in Google books, the quote comes from the book Mémoires d'Alexandre Golden Veiser.
Alexandre Golden Veiser is almost certainly Alexander Borisovich Goldenweiser, a Russian composer who, according to Wikipedia, was a good friend of Leo Tolstoy. According to Russian Wikipedia, as far as I can tell (I don't speak any Russian), he published two memoirs of his relationship with Tolstoy: Вблизи Толстого (Vblizi Tolstogo,) and Лев Толстой и музыка (Leo Tolstoy and Music), and I would assume that this quote is in one of them.
So given that Goldenweiser and Tolstoy were both Russian, and Goldenweiser's book is written in Russian, I would assume that the original of the quote is in Russian.