First, the afternoon/evening contrast doesn't have any substantive meaning (the key to understanding the phrase doesn't have anything to do with something occurring in the afternoon vs. something occurring in the evening): the phrase could just as easily have been, "To be clever one day argues that one is dining nowhere the next."
As for an interpretation of the phrase, it helps to look at the context of the Reginald dialogues as Saki created them and the character of Reginald and this dialogue ("Reginald on the Academy") in particular. The dialogues often consist of witty repartee which grant the reader, through dramatic irony, to see one of the interlocutors taken down in a way that the interlocutor him or herself isn't even aware of.
The phrase in question is best understood as a benign jab at Reginald made by 'the Other', one that in a case of dramatic irony seems to go right over Reginald's head. Reginald has just been sort of "humblebragging" by repeating the clever retort he made earlier in the day about happiness in life being avoidance of the unattainable (which makes the phrase originally asked about doubly ironic given that he then starts fishing for an invitation to dinner that he doesn't get). The woman at the academy, he also repeats, said something about (presumably Reginald being) 'so clever'. The Other takes Reginald down a notch by uttering the phrase in question.
The meaning is simply that smart people, yes, but even more people who think themselves smart and witty (and 'clever') aren't usually or even often the people that one likes to be around and have at dinners, which receives its brutal confirmation a couple of lines later when Reginald asks about that invitation to dinner the Other presumably had extended to Reginald, only to be informed by the Other that there was no such invitation.