The Defeat of the City
It's the protagonist who issues the challenge, but the other details seem to match:
That night when the greetings and the supper were over, the entire family, including Buff, the yellow dog, bestrewed itself upon the front porch. Alicia, not haughty but silent, sat in the shadow dressed in an exquisite pale-gray tea gown.
Robert tore off his coat and vest and hurled them into a lilac bush.
“Come out here, you landlubber,” he cried to Tom, “and I’ll put grass seed on your back. I think you called me a ‘dude’ a while ago. Come along and cut your capers.”
Tom understood the invitation and accepted it with delight. Three times they wrestled on the grass, “side holds,” even as the giants of the mat. And twice was Tom forced to bite grass at the hands of the distinguished lawyer. Dishevelled, panting, each still boasting of his own prowess, they stumbled back to the porch.
Then Alicia moved as though she were about to speak, but she did not. Through it all she sat immovable, a slim, white spirit in the dusk that no man might question or read.
By and by she asked permission to ascend to her room, saying that she was tired.
“Robert,” said the calm, cool voice of his judge, “I thought I married a gentleman.”
Yes, it was coming. And yet, in the face of it, Robert Walmsley was eagerly regarding a certain branch of the apple tree upon which he used to climb out of that very window. He believed he could do it now. He wondered how many blossoms there were on the tree—ten millions? But here was some one speaking again:
“I thought I married a gentleman,” the voice went on, “but—”
Why had she come and was standing so close by his side?
“But I find that I have married”—was this Alicia talking?—“something better—a man—Bob, dear, kiss me, won’t you?”