The implication seems to be that Dr Buford preferred to spend time gardening instead of being a social climber. We don't have much more information about him in the story, so this passage is nearly all we have to go on, but:
"his obsession was anything that grew in the ground"
This suggests he was a keen gardener, like his daughter Miss Maudie:
Miss Maudie hated her house: time spent indoors was time wasted. She was a widow, a chameleon lady who worked in her flower beds in an old straw hat and men’s coveralls, but after her five o’clock bath she would appear on the porch and reign over the street in magisterial beauty.
She loved everything that grew in God’s earth, even the weeds. With one exception. If she found a blade of nut grass in her yard it was like the Second Battle of the Marne: she swooped down upon it with a tin tub and subjected it to blasts from beneath with a poisonous substance she said was so powerful it’d kill us all if we didn’t stand out of the way.
"so he stayed poor"
We are getting an impression of a man who spent time on his passions instead of on his work and making money. Instead of becoming rich and famous, he would rather stay at home and do gardening because he enjoyed it.
The point of this passage is to contrast Dr Buford with Jack Finch. Like Dr Buford, both Jack Finch and his original ancestor Simon had professions in medicine; but Simon Finch "made a pile practicing medicine", thus enabling his family to remain reasonably well-off for generations:
my father, Atticus Finch, went to Montgomery to read law, and his younger brother went to Boston to study medicine. [...] During his first five years in Maycomb, Atticus practiced economy more than anything; for several years thereafter he invested his earnings in his brother’s education. John Hale Finch was ten years younger than my father, and chose to study medicine at a time when cotton was not worth growing; but after getting Uncle Jack started, Atticus derived a reasonable income from the law.
If Jack Finch had spent as much time as possible in his garden, he would not have been able to make a very successful career. Thus the contrast between him and Dr Buford:
Dr Buford’s profession was medicine and his obsession was anything that grew in the ground, so he stayed poor. Uncle Jack Finch confined his passion for digging to his window-boxes in Nashville and stayed rich.
In other words: Dr Buford practised medicine as a job, but preferred enjoying himself in the garden, so he didn't "make a pile" like Simon Finch; by contrast, Jack Finch did gardening only in small window-boxes and had time to make a more successful career.
