It means they live very close to poverty, and in a rather inconspicuous manner. Here is a definition from the Free Dictionary:
in a small way
In an [sic] way that is not elaborate or significant.
She doesn't want to have a big birthday party, so we're just going to celebrate in a small way this weekend.
The key words that capture the Bates's situation are "not elaborate or significant." When the late Mr Bates was vicar, they would naturally be a family of some significance in Highbury. Consider how Mr Elton is regarded socially in the village: though not independently wealty, he is invited as an equal to the homes of the rich and powerful, such as Mr Woodhouse and Mr Knightley; he sets his sights on marrying Emma, the girl in the village with the most money and the highest social status; that potential match is considered unsurprising for the most part (except, of course, by Emma herself, who considers him beneath her); and he does in fact eventually marry a wealthy if vulgar woman, whose marriage to him is considered a step up socially for her, because her family made its money in (eek) trade rather than inherited real property.
The death of Mr Bates would mean the loss of any income from his role as vicar, so they have to make do with very little money. From their relatively exalted position, therefore, the Bateses have fallen into poverty. Their means do not match their social status any more, and rather than trying to keep up with their former income and consequence, they live in a small way.
The heartbreak of Miss Bates's situation, in particular, is noted by Mr Knightley when he remonstrates with Emma for having mocked her to her face:
Were she your equal in situation—but, Emma, consider how far this is from being the case. She is poor; she has sunk from the comforts she was born to; and, if she live to old age, must probably sink more. Her situation should secure your compassion. It was badly done, indeed! You, whom she had known from an infant, whom she had seen grow up from a period when her notice was an honour, to have you now, in thoughtless spirits, and the pride of the moment, laugh at her, humble her—and before her niece, too—and before others, many of whom (certainly some,) would be entirely guided by your treatment of her. [emphasis added]
There was a time when to be taken notice of by Miss Bates was considered an honor; now, she lives inconspicuously, in a small way, having lost the means to live honorably in any other way.