Conan Doyle is giving the reader a clue as to Holmes' character. Educated readers at the time of publication would have picked up on these clues.
At that time, Baker Street was an upmarket residential part of London. To the western side - so not industrial, north of the river - so more fashionable, centrally placed near to major roads and railway stations - so good for travel, not in the really expensive parts of the metropolis - so Holmes is not rich.
There are other famous - real - people who lived in Baker Street around this time or earlier, including H.G.Wells and William Pitt the Younger.
In order for someone to be able to afford rooms in Baker Street, he would have needed to be comfortably off.
All the streets and districts of London mentioned in the Sherlock Holmes stories existed at the time. So it makes sense for the author to pick a real street with a certain cachet to round out the character.