John Forster's 'Life of Charles Dickens' published in several editions in the early 1870s includes the useful line
We are to observe also that it is never anything complete that is thus taken from life by a genuine writer, but only leading traits, or such as may give greater finish;
While this was written in regard to the identification of real life characters with fictional ones, it is also a useful caveat for buildings and other locations where there may be doubt over the identify of the original.
Forster does not reveal any specifics as to what building may have inspired Bleak House, focusing more on the Chancery case which gave the underlying plot. What is clear though is that Dickens didn't plant himself in one place to write a novel. Forster tells us:
Bleak House was begun in his new abode of Tavistock House at the end of November 1851; was carried on, amid the excitements of the Guild performances, through the following year; was finished at Boulogne in the August of 1853
The inability to "grind sparks out of his dull blade," as he characterized his present labour at Bleak House, still fretting him, he struck out a scheme for Paris. "I could not get to Switzerland very well at this time of year. The Jura would be covered with snow. And if I went to Geneva I don't know where I might not go to." It ended at last in a flight to Dover; but he found time before he left, amid many occupations and some anxieties, for a good-natured journey to Walworth to see a youth rehearse who was supposed to have talents for the stage
Able at last to settle to his work, he stayed in Dover three months; and early in October, sending home his family caravan, crossed to Boulogne
There is no mention of him writing the novel in Broadstairs, which is around 23 miles from Dover.
At this point it is probably useful to note that Forster was a friend of Dickens and according to Wikipedia at least, would have read all of his works in manuscript, as such he will not have been relying purely on published sources for his information in the Biography.
So Dickens was probably not writing Bleak House from within the body of a building which inspired it.
The 'Herts Memories' website which is the source wikipedia gives for the location of Bleak House's inspiration in St Alban's seems to rely on the knowledge that Dickens was known to visit Knebworth House, rather than any more direct association. The putative location in ST Alban's lies less than a quarter of a mile from the town's centre, while Bleak House is not in St Alban's but near it. Even allowing for subsequent development subsuming outerlying areas, the distance described in the novel would not be so contracted:
These delays so protracted the journey that the short day was spent and the long night had closed in before we came to St. Albans, near to which town Bleak House was, we knew.
By that time we were so anxious and nervous that even Richard confessed, as we rattled over the stones of the old street, to feeling an irrational desire to drive back again. As to Ada and me, whom he had wrapped up with great care, the night being sharp and frosty, we trembled from head to foot. When we turned out of the town, round a corner, and Richard told us that the post-boy, who had for a long time sympathized with our heightened expectation, was looking back and nodding, we both stood up in the carriage (Richard holding Ada lest she should be jolted down) and gazed round upon the open country and the starlight night for our destination. There was a light sparkling on the top of a hill before us, and the driver, pointing to it with his whip and crying, "That's Bleak House!"
British History Online carries the text of a 1908 history of Hertfordshire which states:
On the side of St. Peter's Street opposite to the entrance to the Hatfield Road, is Catherine Lane, in which are some old cottages, and in continuation is Folly Lane, where is an eighteenth-century house, the residence of Mr. A. J. Dorell, called Folly House, or Bleak House, which is supposed to be the house from which Charles Dickens named his novel, although the description given by Dickens does not correspond to the present building.
'The city of St Albans: Introduction', in A History of the County of Hertford: Volume 2, ed. William Page (London, 1908), pp. 469-477. British History Online [accessed 17 August 2021].
perhaps understandably that book does not give a source for the supposition, but the writing suggest that it is to be taken with a pinch of salt.
Returning to Forster's Biography, in Footnote 169 he appends various working titles that the work went through:
I subjoin the dozen titles successively proposed for Bleak House. 1. "Tom-all-Alone's. The Ruined House;" 2. "Tom-all-Alone's. The Solitary House that was always shut up;" 3. "Bleak House Academy;" 4. "The East Wind;" 5. "Tom-all-Alone's. The Ruined [House, Building, Factory, Mill] that got into Chancery and never got out;" 6. "Tom-all-Alone's. The Solitary House where the Grass grew;" 7. "Tom-all-Alone's. The Solitary House that was always shut up and never Lighted;" 8. "Tom-all-Alone's. The Ruined Mill, that got into Chancery and never got out;" 9. "Tom-all-Alone's. The Solitary House where the Wind howled;" 10. "Tom-all-Alone's. The Ruined House that got into Chancery and never got out;" 11. "Bleak House and the East Wind. How they both got into Chancery and never got out;" 12. "Bleak House."
The sequence of these may be read to suggest that the quality of bleakness was intrinsic to the character of the place, and hence may not have needed to be borrowed from any factual house, though it does not rule it out.
It would probably take a dissertation and access to many more documents to arrive at an unchallengeable assertion as to the origins of any house that inspired the one in the novel, but it seems likely that Ford House may have played into it, being a house of appropriate scale with which Dicken's was familiar, but also that he was a well and widely travelled man who would have been introduced to many and varies houses both as a visitor and a passerby and likely drew on houses he knew as well as houses he had read about and only imagined in his construction of Bleak House. It does appear however that the house on Folly Lane St Alban's has only a circumstantial and tenuous claim, which was being written of in doubting terms a mere half century or so after the publication of the work.