Walkinshaw has just said that the two men who were with Markenmore were strangers. Chilford's response is:
You cannot be sure they were strangers. Markenmore might have known them from before, as he has business dealings with a wide range of people. You are assuming that Markenmore came here to meet his brother and sister. But it is equally probable that he came here to meet the two men.
Hope that clarifies the meaning a bit!