He might be addressing Frodo more as a generic hobbit than as himself.
This is immediately after the following short speech from Frodo:
"No, Sam!" said Frodo. "Do not kill him even now. For he has not hurt me. And in any case I do not wish him to be slain in this evil mood. He was great once, of a noble kind that we should not dare to raise our hands against. He is fallen, and his cure is beyond us; but I would still spare him, in the hope that he may find it."
It is Frodo's knowledge of Saruman's nature, his level-headed wisdom in telling the others to spare his life, and his easy, unconscious claiming of the moral high ground that causes Saruman's "wonder and respect and hatred". But is it Frodo saying this that surprises Saruman so, or just the fact that any hobbit had such wisdom in him?
Unlike Gandalf, Saruman has never been one to respect small things or see the greatness in them. He was always ambitious, his mind on high affairs, and hobbits were beneath his attention until he found out about the Ring. While Gandalf the wanderer was already a known face in the Shire even before the events of The Hobbit, let alone The Lord of the Rings, Saruman remained in his tower at Isengard, too proud to concern himself with lesser beings. Even after he knew a hobbit was the Ring-bearer, I doubt he appreciated the inner strength and resilience of these little people.
Thus we can see why it would be a surprise to him for any hobbit to speak such noble and insightful words as Frodo did. He thought of them as insignificant, unworthy of any respect, and suddenly one of them rears up and talks like a lord of men. Saruman, then, stares at this upstart, who must surely have "grown very much" from a Halfling's humble roots to be able to address an Istari so.
Perhaps significantly, it is precisely Saruman's lack of respect towards those he considers beneath him which causes his death only moments later. Despite all they've been through together, he doesn't respect Wormtongue but kicks him and treats him like a dog. If he'd been kinder to his last loyal companion, or even had enough respect for him to realise that he was capable of "not being able to stand it any more", he would have lived a longer life.
Or he might have other ways of gaining information about people.
We already know the power of the Voice of Saruman, the power to cajole and persuade people beyond what their reason tells them. And this Istari has other tricks up his sleeve as well, even in his weakened state. As we saw back in The Two Towers:
"But come now," said the soft voice. "Two at least of you I know by name. Gandalf I know too well to have much hope that he seeks help or counsel here. But you, Théoden Lord of the Mark of Rohan, are declared by your noble devices, and still more by the fair countenance of the House of Eorl. O worthy son of Thengel the Thrice-renowned! Why have you not come before, and as a friend? Much have I desired to see you, mightiest king of western lands, and especially in these latter years, to save you from the unwise and evil counsels that beset you! Is it yet too late? Despite the injuries that have been done to me, in which the men of Rohan, alas! have had some part, still I would save you, and deliver you from the ruin that draws nigh inevitably, if you ride upon this road which you have taken. Indeed I alone can aid you now."
Théoden opened his mouth as if to speak, but he said nothing. He looked up at the face of Saruman with its dark solemn eyes bent down upon him, and then to Gandalf at his side; and he seemed to hesitate. Gandalf made no sign; but stood silent as stone, as one waiting patiently for some call that has not yet come. The Riders stirred at first, murmuring with approval of the words of Saruman; and then they too were silent, as men spell-bound. It seemed to them that Gandalf had never spoken so fair and fittingly to their lord. Rough and proud now seemed all his dealings with Théoden. And over their hearts crept a shadow, the fear of a great danger: the end of the Mark in a darkness to which Gandalf was driving them, while Saruman stood beside a door of escape, holding it half open so that a ray of light came through. There was a heavy silence.
It was Gimli the dwarf who broke in suddenly. "The words of this wizard stand on their heads," he growled, gripping the handle of his axe. "In the language of Orthanc help means ruin, and saving means slaying, that is plain. But we do not come here to beg."
"Peace!" said Saruman, and for a fleeting moment his voice was less suave, and a light flickered in his eyes and was gone. "I do not speak to you yet, Gimli Glóin’s son," he said. "Far away is your home and small concern of yours are the troubles of this land. But it was not by design of your own that you became embroiled in them, and so I will not blame such part as you have played – a valiant one, I doubt not. But I pray you, allow me first to speak with the King of Rohan, my neighbour, and once my friend."
-- The Two Towers, Book III, Chapter 10: "The Voice of Saruman"
The most obvious tool Saruman is using here is his Voice, his power of persuasion, in a last-ditch attempt to hypnotise Theoden and the men of the Mark into his thrall. But he also seems to have some power of insight or recognition. How does he know Gimli's name and origins? As far as we know, he's never met the dwarf before, and by the time the Fellowship of the Ring was formed, Saruman's treachery was well known. He must have conduits of information of which we know nothing. Perhaps he used the same sources to learn about Frodo, long before meeting him face to face.
What means does he use to spy on people and find out about them before he meets them? Well, it could be some secret Maiar power, some art of divining a person's name and nature from a glance at their face ... or it could be as simple a tool as spies. We know that at least one of his servants - and a trusted one at that - had encountered Frodo back in Bree in The Fellowship of the Ring:
Some while ago one of Saruman's most trusted servants (yet a ruffianly fellow, an outlaw driven from Dunland, where many said that he had Orc-blood) had returned from the borders of the Shire, where he had been negotiating for the purpose of "leaf" and other supplies ... He was the squint-eyed southerner at the Inn.
-- Unfinished Tales, "The Hunt for the Ring"
And we know that he, like Sauron, could use birds as his agents to scout for him. Perhaps these spies of his, human or avian, were Saruman's tools for gathering information, so that he always appeared well-informed and knew much about people before he ever set eyes on them.
Much of this answer was inspired by the excellent answer on another SE site from Darth Melkor (user8719), although the words and some of the ideas are my own.
