In this answer, I'm going to address only the speculation from the last paragraph of the post:
Perhaps, the limited third-person point of view a recent innovation, and unheard-of 100 years ago
This is far from the case: in fact, by the early 20th century the limited third-person point of view was well along the road to taking over English fiction.
So in this answer, I'm going to summarize the development of the limited third-person point of view, based on the account in Wayne Booth's The Rhetoric of Fiction (1961), which I commend to you. In what follows, if I don't give a citation for a point you can assume that I got it from Booth. Necessarily this is going to have to skip over a mass of detail.
Austen
A point Booth makes is that there isn't a sharp distinction betweeen "omniscient" and "limited" modes of third-person narration, but rather, there is a continuum in the degree of omniscience of the narrator. Early third-person novels, such as Henry Fielding's The History of Tom Jones, a Foundling (1749), freely used the narrative voice to jump between scenes and characters, and to discurse and soliloquize directly to the reader.
But already by Jane Austen's Emma (1813) there is a more sparing use of the narrative voice. The narrative begins in omniscient mode, passing moral judgment on the characters:
The real evils, indeed, of Emma’s situation were the power of having rather too much her own way, and a disposition to think a little too well of herself; these were the disadvantages which threatened alloy to her many enjoyments. The danger, however, was at present so unperceived, that they did not by any means rank as misfortunes with her.
and showing the viewpoints of different characters:
The happiness of Miss Smith was quite equal to [Emma's] intentions. Miss Woodhouse was so great a personage in Highbury, that the prospect of the introduction had given as much panic as pleasure; but the humble, grateful little girl went off with highly gratified feelings, delighted with the affability with which Miss Woodhouse had treated her all the evening, and actually shaken hands with her at last!
Nonetheless, nearly all the scenes are ones where Emma is present and a witness or participant in the events and dialogue, and gradually the narrative settles on Emma's point of view almost exclusively:
When it came to such a pitch as this, she was not able to refrain from a start, or a heavy sigh, or even from walking about the room for a few seconds—and the only source whence any thing like consolation or composure could be drawn, was in the resolution of her own better conduct, and the hope that, however inferior in spirit and gaiety might be the following and every future winter of her life to the past, it would yet find her more rational, more acquainted with herself, and leave her less to regret when it were gone.
Booth points out that this narrative strategy has a particular rhetorical effect. In Emma there is a delicate balancing act between impartial judgment of Emma's faults and sympathy for her predicament. The omniscience of the opening helps establish the former and the gradual adoption of Emma's point of view the latter.
Ford
By the early 20th century, this flexible use of the omniscient narrator came to be criticised as an unrealistic approach that interfered with the reader's suspension of disbelief. Ford Madox Ford presents this criticism in The English Novel: from the Earliest Days to the Death of Joseph Conrad (1930). First, the omniscient narrator draws the fictionality to the reader's attention:
The struggle—the aspiration—of the novelist down the ages has been to evolve a water-tight convention for the framework of the novel. He aspires—and for centuries has aspired—so to construct his stories and so to manage their surfaces that the carried-away and rapt reader shall really think himself to be in Brussels on the first of Waterloo days or in Grand Central Station waiting for the Knickerbocker Express to come in from Boston though actually he may be sitting in a cane lounge on a beach of Bermuda in December. This is not easy. […]
[Samuel] Richardson, going a good deal further, has left it on record that he was actually bothered by the problem of the novelistic convention and that he racked his brain a long time before arriving at the one he finally adopted [i.e., the epistolary novel]. He asked himself, that is to say, how the reader was to be convinced that the author—and by analogy still more his characters—how could they know all the details that go to making up a book? If, to reduce the matter to its most elementary form, Sir Charles Grandison is walking in the Yew Walk, how can he know what characters are present and what conversations are being carried on in the Cedar Parlour, and since, to satisfy the reader, the author is to be supposed to be cognizant of all that passes in his novel, how is he to know simultaneously what is happening in both places?
Second, the moral judgments of the omniscient narrator prevent readers from making their own judgments:
At any rate, with or after Stendhal, it became evident that, if the novel was to have what is called vraisemblance [verisimilitude], if it was so to render life as to engross its reader, the novelist must not take sides either with the virtuous whose virtues cause them to prosper or with the vicious whose very virtues drive them always nearer and nearer to the gallows or the pauper's grave. […] So that it is not so much the function of the novelist to hold the balance straight as, dispensing with all scales or instruments for measuring, to show all the human beings of his creation going about their avocations. He has, that is to say, to render and not to tell.
James
Henry James adopted a particular solution to these problems, the "lucid reflector" (the phrase comes from his Notes on Novelists, 1914). In the preface to The Ambassadors (1903) he described his solution to the problems of realism and objectivity (emphasis mine):
… every question of form and pressure, I easily remember, paled in the light of the major propriety, recognised as soon as really weighed; that of employing but one centre and keeping it all within my hero's compass. The thing was to be so much this worthy's intimate adventure that even the projection of his consciousness upon it from beginning to end without intermission or deviation would probably still leave a part of its value for him, and a fortiori for ourselves, unexpressed. I might, however, express every grain of it that there would be room for—on condition of contriving a splendid particular economy. Other persons in no small number were to people the scene, and each with his or her axe to grind, his or her situation to treat, his or her coherency not to fail of, his or her relation to my leading motive, in a word, to establish and carry on. But Strether's sense of these things, and Strether's only, should avail me for showing them; I should know them but through his more or less groping knowledge of them, since his very gropings would figure among his most interesting motions, and a full observance of the rich rigour I speak of would give me more of the effect I should be most "after" than all other possible observances together.
To précis this characteristically Jamesian paragraph, he decided to introduce a character (Strether) who was an observer of the main action of the story, and ensured that the narrative consisted only of Strether's observations and reflections on them. (Booth notes that this decision had a warping effect on the novel. We know from James's notes that he intended the subject of The Ambassadors to be the romance of Chad Newsome and Marie de Vionnet. But by introducing Strether as the "reflector" and limiting the narration to Strether's point of view, the main subject changed to that of Strether's intermediation between the other characters.)
The victory of limited narration was essentially complete by the mid-20th century. Ford's "to render and not to tell" became the modern writing-class orthodoxy "show, don't tell". But Booth points out that this limits the kinds of stories that writer can tell. In particular, without a reliable omniscient narrator to guide them, the reader might fail to understand the writer's intentions, for example the subtle balance between judgment and sympathy in Emma might be missed.
