Are there any good, comprehensive primary source readers for criticism of Shakespeare and his individual plays, where the criticism is from before the 20th century?
The introductions of most modern editions of the plays will summarise past reception of the plays, with a few quotes. Are there any collections that just reprint these documents?
Collections that reprint criticism normally don't focus on pre-twentieth-century criticism; in modern collections, 20th-century criticism typically dominates. One of the best series that reprint criticism is the Macmillan Casebooks, which contains, for example, the following volumes:
The Casebook series and its successor, the "New Casebooks", don't cover each Shakespeare play. The New Casebooks series appears to focus specifically on 20th-century criticism (e.g. Hamlet, edited by Martin Coyle, and King Lear, edited by Kiernan Ryan and Shakespeare's History Plays: Richard II to Henry V edited by Graham Holderness).
Also worthwhile is the series William Shakespeare: The Critical Heritage Volume, edited by Brian Vickers, which is organised in chronological order rather than play-by-play:
Note that this series focuses on Shakespeare criticism and reception in England.
The volumes in Bloomsbury's Arden Early Modern Drama Guides all have "A Critical Reader" in the subtitle but don't reprint any pre-twentieth-century criticism.
The Norton Critical Editions mentioned in llywrch's answer contain some excerpts from existing criticism but these texts tend to be incomplete. (And the series does not cover all of Shakespeare's plays, though one day, it might.) For example, the edition of The Tempest (edited by Peter Hulme and William S. Sherman, 2004) contains excerpts of criticism by John Dryden, Nicholas Rowe, Samuel Taylor Coleridge, Ludwig Tieck and Fanny Kemble (and a much larger sample of 20th-century criticism). The edition of Macbeth (edited by Robert S. Miola, 2014) contains criticism by Simon Forman (a contemporary eyewitness account), Samuel Johnson, Samuel Taylor Coleridge and Thomas de Quincey. The edition of Hamlet (edited by Cyrus Hoy, 1992) contains excerpts from John Dennis, Samuel Johnson, William Richardson, Henry Mackenzie, Johann Wolfgang Goethe, August Willhelm Schlegel, Samuel Taylor Coleridge and William Hazlitt (and a larger sample of 20th-century criticism).
For other pre-twentieth-century criticism (or similar engagements with Shakespeare), it is worth seeking out specific authors, such as the following, listed in chronological order:
While I'm not sure what you mean by "comprehensive primary source readers for criticism of Shakespeare," to answer your primary question: One collection of pre-20th century Shakespeare criticism focusing on individual plays is William Hazlitt's "Characters of Shakespeare's Plays" (1817). Hazlitt deals with 35 plays and others of which he considers Shakespeare's authorship doubtful. Hazlitt is (just my opinion, of course) clever, insightful and a pleasure to read.
The publishing house W.W. Norton publishes editions of the usual titles one would study in school, supplemented with critical essays. (Looking at their website, I see they offer one about Shakespeare, but I can't get a look at the table of contents to see what essays are included.) Of course, essays after 1900 would be included, & these are often abridged, but at least this would give you a place to start.
Another place to look would be the books in the public domain at the Internet Archive. If you know the author's name, or the name of the essay, you can find the entire critical work there. But this would not not provide you with a collection, merely a hunting-ground to seek this material.
