"Digital rights management", DRM, is almost a standard in the e-book industry. Have copyright holders ever tried to protect physical books from scanning, for example in a way how banknotes are protected against counterfeiting?
Look at it this way - the reason why companies need DRM is because digital files can always be copied verbatim, i.e. bit for bit. Companies cannot prevent files from being copied and re-distributed so they need DRM - a system that alters the files in a way that the contents can only be accessed under specific circumstances, e.g. by entering a license key, inserting an official CD, etc.
With hard copies, copying "files" is significantly harder, because one has to get a hold of the physical copy of the product. What was equal to hitting the 'download' button, now requires taking a stroll to the library, or paying money for an actual copy.
Oh, and some more diligent copy centres will not allow you to copy more than a certain per cent of the book. While this doesn't apply to every country, it may still be an obstacle.
Coming back to your actual question, what you are describing is called, per Wikipedia, the "analog hole". While it may be nigh-impossible to decipher a DRM-protected book, or crack a DRM-protected game, once you have a physical copy of an object there is a number of ways you can copy it. Off top of my head, I can already think of a "lamp + tripod + DSLR camera stand", which is not very hard to construct, but is good for practical purposes. Combine this with optical character recognition, and you have a fault-proof home-made analog piracy system. Why fault-proof? Because digital cameras use the same working principle as the human eye, so DRM, in this case, will be more of a double-edged stick than it is with digital content.
There are multiple technologies available, that make traditional scanning and copying impossible. For instance, there is a technology that will make "VOID" texts appear on scanned copy:
All of our paper products are protected by AuthentiGuard Pantograph 4000TM. This patented technology involves the printing of hidden text messages ("VOID", "COPY", "UNAUTHORIZED COPY", etc.) into the background of a document. Text information or other artwork is placed onto this paper. Then, when this "original" is copied of scanned, the hidden word pattern becomes visible on the duplicate. The end result - a clearly distinguishable copy that can not pass off as the original.
Same company sells special paper for books as well, though I was unable to find examples of it being used,
On a more light-hearted note, there is a book that is printed with a special kind of ink: it is sold in an air-tight package, as the ink starts to fade upon contact with air. It is called El Libro Que No Puede Esperar - The Book That Can't Wait, This provides a good incentive to actually read the book, before it becomes blank paper.
This isn't against scanning specifically, but creators of ancient and medieval texts did attempt to protect their work against theft, vandalism and misattribution by writing curses on the book. You might consider this a form of early DRM.
For example, from the Wikipedia article on Book curses:
[...] Ashurbanipal, King of Assyria from 668 to 627 BCE, had the following curse written on many or all of the tablets collected at the library at Ninevah [...]:
I have transcribed upon tablets the noble products of the work of the scribe which none of the kings who have gone before me had learned, together with the wisdom of Nabu insofar as it existeth [in writing]. I have arranged them in classes, I have revised them and I have placed them in my palace, that I, even I, the ruler who knoweth the light of Ashur, the king of the gods, may read them. Whosoever shall carry off this tablet, or shall inscribe his name on it, side by side with mine own, may Ashur and Belit overthrow him in wrath and anger, and may they destroy his name and posterity in the land.
Back in the day, in the early 90s, I bought some software whose manual was printed on red paper, which was meant to discourage photocopying on black and white photocopiers. If you photocopied it, it would come out as a big mess because there wasn't much contrast between the print and the background.
Presumably this technique was used elsewhere too.
There's another version of DRM that you find in paper reference sources. (This where the data itself is not subject to protection.) Deliberately introduced harmless errors.
This does nothing to stop people from copying the material for personal use but it does keep them from copying it for commercial use.
"You say you didn't simply copy my street map. Please explain to the court how Silly Alley ended up in your map? As shown by this photograph there is no alley at that location."
Or encyclopedia articles that discuss a person or place that doesn't exist, same concept.
I'd say that the closest thing to copy protection that's ever included in physical books would be a Trap street.
These are fake elements included by cartographers in maps to catch people illegally stealing their work. Basically, you include a street, town, river, island etc that does not actually exist in a map that you have created, and if you later catch another publisher selling a map that includes that item, you sue them for illegally profiting from your work.
There are a number of interesting examples in this article: The Fake Places That Only Exist to Catch Copycat Mapmakers.
I have heard of movie scripts being printed on red or dark brown paper to prevent photocopying on set. Photocopiers at the time only supported full black or white, so if a colour was closer to black, it would become black. Red being closer to black than white resulted in pages being full black. But is different enough to be readable by humans. Not technically a book, but similar.
Of course that wouldn't work now days with modern cameras/scanners.
But modern cameras often see into the infrared spectrum so a layer of something the had a high reactivity for the infrared spectrum might. Cinemas have used bright infrared lights to prevent people recording the movie with a camera. Night vision on cameras is normally implemented with infrared lights. But not all cameras do this. Its probably also not to hard to filter out the infrared light.
It's worth considering that DRM on digital ebooks can still be bypassed using the exact same techniques as copying a physical book. You can scan/photograph the pages on an ebook reader or tablet for example. Of course it's generally quicker to crack the DRM and not have to bother.
In this way DRM just makes it more effort to copy than it's worth for many people. Although some people will bypass it, and you only need one since if they do it then, they can share it with everyone via the internet.
The other thing it does is reduce the quality of the copy. A scanned book might have OCR errors. It might not get formatted the same way. It might be missing metadata like the table of contents. Page numbers might end up in the text. A cam of a movie is bad quality, washed out, bad audio, framerate might be out of sync causing blurred frames. Capping a raw decoded video/audio feed will loose quality when it gets reencoded due to it going through lossy compression twice (that won't apply to lossless codecs, but that's generally only audio).
Many video game manuals have forms of copy protection in.
Some games require you to use a lookup sheet from the manual or similar. The lookup sheet is sometimes a decoder wheel (such as in Monkey Island) or printed in a way that is hard to copy (such as in Sim City)
These aren't really copy protection of manuals, however some games spread the copyright across the entire manual. The reason for copying a manual was primarily to bypass the copy protection of the game. Therefore as a sort of copy protection, they would often force you to have to copy the entire manual, which in the 90s was not a simple task as photocopy devices were not as ubiquitous as they are today. Examples are games like Lion King which required you to enter a word from a specific paragraph on a specific page in the manual, and Eye of the Beholder which had a symbol on every page and required you to enter a word from a page matching the symbol.
Both were deterrents to copying the manual by requiring you to copy the entire thing, which was a significant number of pages.
Copy protection through sheer volume is an interesting one, I think! Plus, it stops you from manually writing down the key parts, such as with a single large card, 20 minutes of laborious copying and you could get a key portion of it down.
Interestingly, the Lion King's copy protection could just be brute forced, if you found one word you could repeatedly launch the game and type in that word until eventually it was correct as it loaded quickly and prompted you immediately upon launch. The dictionary it had of page/line/word combinations was very small. Eye of the Beholder had symbols and a much bigger dictionary so it was harder to copy them manually. In addition, it prompted you after you completed the first level, so you couldn't restart the game quickly if you failed to try again.