Book scanning or book digitization (also: magazine scanning or magazine digitization) is the process of converting physical books and magazines into digital media such as images, electronic text, or electronic books (e-books) by using an image scanner.[1] Large scale book scanning projects have made many books available online.[2]
Digital books can be easily distributed, reproduced, and read on-screen. Common file formats are DjVu, Portable Document Format (PDF), and Tag Image File Format (TIFF). To convert the raw images optical character recognition (OCR) is used to turn book pages into a digital text format like ASCII or other similar format, which reduces the file size and allows the text to be reformatted, searched, or processed by other applications.
Image scanners may be manual or automated. In an ordinary commercial image scanner, the book is placed on a flat glass plate (or platen), and a light and optical array moves across the book underneath the glass. In manual book scanners, the glass plate extends to the edge of the scanner, making it easier to line up the book's spine.
A problem with scanning bound books is that when a book that is not very thin is laid flat, the part of the page close to the spine (the gutter) is significantly curved, distorting the text in that part of the scan. One solution is to separate the book into separate pages by cutting or unbinding. A non-destructive method is to hold the book in a V-shaped holder and photograph it, rather than lay it flat and scan it. The curvature in the gutter is much less pronounced this way.[3] Pages may be turned by hand or by automated paper transport devices. Transparent plastic or glass sheets are usually pressed against the page to flatten it.
After scanning, software adjusts the document images by lining it up, cropping it, picture-editing it, and converting it to text and final e-book form. Human proofreaders usually check the output for errors.
Scanning at is adequate for conversion to digital text output, but for archival reproduction of rare, elaborate or illustrated books, much higher resolution is used. High-end scanners capable of thousands of pages per hour can cost thousands of dollars, but do-it-yourself (DIY), manual book scanners capable of 1200 pages per hour have been built for US$300.[4]
Commercial book scanners are not like normal scanners; these book scanners are usually a high quality digital camera with light sources on either side of the camera mounted on some sort of frame to provide easy access for a person or machine to flip through the pages of the book. Some models involve V-shaped book cradles, which provide support for book spines and also center book position automatically.
The advantage of this type of scanner is that it is very fast, compared to the productivity of overhead scanners.
Projects like Project Gutenberg (est. 1971),[5] Million Book Project (est. circa 2001), Google Books (est. 2004), and the Open Content Alliance (est. 2005) scan books on a large scale.[6] [7]
One of the main challenges to this is the sheer volume of books that must be scanned. In 2010 the total number of works appearing as books in human history was estimated to be around 130 million.[8] All of these must be scanned and then made searchable online for the public to use as a universal library. Currently, there are three main ways that large organizations are relying on: outsourcing, scanning in-house using commercial book scanners, and scanning in-house using robotic scanning solutions.
As for outsourcing, books are often shipped to be scanned by low-cost sources to India or China. Alternatively, due to convenience, safety and technology improvement, many organizations choose to scan in-house by using either overhead scanners which are time-consuming, or digital camera-based scanning machines which are substantially faster and is a method employed by Internet Archive as well as Google. Traditional methods have included cutting off the book's spine and scanning the pages in a scanner with automatic page-feeding capability, with subsequent rebinding of the loose pages.
Once the page is scanned, the data is either entered manually or via OCR, another major cost of the book scanning projects.
Due to copyright issues, most scanned books are those that are out of copyright; however, Google Books is known to scan books still protected under copyright unless the publisher specifically prohibits this.[9] [10]
There are many collaborative digitization projects throughout the United States. Two of the earliest projects were the Collaborative Digitization Project in Colorado and NC ECHO – North Carolina Exploring Cultural Heritage Online,[11] based at the State Library of North Carolina.
These projects establish and publish best practices for digitization and work with regional partners to digitize cultural heritage materials. Additional criteria for best practices have more recently been established in the UK, Australia and the European Union.[12] Wisconsin Heritage Online[13] is a collaborative digitization project modeled after the Colorado Collaborative Digitization Project. Wisconsin uses a wiki[14] to build and distribute collaborative documentation. Georgia's collaborative digitization program, the Digital Library of Georgia,[15] presents a seamless virtual library on the state's history and life, including more than a hundred digital collections from 60 institutions and 100 agencies of government. The Digital Library of Georgia is a GALILEO[16] initiative based at the University of Georgia Libraries.
In the twentieth century, the Hill Museum and Manuscript Library photographed books in Ethiopia that were subsequently destroyed amidst political violence in 1975. The library has since worked to photograph manuscripts in Middle Eastern countries.[17]
In South Asia, the Nanakshahi trust is digitizing manuscripts of Gurmukhī script.
In Australia, there have been many collaborative projects between the National Library of Australia and universities to improve the repository infrastructure that digitized information would be stored in.[18] Some of these projects include, the ARROW (Australian Research Repositories Online to the World) project and the APSR (Australian Partnership for Sustainable Repository) project.
For book scanning on a low budget, the least expensive way to scan a book or magazine is to cut off the binding. This converts the book or magazine into a sheaf of separate sheets which can be loaded into a standard automatic document feeder (ADF) and scanned using inexpensive and common scanning technology. The method is not suitable for rare or valuable books. There are two technical difficulties with this process, first with the cutting and second with the scanning.
More precise and less destructive than cutting pages is to unbind by hand using suitable tools. This technique has been successfully employed for tens of thousands of pages of archival original paper scanned for the Riazanov Library digital archive project from newspapers and magazines and pamphlets, varying from 50 to 100 years old and more, and often composed of fragile, brittle paper. Although the monetary value for some collectors (and for most sellers of this sort of material) is destroyed by unbinding, it in many cases actually greatly assists preservation of the pages, making them more accessible to researchers and less likely to be damaged when subsequently examined. A disadvantage is that unbound stacks of pages are "fluffed up", and therefore more exposed to oxygen in the air, which may in some cases speed deterioration. This can be addressed by putting weights on the pages after they are unbound, and storage in appropriate containers.
Hand unbinding will preserve text that runs into the gutters of bindings, and most critically allows more easy and complete high quality scans to be made of two-page-wide material, such as center cartoons, graphic art, and photos in magazines. The digital archive of The Liberator 1918-1924 on Marxists Internet Archive demonstrates the quality of two-page-wide graphic art scans made possible by careful hand unbinding, then scanning.
Unbinding techniques vary with the binding technology, from simply removing a few staples, to unbending and removing nails, to meticulously grinding down layers of glue on the spine of a book to precisely the right point, followed by laborious removal of the string used to hold the book together.
With some newspapers (such as Labor Action 1950-1952) there are columns on the center of facing pages that run across the pages. Chopping off part of the spine of a bound volume of such papers will lose part of this text. Even the Greenwood Reprint of this publication failed to preserve the text content of those center columns, cutting off significant amounts of text there. Only when bound volumes of the original newspaper were meticulously unbound, and the opened pairs of center pages were scanned as a single page on a flat bed scanner was the center column content made digitally available. Alternatively, one can present the two facing center pages as three scans: one of each individual page, and one of a page sized area situated over the center of the two pages.
One way of cutting a stack of 500 to 1,000 pages in one pass is to use a guillotine paper cutter, a large steel table with a paper vise that screws down onto the stack and firmly secures it before cutting. A large sharpened steel blade which moves straight down cuts the entire length of each sheet in one operation. A lever on the blade permits several hundred pounds of force to be applied to the blade for a quick one-pass cut.
A clean cut through a thick stack of paper cannot be made with a traditional inexpensive sickle-shaped hinged paper cutter. These cutters are only intended for a few sheets, with up to ten sheets being the practical cutting limit. A large stack of paper applies torsional forces on the hinge, pulling the blade away from the cutting edge on the table. The cut becomes more inaccurate as the cut moves away from the hinge, and the force required to hold the blade against the cutting edge increases as the cut moves away from the hinge.
The guillotine cutting process dulls the blade over time, requiring that it be resharpened. Coated paper such as slick magazine paper dulls the blade more quickly than plain book paper, due to the kaolinite clay coating. Additionally, removing the binding of an entire hardcover book causes excessive wear due to cutting through the cover's stiff backing material. Instead the outer cover can be removed and only interior pages need be cut.
An alternate method of unbinding books is to use a table saw. While this method is potentially dangerous and does not leave as smooth an edge as the guillotine paper cutter method, it is more readily available to the average person. The ideal method is to clamp the book between two thick boards using heavy machine screws to provide the clamping force. The entire wood and book package is fed through the table saw using the rip fence as a guide. A sharp fine carbide tooth blade is ideal for generating an acceptable cut. The quality of the cut depends on the blade, feed rate, type of paper, paper coating, and binding material.
Once the paper is liberated from the spine, it can be scanned one sheet at a time using a flatbed scanner or automatic document feeder (ADF).
Pages with a decorative riffled edging or curving in an arc due to a non-flat binding can be difficult to scan using an ADF, as they are designed to scan pages of uniform shape and size, and variably sized or shaped pages can lead to improper scanning. The riffled edges or curved edge can be guillotined off to render the outer edges flat and smooth before the binding is cut.
The coated paper of magazines and bound textbooks can make them difficult for the rollers in an ADF to pick up and guide along the paper path. An ADF which uses a series of rollers and channels to flip sheets over may jam or misfeed when fed coated paper. Generally there are fewer problems by using as straight of a paper path as is possible, with few bends and curves. The clay can also rub off the paper over time and coat sticky pickup rollers, causing them to loosely grip the paper. The ADF rollers may need periodic cleaning to prevent this slipping.
Magazines can pose a bulk-scanning challenge due to small nonuniform sheets of paper in the stack, such as magazine subscription cards and fold out pages. These need to be removed before the bulk scan begins, and are either scanned separately if they include worthwhile content, or are simply left out of the scan process.
Software driven machines and robots have been developed to scan books without the need of unbinding them in order to preserve both the contents of the document and create a digital image archive of its current state. This recent trend has been due in part to ever improving imaging technologies that allow a high quality digital archive image to be captured with little or no damage to a rare or fragile book in a reasonably short period of time.
The first fully automated book scanner was the DL (Digitizing Line) scanner, manufactured by 4DigitalBooks in Switzerland. The first known installation was at Stanford University in 2001.[19] [20] The scanner received a Dow Jones Runner-Up award under Business Applications Category in 2001.[21]
In 2007 the company TREVENTUS presented an automated book scanner with a book opening angle for scanning of 60°. Which is an improvement in the area of conservation of the books during scanning. The company was awarded with the European Union "ICT Grand Prize 2007",[22] [23] for its development of the ScanRobot. This technology was also used in a mass digitization project from the Bavarian State Library[24] where 8,900 books from the 16th century were digitized in 18 months using three of these v-shape scanners.
Indus International, Inc, based in West Salem, Wisconsin, produces scanners which were bought by some US entities for services such as interlibrary loan.[25]
Most high-end commercial robotic scanners use air and suction technology, while some use newer approaches such as bionic fingers for turning pages. Some scanners take advantage of ultrasonic or photoelectric sensors to detect dual pages and prevent skipping of pages. With reports of machines being able to scan up to 2,900 pages per hour,[26] robotic book scanners are specifically designed for large-scale digitization projects.
Google's patent 7508978 shows an infrared camera technology which allows detection and automatic adjustment of the three-dimensional shape of the page.[27] [28] Researchers from the University of Tokyo have an experimental non-destructive book scanner[29] that includes a 3D surface scanner to allow images of a curved page to be straightened in software. Thus the book or magazine can be scanned as quickly as the operator can flip through the pages, about 200 pages per minute.
There are techniques to minimise and to correct for distortion in the page gutter.[30]