Anybody working with scanned documents faces this issue: Scanned PDFs are often low contrast, contain scanning artifacts such as book comb bindings, and are skewed and rotated somehow. This not only makes them difficult to read on e-reader and tablet devices but also reduces the size of usable content on printed A4 pages.

Common problems with scanned documents

I have been printing many classical music pieces from IMSLP.com which offers a collection of sheet music in the public domain. This included graded sight-reading course books with hundreds of pages of PDF scans. These contain the following issues:

Advertisement Begins

Advertisement End

As you can see, these problem areas not only waste printer toner, they also reduce the quality of printed material and create eye strain and readability issues. For some time now I wanted to write a script to correct these problems as it is fairly straight forward using image manipulation tools like ImageMagick and Bash tools like parallel to take care of efficiently executing the same operation on hundreds of image files. Each problem area above can be solved with the corresponding step listed here:

1. Crop unwanted scanning artifacts such as book comb bindings by defining a crop margin using a GUI
2. Crop white space borders to increase the size of usable content on tablets/e-readers
3. Deskew crooked pages (straightens up the page and reduces unnecessary margins)
4. Increase contrast by applying a black threshold to remove grey background
5. Convert image to true black and white to save on printer ink/toner and increase readability)
6. Sharpen text

Depending on the quality of original PDF, this allowed me to increase the size of music staves by 10.7% on the final printed page, while also increasing readability due to maximum contrast and saving printer toner by removing unwanted/distracting scanning artifacts.

YouTube Video

The example PDF file in the video is of terrible quality to begin with… and I wouldn’t normally bother with such a file (there are typically many alternative versions for the same piece on IMSLP.com. It’s just a matter of finding a better quality PDF scan and then applying the series of commands to get a final output that is high contrast, cropped, straightened and sharpened.

Main Benefits of this solution

The main differentiator of using this solution:

• Scalability to hundreds of pages – this process scales to hundreds of pages well
• Reduce waiting/processing time by using available CPU and memory resources optimally (multi threading)
• Completely local tool – no upload to online servers required
• Supports different page layouts (e.g. facing pages as in booklet, or single plage layouts and any custom arrangement as well, see GUI options in next section)
• Higher cropping accuracy because of the ability to manually tweak individual pages and apply settings in bulk.
• Free of charge

The Python Tknter Graphical User Interface

Now stating the obvious: I really did not spend much time on this UI…. as soon as I got it working for the intended use case I stopped enhancing it. If there is enough interest I might clean it up. For now it offers basic but powerful functionality for manipulating PDF scans for any computer running Python.

• Apply crop borders using arrow keys for top, bottom, left and right margins
• Use SHIFT + arrow keys to reduce margins
• Save time by applying the same crop margins to many files in bulk
  • All pages, Even pages, Odd pages, Remaining pages in range, uncropped pages
• Save crop settings to CSV file

Now, it might surprise you to hear me (an automation addict) say this, however, for me it is worth spending 15 minutes manually tweaking a file if that means the final printed booklet is easier to read, cheaper to print and good-looking in general. I am planning to spend a lot of time learning, playing and enjoying these music pieces. It really is no comparison to spending that little bit of time beforehand cleaning the documents.

Using Bash Unix tools for this purpose

Part of my interest in this exercise is to play with the parallel command and learn about ImageMagick’s image manipulation options. Here is a table of the commands used and their purpose.

Kitchen Multi-Timer Pro

Now you’re cooking

Multi Timer Pro is your ultimate meal prep companion, keeping track of multiple cooking times and making adjustments on the fly. Give it a try today and become a better home cook!

Split.sh File

I included the split.sh file below. It splits a PDF into smaller PDFs based on the batch size (5 pages max per file). It gives each file a helpful name showing the included page range (e.g. pages 1–5, 5-10, 11-15 etc)

BATCH_SIZE=5
TOTAL=$((`pdfinfo input.pdf | grep -- ^Pages | tr -dc '[0-9]'`))

for i in `seq 1 $BATCH_SIZE $TOTAL`
do 
j=$((i+$BATCH_SIZE-1))

gs -sDEVICE=pdfwrite -dNOPAUSE -dBATCH -dFirstPage=$i -dLastPage=$j -sOutputFile=pdfsplit/output_$i-$j.pdf input.pdf
done

How to clean up scanned PDFs using my tool

ImageMagick Policies

I did have to tweak ImageMagick settings to increase memory and CPU usage and enable PDF editing. Simply uncomment or edit the following rows in /etc/ImageMagick-6/policy.xml. Adjust based on your system’s resources.

The commands run multiple instances of ImageMagick concurrently in order to speed up the processing.

<policy domain="resource" name="memory" value="2GiB"/>
<policy domain="resource" name="map" value="2GiB"/>
<policy domain="resource" name="disk" value="10GiB"/>
<!-- comment out the line below to enable PDF editing -->
<!-- <policy domain="coder" rights="none" pattern="PDF" /> -->