These are related changes, since the previous approach of using
`PDF.stream.name` to distinguish between on-path and filelike-object PDFs
doesn't work for zipped files (and possibly other filelike-objects that
have a `name` property).
Note: this removes the fragile-looking type manipulation in intersects_bbox as
making it work with iterables is very ugly and I am not convinced of its
usefulness
This commit adds convenience methods to repair PDFs on the fly and/or to
write repaired PDFs to disk.
Currently, this does so via Ghostscript using the method we've asked
many users to try by following the instructions at
https://superuser.com/questions/278562/how-can-i-fix-repair-a-corrupted-pdf-file
Now, hopefully, this saves folks a few steps.
This commit swaps out Wand (and its non-Python dependencies ImageMagick
and Ghostscript) for pypdfium2 for PageImage rendering. This has some
advantages:
- Less finicky: Wand often caused users problems, due to "MagickWand
shared library not found" and "PolicyError: not authorized `PDF'"
issues. By contrast, pypdfium2 seems (at least at first) to more
self-contained and not require any system-tweaking.
- Faster: pypdfium2 appears to render images more quickly than Wand (see
@cmdlineuser's tests in #899)
- More flexible: pypdfium2 appears to generate images with greater color
depth; by default, pdfplumber quantizes those images so that they
save/display compactly (in fact, with smaller file sizes than the
previous code), this commit also adds parameters to retain all/more of
the original, more detailed colors.
Thanks to @cmdlineuser in #899 for the suggestion.
Inspired by https://github.com/jsvine/pdfplumber/issues/828
The PDF reference allows for "colors" to be defined as a series of
numbers and/or (much less commonly) patterns.
(See p. 288 and section 4.6 here:
https://ghostscript.com/~robin/pdf_reference17.pdf)
This commit separates out the pattern component of colors into their own
attributes, `stroking_pattern` and `non_stroking_pattern` so that they
don't muddle the interpretation of standard colors' tuple-of-numbers
representation.
This commit also adds code that attempts to fetch the `ncs`/`scs` color
space of each object. Due to current limitations of pdfminer.six,
however, the only such color space immediately available is the `ncs`
(non-stroking color space) property of char objects.
This commit normalizes the type representation of `stroking_color` and
`non_stroking_color` values. Thanks to @dhdaines for pointing out this
inconsistency.
Previously, `pdfplumber` passed along `pdfminer.six`'s colors without
normalization. Due to quirks in `pdfminer.six`'s color handling, this
meant that those values could be floats, ints, lists, or tuples. This
commit normalizes all color values (when non-None) into n-tuples, where
(val,) represents grayscale colors, (val, val, val) represents RBG, and
(val, val, val, val) represents CMYK colors.
This should solve the consistency issue, although might cause breaking
changes to code that filters for non-tuple values — e.g., `[c for c in
page.chars if c == [1, 0 0]]`. Although breaking changes are unpleasant,
I think the tradeoff for longer-term consistency is worth it.
When using relative=True for a re-crop, pdfplumber was passing the wrong
bounding box to the cropping function. This commit fixes that bug and
also refactors CroppedPage.__init__(...) for clarity and consistency's
sake.
Most of the groundwork was already there to add a PDF/Page.curve_edges
property. And, inspired, by
https://github.com/jsvine/pdfplumber/discussions/858 and related issues,
we now include 0/90/180/270-degree oriented curve segments into the
default table-detection strategy. As before, you can still switch to the
"lines_strict" to use only lines defined as such (rather than also using
rect and curve edges).
Inspired by https://github.com/jsvine/pdfplumber/discussions/852,
it turns out that .search(...) gets us most of the way, already.
Added a few params (`main_group`, `return_groups`, `return_chars`) to
.search(...) to enable this, which also make that method more generally
flexible.
Calculating the bounding boxes of the words is, upon reflection and
testing, not necessary. Instead, all we need is the latest character in
the current word.
With regular expressions, patterns made up entirely of optional groups
(e.g., r"(dfsdgfwerw)?") can return match objects that are, effectively,
empty. These were being treated as real search results, and throwing
errors in the process. Now they're being treated as non-results.
Separately but relatedly, whitespace-only searches were throwing errors.
This was due to (a) how PDFs generally represent whitespace (implicitly,
rather than explicit space characters), and (b) how pdfplumber
internally represents those spaces while performing layout analysis.
This caused search results to have no explicit bounding box, throwing
errors. Now, similarly to handling empty search results, we handle
all-whitespace search results by considering them to be non-results.
Any `text_`-prefixed table extraction setting now automatically gets
passed to `.extract_text(...)` via `table.extract(...)`.
This introduces one minor but breaking change, which is that
`keep_blank_chars` (previously, a valid table extraction setting) now
needs to be passed as `text_keep_blank_chars`.
... preferring to composite with a white background instead of removing
the alpha channel. This seems to more reliably produce high-quality
conversions (easier to read, fewer conversion artifacts) than either the
prior or other previous approaches.
Jeremy Singer-Vine