Thanks to PR from @jeremybmerrill, who writes:
> This form-parsing example handles form fields recursively contained within other form fields, removes the incorrect-assumption that field-names are unique and includes the alternate field name in output (which is often a very useful guide to what's in a field).
> the prior form-parsing example used the field-name ("T" entry) as the key in the form_data dict, implicitly assuming that the field name is globally unique within a document. That's not a correct assumption; nested field names are often simply a numeric index like 1 or 0. The prior example also entirely ignored the TU entry alternate field name.
These are related changes, since the previous approach of using
`PDF.stream.name` to distinguish between on-path and filelike-object PDFs
doesn't work for zipped files (and possibly other filelike-objects that
have a `name` property).
Note: this removes the fragile-looking type manipulation in intersects_bbox as
making it work with iterables is very ugly and I am not convinced of its
usefulness
This commit adds convenience methods to repair PDFs on the fly and/or to
write repaired PDFs to disk.
Currently, this does so via Ghostscript using the method we've asked
many users to try by following the instructions at
https://superuser.com/questions/278562/how-can-i-fix-repair-a-corrupted-pdf-file
Now, hopefully, this saves folks a few steps.
This commit swaps out Wand (and its non-Python dependencies ImageMagick
and Ghostscript) for pypdfium2 for PageImage rendering. This has some
advantages:
- Less finicky: Wand often caused users problems, due to "MagickWand
shared library not found" and "PolicyError: not authorized `PDF'"
issues. By contrast, pypdfium2 seems (at least at first) to more
self-contained and not require any system-tweaking.
- Faster: pypdfium2 appears to render images more quickly than Wand (see
@cmdlineuser's tests in #899)
- More flexible: pypdfium2 appears to generate images with greater color
depth; by default, pdfplumber quantizes those images so that they
save/display compactly (in fact, with smaller file sizes than the
previous code), this commit also adds parameters to retain all/more of
the original, more detailed colors.
Thanks to @cmdlineuser in #899 for the suggestion.
Inspired by https://github.com/jsvine/pdfplumber/issues/828
The PDF reference allows for "colors" to be defined as a series of
numbers and/or (much less commonly) patterns.
(See p. 288 and section 4.6 here:
https://ghostscript.com/~robin/pdf_reference17.pdf)
This commit separates out the pattern component of colors into their own
attributes, `stroking_pattern` and `non_stroking_pattern` so that they
don't muddle the interpretation of standard colors' tuple-of-numbers
representation.
This commit also adds code that attempts to fetch the `ncs`/`scs` color
space of each object. Due to current limitations of pdfminer.six,
however, the only such color space immediately available is the `ncs`
(non-stroking color space) property of char objects.
Previously, `pdfplumber.Page` had these table-getting methods:
- `.find_tables(...)`
- `.extract_tables(...)`
- `.extract_table(...)`
For consistency/completeness's sake, this commit adds:
- `.find_table(...)`
... which, analogous to `.extract_table(...)`, returns the largest table
on the page.
Indeed, `.extract_table(...)` now uses `.find_table(...)` beneath the
hood.
Thanks to @pdille for the suggestion, here:
https://github.com/jsvine/pdfplumber/discussions/864#discussioncomment-5668209
This commit normalizes the type representation of `stroking_color` and
`non_stroking_color` values. Thanks to @dhdaines for pointing out this
inconsistency.
Previously, `pdfplumber` passed along `pdfminer.six`'s colors without
normalization. Due to quirks in `pdfminer.six`'s color handling, this
meant that those values could be floats, ints, lists, or tuples. This
commit normalizes all color values (when non-None) into n-tuples, where
(val,) represents grayscale colors, (val, val, val) represents RBG, and
(val, val, val, val) represents CMYK colors.
This should solve the consistency issue, although might cause breaking
changes to code that filters for non-tuple values — e.g., `[c for c in
page.chars if c == [1, 0 0]]`. Although breaking changes are unpleasant,
I think the tradeoff for longer-term consistency is worth it.
When using relative=True for a re-crop, pdfplumber was passing the wrong
bounding box to the cropping function. This commit fixes that bug and
also refactors CroppedPage.__init__(...) for clarity and consistency's
sake.
Main edge-case was with `use_text_flow` on text-lines that then
backtracked. But this rewrite also aims to make the logic more explicit
and easier to follow.
As noted in #912, `use_text_flow` was not being handled consistently, as
characters and words were being re-sorted without checking first if this
parameter was set to `True`.
Jeremy Singer-Vine