This commit reinstates access to higher-level layout objects (such as
`textboxhorizontal`) when `laparams` is passed to
`pdfplumber.open(...)`. Had been removed in `0.5.24` via 1f87898.
Also adds a test for this behavior.
h/t @xv44586 for the initial inspiration 👍
These new methods return a version of the chars/page with duplicate
chars — those sharing the same text, fontname, size, and positioning
(within `tolerance` x/y) as other characters — removed.
"direction" == 1 when upright words go LTR and non-upright words go TTB,
and -1 when the opposite is true. This attribute should help in reusing
the results of `.extract_words` in other functions, so that the user
knows how the extracted word corresponds to the word's layout on the
page.
Taking @samkit-jain's smart question about sort-order into account, this
commit reworks `.extract_words`, making the sorting more explicit, and
adding an option to *not* sort, via `use_text_flow=True`, which follows
the original PDF's text flow, rather than presorting all characters top-down,
left-right.
To do so required rethinking the word-delineation logic,
which in turn motivated a shift toward a more object-oriented approach
(especially to keep track of settings without passing them from function
to function). Now `.extract_text`'s logic is encapsulated in a new class,
`TextExtractor`.
This commit refactors and hopefully makes clearer the logic in
utils.extract_words. It also adds a new parameter, `extra_attrs`, which
allows the user to pass a list of attributes on which to group all
characters.
For instance, passing `extra_attrs=["fontname", "size"]` will not allow
characters with different font names or sizes to become part of the same
word. As a benefit, those resulting word dicts will contain `"fontname"`
and `"size"` attributes — providing a long-requested feature (cf. issue
The same goals can be acheived by just testing the first page, instead
of the entire PDF, trimming approximately 20% off the total test suite's
execution time, although YMMV.
The test, responding to PR #77, was particularly slow and the
accompanying PDF somewhat large. Now that the fixes to the issues
identified in the PR are fairly tightly integrated into pdfplumber, we
can remove the tests with reasonable confidence, speeding up development
time.
Moves most of the logic previously in cli.py to convert.py, for usage by
other submodules. Adds Container.to_json and Container.to_csv. Makes
adjustments/fixes to other parts of the library, based on edge-cases
encountered (such as infinite recursion in anntations).
- Fixes `.extract_words`, which had been returning incorrect results when `horizontal_ltr = False`
- Fixes `.resize_object`, which had been failing in various permutations
- Brings utils.py test coverage to 100%
Previously, classes were being defined dynamically, in a confusing
manner. This new logic should be easier to understand.
This commit also adds a few assertions that bring coverage for page.py
to 100%.
.annos was non-functional, based on a misunderstanding of how
Annotation objects were represented in the PDF object. Also shifts
language from "annos" to "annots" to mirror pdfminer's nomenclature.
See https://github.com/jsvine/pdfplumber/issues/203
Root cause of decimalization bug flagged in that issue was
pdfminer.six's .resolve_all method not recusively resolving items within
tuples. Now we define our own method, with some slight optimization
tweaks.
Jeremy Singer-Vine