Add edge_min_length_prefilter table setting #1274

... for initial edge filtering. Lowering this setting enables capturing
small edge segments (e.g., dashed lines) that would be filtered out with
the default minimum length of 1. Raising this setting would be less
common but plausible.

Thanks to @bronislav for suggesting.
This commit is contained in:
Jeremy Singer-Vine
2025-07-20 08:08:14 -04:00
parent 503dfb6cd7
commit 42a004f6b2
3 changed files with 27 additions and 4 deletions
+5
View File
@@ -2,6 +2,11 @@
All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](http://keepachangelog.com/).
## [Unreleased]
### Added
- Add `edge_min_length_prefilter` table setting for initial edge filtering. Lowering this setting enables capturing small edge segments (e.g., dashed lines) that would be filtered out with the default minimum length of 1. Raising this setting would be less common but plausible. (h/t @bronislav). ([#1274](https://github.com/jsvine/pdfplumber/issues/1274)).
## [0.11.7] - 2025-06-12
### Added
+2
View File
@@ -402,6 +402,7 @@ By default, `extract_tables` uses the page's vertical and horizontal lines (or r
"join_x_tolerance": 3,
"join_y_tolerance": 3,
"edge_min_length": 3,
"edge_min_length_prefilter": 1,
"min_words_vertical": 3,
"min_words_horizontal": 1,
"intersection_tolerance": 3,
@@ -423,6 +424,7 @@ By default, `extract_tables` uses the page's vertical and horizontal lines (or r
|`"snap_tolerance"`, `"snap_x_tolerance"`, `"snap_y_tolerance"`| Parallel lines within `snap_tolerance` points will be "snapped" to the same horizontal or vertical position.|
|`"join_tolerance"`, `"join_x_tolerance"`, `"join_y_tolerance"`| Line segments on the same infinite line, and whose ends are within `join_tolerance` of one another, will be "joined" into a single line segment.|
|`"edge_min_length"`| Edges shorter than `edge_min_length` will be discarded before attempting to reconstruct the table.|
|`"edge_min_length_prefilter"`| Edges shorter than `edge_min_length_prefilter` will be discarded during initial edge filtering from the page. Lowering this value (e.g., to `0.5`) can help capture small dashed lines that might otherwise be filtered out.|
|`"min_words_vertical"`| When using `"vertical_strategy": "text"`, at least `min_words_vertical` words must share the same alignment.|
|`"min_words_horizontal"`| When using `"horizontal_strategy": "text"`, at least `min_words_horizontal` words must share the same alignment.|
|`"intersection_tolerance"`, `"intersection_x_tolerance"`, `"intersection_y_tolerance"`| When combining edges into cells, orthogonal edges must be within `intersection_tolerance` points to be considered intersecting.|
+20 -4
View File
@@ -466,6 +466,7 @@ NON_NEGATIVE_SETTINGS = [
"join_x_tolerance",
"join_y_tolerance",
"edge_min_length",
"edge_min_length_prefilter",
"min_words_vertical",
"min_words_horizontal",
"intersection_tolerance",
@@ -494,6 +495,7 @@ class TableSettings:
join_x_tolerance: T_num = UNSET
join_y_tolerance: T_num = UNSET
edge_min_length: T_num = 3
edge_min_length_prefilter: T_num = 1
min_words_vertical: int = DEFAULT_MIN_WORDS_VERTICAL
min_words_horizontal: int = DEFAULT_MIN_WORDS_HORIZONTAL
intersection_tolerance: T_num = 3
@@ -637,9 +639,16 @@ class TableFinder(object):
)
if v_strat == "lines":
v_base = utils.filter_edges(self.page.edges, "v")
v_base = utils.filter_edges(
self.page.edges, "v", min_length=settings.edge_min_length_prefilter
)
elif v_strat == "lines_strict":
v_base = utils.filter_edges(self.page.edges, "v", edge_type="line")
v_base = utils.filter_edges(
self.page.edges,
"v",
edge_type="line",
min_length=settings.edge_min_length_prefilter,
)
elif v_strat == "text":
v_base = words_to_edges_v(words, word_threshold=settings.min_words_vertical)
elif v_strat == "explicit":
@@ -666,9 +675,16 @@ class TableFinder(object):
)
if h_strat == "lines":
h_base = utils.filter_edges(self.page.edges, "h")
h_base = utils.filter_edges(
self.page.edges, "h", min_length=settings.edge_min_length_prefilter
)
elif h_strat == "lines_strict":
h_base = utils.filter_edges(self.page.edges, "h", edge_type="line")
h_base = utils.filter_edges(
self.page.edges,
"h",
edge_type="line",
min_length=settings.edge_min_length_prefilter,
)
elif h_strat == "text":
h_base = words_to_edges_h(
words, word_threshold=settings.min_words_horizontal