Hi, @badGarnet , Can you review my PR please?

@badGarnet Please review my PR

Awesome work dude!

And I'm curious, is there any reason it's limited to H1-H4, rather than H1-H6?

Awesome work dude!

And I'm curious, is there any reason it's limited to H1-H4, rather than H1-H6?

The H1–H4 limit follows the issue title (#4204), which requested "H1, H2, H3, H4". The code can be extended to H6 if you want.

Oh okay, makes sense. I just named those specifically so that it was easier for people to search for.

I think supporting up to H6 will probably help cover as many use cases as possible.

Oh okay, makes sense. I just named those specifically so that it was easier for people to search for.

I think supporting up to H6 will probably help cover as many use cases as possible.

Ok, I will update code

hi, @codebymikey , I updated code for H1~H6, Please check. Thank you for your review

Hi, @codebymikey , Please comment if you have another feedback

Nope, all done. Probably just need to be rebased with upstream, and wait for a maintainer to review.

Thanks again for implementing!

Nope, all done. Probably just need to be rebased with upstream, and wait for a maintainer to review.

Thanks again for implementing!

Thank your for your review

Hi, @codebymikey , when can maintainer review this PR?

Not sure, as I'm not a maintainer.

But based off the current activity in the project, it probably shouldn't take more than a couple days to get some.

Not sure, as I'm not a maintainer.

But based off the current activity in the project, it probably shouldn't take more than a couple days to get some.

Thank you

Hi, @codebymikey , When can maintainer review my PR?

Hi, Anyone can review my PR?

Hi, @codebymikey . why can't this PR be merged. please help me to merge this.

I'm not a maintainer, so can't merge this for you.

I'm not sure why it's not getting any attention from the maintainers though. Might be worth nudging an active maintainer like @PastelStorm or @badGarnet for their feedback if you want it looked at quicker.

Also, the PR probably needs a rebase too.

Hi, @PastelStorm, @badGarnet , Could you merge this for me?

@Angel98518 @codebymikey apologies for not reviewing this PR in a timely manner. I will review it in a moment.

Findings (ordered by severity)

• High — Outline nesting is parsed with the wrong level for nested list structures

if isinstance(outline_item, list):
    for item in outline_item:
        _extract_outline_recursive(item, level)

In pypdf, nested outline hierarchies are commonly represented using nested lists. This recursion keeps the same level when descending into a nested list, so child headings can be flattened to the parent level. That directly causes wrong heading_level assignments.

• Medium — New tests are mostly vacuous / non-assertive, so regressions can slip through

# Create a minimal PDF for testing
# In a real scenario, this would be a PDF with an outline
outline = extract_pdf_outline(filename=str(tmp_path / "test.pdf"))
# Should return empty list if file doesn't exist or has no outline
assert isinstance(outline, list)

levels = [e.metadata.heading_level for e in result if e.metadata and e.metadata.heading_level is not None]
assert len(levels) >= 0  # May or may not assign levels depending on heuristics

if elements[0].metadata.heading_level is not None:
    assert 1 <= elements[0].metadata.heading_level <= 6

These pass even if the feature does nothing. There’s no assertion of expected behavior for real outlines, no negative-case precision checks, and no integration assertion in partition_pdf_or_image.

• Medium — Same heading-inference block is duplicated 3 times in partition_pdf_or_image

# Infer heading levels for PDF documents
if not is_image:
    try:
        # Prepare file for outline extraction
        file_for_outline = None
        if file is not None:
            file.seek(0)
            file_for_outline = file.read() if hasattr(file, 'read') else file
        elements = infer_heading_levels(
            elements,
            filename=filename,
            file=file_for_outline,
            use_outline=True,
            use_font_analysis=True,
        )
    except Exception as e:
        logger.debug(f"Failed to infer heading levels: {e}")

Very similar blocks are repeated in HI_RES/FAST/OCR_ONLY. This increases drift risk and makes future fixes inconsistent. A helper (e.g., _maybe_infer_heading_levels(...)) would avoid this.

• Low — Broad exception swallowing can hide real bugs and make diagnosis hard

except Exception as e:
    # If outline extraction fails, return empty list
    # This is not a critical error - we can still use font size analysis
    pass

try:
    outline_entries = extract_pdf_outline(filename=filename, file=file)
    if outline_entries:
        infer_heading_levels_from_outline(elements, outline_entries)
except Exception:
    # If outline extraction fails, continue with font analysis
    pass

Combined with caller-level catch-and-log in pdf.py, failures can become silent no-ops. At least debug-log the exception in pdf_hierarchy.py to preserve observability.

• Low — Dead/unused code and typing issues in new module

def analyze_font_sizes_from_pdfminer(
    elements: list[Element],
    layout_elements_map: Optional[dict[str, any]] = None,
    page_width: float = 612.0,
    page_height: float = 792.0,
) -> dict[str, float]:

word_count = len(text.split())
char_count = len(text)
is_mostly_uppercase = (

elements, page_width, page_height, and char_count are unused. Also any is used as a type (dict[str, any]) instead of Any, which is incorrect typing.

Thank you @PastelStorm

Thank you @PastelStorm

please address the review above and rebase the branch and I'll run the CI. Hope to merge it soon!

@PastelStorm Please re-run CI again

Re run CI

Re run CI

Please run linter and tests locally first before blindly pushing changes.

Re run CI

Hi, @PastelStorm , I have tested on my local.
please re-run

Hi, @PastelStorm , Please review again.

Re-run

Re-run

Hi, @PastelStorm ,Could you please re-run CI?

Code Review: feat/pdf-hierarchical-headings-4204

Summary

This PR adds a heading_level metadata field (H1-H6) to Title elements produced by PDF partitioning. It introduces a new module pdf_hierarchy.py with two inference strategies: PDF outline/bookmarks and a font-size/heuristic fallback. The feature is integrated into all three PDF strategies (hi_res, fast, ocr_only).

1. Dead Code: Entire PDFMiner font-info extraction pipeline is unused

The following functions are effectively dead code:

• extract_font_info_from_layout_element() (lines 99-152)
• analyze_font_sizes_from_pdfminer() (lines 155-174)

They form a pipeline for extracting font information from PDFMiner layout elements, but they're only invoked through infer_heading_levels_from_font_sizes, which receives layout_elements_map=None from every call site:

def infer_heading_levels(
    elements: list[Element],
    filename: Optional[str] = None,
    file: Optional[io.BytesIO | bytes] = None,
    use_outline: bool = True,
    use_font_analysis: bool = True,
) -> list[Element]:
    # ...
    if use_font_analysis:
        # ...
        if elements_without_level:
            infer_heading_levels_from_font_sizes(elements_without_level)
            # ^ no layout_elements_map is ever passed

This means the "font size analysis" strategy never has actual font sizes. It always falls through to the heuristic branch (word count + capitalization), making the function name misleading.

2. Fragile & Arbitrary Heuristic Fallback

When font data is unavailable (always, per issue #1), the fallback scores titles by word count and capitalization using hardcoded magic numbers:

                word_count = len(text.split())
                is_mostly_uppercase = text.isupper() or (
                    len(text) > 0
                    and text[0].isupper()
                    and sum(1 for c in text if c.isupper()) / max(len(text), 1) > 0.5
                )

                base_score = 20.0
                word_penalty = word_count * 0.5
                capitalization_bonus = 5.0 if is_mostly_uppercase else 0.0
                score = base_score - word_penalty + capitalization_bonus

Problems:

• Shorter titles rank higher, but "Chapter 1" (2 words) would outrank "Introduction to Machine Learning" (4 words) regardless of actual heading level.
• The is_mostly_uppercase check has a counter-intuitive threshold: any title starting with a capital letter where >50% of chars are uppercase gets the bonus, so "GPU" (a 3-letter acronym) would rank as H1.
• The magic numbers (20.0, 0.5, 5.0) have no justification.

3. Per-Page Independent Heading Assignment is Architecturally Wrong

    titles_by_page: Dict[int, List[Element]] = defaultdict(list)
    for element in title_elements:
        page_num = element.metadata.page_number or 1
        titles_by_page[page_num].append(element)

    for page_num, page_titles in titles_by_page.items():
        if len(page_titles) < 2:
            # Single title on page gets level 1
            for element in page_titles:
                if element.metadata.heading_level is None:
                    element.metadata.heading_level = 1
            continue

Heading levels are computed per-page in isolation. This means:

• A subsection title that happens to be the only title on a page gets H1.
• The same text appearing on two different pages can get different heading levels depending on what other titles share that page.
• Document-wide heading hierarchy is completely lost.

4. Comment/Code Mismatch in elements.py

The field declaration comment says H1-H4, but the code supports H1-H6:

    # -- heading level (1-4) for hierarchical document structure (H1, H2, H3, H4) --
    heading_level: Optional[int]

5. _maybe_infer_heading_levels Closure Has File Side Effects

    def _maybe_infer_heading_levels(
        elements: list[Element],
    ) -> list[Element]:
        """Infer heading levels for PDF documents when appropriate."""
        if is_image:
            return elements

        try:
            file_for_outline: Optional[bytes | IO[bytes]] = None
            if file is not None:
                if hasattr(file, "seek"):
                    file.seek(0)
                file_for_outline = file.read() if hasattr(file, "read") else file

            return infer_heading_levels(
                elements,
                filename=filename,
                file=file_for_outline,
                use_outline=True,
                use_font_analysis=True,
            )
        except Exception as e:
            logger.debug(f"Failed to infer heading levels: {e}")
            return elements

Issues:

• file.read() loads the entire PDF into memory a second time (the main partitioning already read it). For large PDFs this doubles peak memory.
• The PDF is then opened a third time inside extract_pdf_outline via PdfReader(io.BytesIO(file)). Three full PDF reads for one partition call.
• After file.read(), the file cursor is at EOF. If any code later tries to use file without seeking, it will silently read zero bytes. The calling code does seek before some paths but not consistently.

6. O(n * m) Fuzzy Matching with O(k^2) Inner Cost

    for element in elements:
        if isinstance(element, Title) and element.metadata:
            element_text = element.text.strip().lower()
            # ...
            if element_text in outline_map:
                # ...
            else:
                for outline_title, level in outline_map.items():
                    similarity = SequenceMatcher(None, element_text, outline_title).ratio()

For each Title element, it iterates all outline entries and calls SequenceMatcher.ratio(), which is O(k^2) in string length. For a 200-page document with ~100 titles and ~50 outline entries, this is 5,000 comparisons each with quadratic string cost. There's no early termination on a perfect match within the fuzzy loop either.

7. Non-Deterministic Set-to-List Conversion

                font_info["font_name"] = (
                    list(font_names)[0] if len(font_names) == 1 else list(font_names)
                )

font_names is a set. When there are multiple fonts, list(font_names) produces an arbitrary order. While this code path is currently dead (see issue #1), it would cause non-deterministic behavior if revived.

8. Duplicate hasattr Check

            for char in layout_element.chars:
                if hasattr(char, "fontname"):
                    font_names.add(char.fontname)
                if hasattr(char, "size"):
                    font_sizes.append(char.size)
                if hasattr(char, "fontname"):
                    font_name_lower = char.fontname.lower()

hasattr(char, "fontname") is checked on line 123 and again on line 127 within the same loop iteration.

9. Outline Key Collisions

        outline_map[title.lower()] = normalized_level

If two different outline entries normalize to the same lowercase string (e.g., "INTRODUCTION" and "Introduction"), the second overwrites the first. Only the last level wins.

10. Silent Exception Swallowing

Multiple places catch Exception broadly and swallow it:

    except Exception as e:
        # If outline extraction fails, return empty list but log for observability.
        logger.debug(f"Failed to extract PDF outline: {e}")

        except Exception as e:
            # If outline extraction fails, continue with font analysis but log for debugging.
            logger.debug(f"Failed during outline-based heading inference: {e}")

        except Exception as e:
            logger.debug(f"Failed to infer heading levels: {e}")
            return elements

Three layers of exception eating. If a real bug (e.g., TypeError, KeyError) occurs deep in the outline parsing, it's silently logged at debug level and the feature just produces no output with no user-visible indication of failure.

11. Outline Parsing: Fragile Even/Odd Alternation Assumption

                if isinstance(outline_item, list):
                    if level == -1:
                        # Top-level: alternate item (level 0) and its children list (level 1)
                        for i in range(len(outline_item)):
                            if i % 2 == 0:
                                _extract_outline_recursive(outline_item[i], 0)
                            else:
                                _extract_outline_recursive(outline_item[i], 1)
                    else:
                        for item in outline_item:
                            _extract_outline_recursive(item, level)

This assumes pypdf always structures the outline as [item, [children], item, [children], ...]. But pypdf can produce outlines where multiple items appear consecutively without child lists, or child lists can be at arbitrary positions. This will misassign levels for PDFs that don't follow this exact pattern.

12. Redundant Clamping

min(max(level, 1), 6) appears three times — on lines 197, 221, and 321 — even though by construction the values are already in range (e.g., line 197 already clamps, then line 221 clamps the already-clamped value again).

13. Test Issues (skipping test content per your request, but noting structural problems)

• test_fuzzy_matching_in_outline doesn't assert matching happened: It uses if elements[0].metadata.heading_level is not None: rather than asserting. The test passes silently if no match was found.
• test_heading_levels_are_in_range is a duplicate of test_infer_heading_levels_from_font_sizes — same setup, same assertion pattern.
• test_infer_heading_levels_integration passes filename=None, file=None, so it never exercises the outline extraction path. It's not testing integration at all.
• Tests never verify correctness of level assignment — they only check that values exist and are in [1, 6]. Any implementation that sets heading_level = 1 on everything would pass all tests.

14. Fixture Update Script Is a Blunt Instrument

        if isinstance(meta, dict) and "heading_level" not in meta:
            meta["heading_level"] = 1
            modified = True

Every Title in every fixture gets heading_level: 1 regardless of actual hierarchy. This masks the fact that the heuristic fallback is assigning arbitrary levels — the tests pass because the expected fixtures were patched to match whatever the code produced, not because the code is correct.

15. Version Bump in Feature PR

The PR bumps the version from 0.21.7 to 0.21.8. This will conflict with any other PR merged before this one that also needs a version bump, and it conflates feature work with release management.

Verdict

The core idea is sound — PDF heading levels are genuinely useful for downstream consumers. The outline-based extraction is the right primary strategy. However:

1. ~100 lines of dead code (the PDFMiner font extraction pipeline) should be removed or actually wired up.
2. The heuristic fallback is unreliable — per-page independent assignment and word-count scoring will produce wrong results frequently. It should either be removed or redesigned to work document-wide with actual font size data.
3. Triple PDF reading is a performance concern for large documents.
4. Tests don't verify correctness, only existence and range.
5. Three layers of exception swallowing make the feature fail silently and opaquely.

Hi, @PastelStorm , I analyzed your comments and fixed code.
please review again.
Thank you for your review.

Re-run

Re-run

Re-run

@Good0987 I am tempted to close this PR for low quality and general misunderstanding of the problem at hand.
I understand this might be one of the first open-source contributions in your career, therefore, I am giving you a lot of grace here. However, I encourage you to follow these steps that apply to any OSS project:

• always run linter and tests locally before pushing
• make sure your feature branch stays updated with the changes from main
• it's 2026, use AI to your advantage, ask two or three different models to review your code before you push
• make sure your docstrings, changelog, readme updates reflect your current changes
• make sure you test the actual behavior and not just add bloat to increase coverage

And also, we are all people. Treat the maintainers with respect, we have day jobs and most of us at Unstructured work very very long hours and some of us work weekends too. I assume you would like to be treated with respect, so please do the same for us. Thank you.

Code Review: Angel98518:feat/pdf-hierarchical-headings-4204 (updated)

Branch: 15 commits, 33 files changed (+1754 / -822), diff vs main

CRITICAL — Skipping azure.sh ingest test is the wrong fix

  'azure.sh'  # Azure fixture output varies with PDF heading-level inference; skip diff check

The commit history tells the story: the author tried three times to make the Azure fixtures match (commits 6f471a83, f3417bca, 9b1326af) and then gave up and skipped the test entirely in commit 004e221a. The comment says the output "varies" — but heading-level inference is deterministic, so "varies" really means "the fixtures don't match actual output and the author couldn't get them right."

This is a project-wide integration test that validates end-to-end Azure Blob Storage ingest correctness. Skipping it means:

• Any regression introduced to Azure pipeline output (unrelated to this feature) will go undetected.
• The heading_level values in the committed Azure fixtures are unvalidated — they're known to not match actual output.
• Other tests in tests_to_ignore (notion.sh, hubspot.sh, local-embed-mixedbreadai.sh) are skipped because they require external credentials or specific environments. azure.sh is fundamentally different — it's a diff-check test that should always pass if fixtures are correct.

The right fix is to generate the azure fixtures by actually running the pipeline, not by using the script below.

HIGH — The fixture update script generates wrong data

def add_heading_level_to_file(path: Path) -> bool:
    """Set heading_level on each Title's metadata by document order. Returns True if modified."""
    text = path.read_text(encoding="utf-8")
    data = json.loads(text)
    if not isinstance(data, list):
        return False
    modified = False
    title_idx = 0
    for item in data:
        if isinstance(item, dict) and item.get("type") == "Title":
            meta = item.get("metadata")
            if isinstance(meta, dict):
                new_level = min(title_idx + 1, 6)
                if meta.get("heading_level") != new_level:
                    meta["heading_level"] = new_level
                    modified = True
            title_idx += 1
    if modified:
        path.write_text(json.dumps(data, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
    return modified

This script assigns heading_level by naive document order (1st Title → H1, 2nd → H2, ...). But the actual inference logic uses PDF outline/bookmarks first, and only falls back to document-order for PDFs without outlines. For any PDF that has an outline (which is common for academic papers and reports — exactly what's in the fixture set), the script produces different values than the actual partitioner. This is the root cause of the azure.sh mismatch and explains why the author couldn't stabilize the fixtures.

This script should not be committed — it generates incorrect expected data. Fixtures should be produced by running the actual partitioner.

HIGH — do_Tj override silently removed

def do_TJ(self, seq):
    start = len(getattr(getattr(self.device, "cur_item", None), "_objs", ()))
    super().do_TJ(seq)
    self._patch_current_chars_with_render_mode(start)

The previous code overrode both do_TJ and do_Tj. The updated code only overrides do_TJ. In pdfminer.six, do_Tj delegates to do_TJ, so in the current version of pdfminer this is correct. However:

• The CHANGELOG still references do_Tj as being optimized, which is misleading.
• If a future pdfminer.six version changes do_Tj to not delegate to do_TJ, this will silently break. The old explicit override was more defensive.

Also, do__q (single-quote operator ') and do__w (double-quote operator ") also call do_TJ directly, so they are covered. But this relies on an implementation detail that isn't documented.

MEDIUM — Unrelated changes bundled into a feature branch

This PR bundles several unrelated changes that should be separate PRs:

1. Major dependency bumps (wrapt 1.x → 2.x, transformers 4.x → 5.x, weaviate-client 3.x → 4.x) — these are breaking semver changes with their own migration needs.
2. CI runner changes (ubuntu-latest → opensource-linux-8core) — infrastructure concern.
3. Weaviate test migration to v4 API (Client → connect_to_embedded, schema.create → collections.create_from_dict).
4. Filetype test skip decorators for Docker (BMP, HEIC, WAV).
5. .gitignore change (.venv → .venv*).
6. release-version-alert.yml continue-on-error: true addition.
7. Three version bumps in a feature branch (0.21.7, 0.21.8, 0.21.9).

Bundling these makes the PR unreviewable and means reverting the heading feature would also revert unrelated fixes. Version bumps especially should not live in a feature branch — they belong in the release process.

MEDIUM — infer_heading_levels_from_font_sizes is O(n*m) and has misleading name

def doc_order_key(el: Element) -> tuple[int, int]:
    page = el.metadata.page_number or 1
    idx = next(i for i, e in enumerate(elements) if e is el)
    return (page, idx)

sorted_titles = sorted(title_elements, key=doc_order_key)

1. Performance: doc_order_key does a linear scan of the full elements list for every title element. If there are N elements and M titles, sorting is O(M * N * log(M)). For large documents this is unnecessarily slow. The fix is trivial: build an identity-to-index map once.
2. Misleading name: The function is called infer_heading_levels_from_font_sizes but doesn't use font sizes at all. The docstring says "document-wide ordering" and layout_elements_map is explicitly deleted as unused. The name should reflect what it actually does (e.g., infer_heading_levels_by_document_order).
3. layout_elements_map parameter accepted and deleted: The del layout_elements_map pattern is a code smell. If the parameter isn't used, removing it from the signature is cleaner than accepting and discarding it.

MEDIUM — _maybe_infer_heading_levels captures mutable file from outer scope

def _maybe_infer_heading_levels(
    elements: list[Element],
) -> list[Element]:
    """Infer heading levels for PDF documents when appropriate."""
    if is_image:
        return elements
    try:
        outline_filename = filename
        file_for_outline: Optional[bytes | IO[bytes]] = None
        if filename is None and file is not None:
            if hasattr(file, "seek"):
                file.seek(0)
            file_for_outline = file
        # ...

This inner function is a closure that captures is_image, filename, and file from the enclosing partition_pdf_or_image. This has two problems:

1. It passes the original file object (not a copy of its bytes) to infer_heading_levels, which then passes it to PdfReader. If PdfReader advances the stream position, the file.seek(0) afterward may not fully recover state — e.g., if infer_heading_levels wraps the file in a new BytesIO.
2. The if filename is None guard is wrong: when filename is the empty string "" (which is the default), this branch is skipped because "" is falsy. But outline_filename is set to "", and PdfReader("") will raise a FileNotFoundError (caught by the broad except). The outline is silently lost for file-based invocations even when the file object is available.

LOW — Fuzzy matching has O(n*m) complexity and potential false positives

for outline_title, level in outline_map.items():
    similarity = SequenceMatcher(None, element_text, outline_title).ratio()
    if similarity > best_match_score and similarity >= fuzzy_match_threshold:
        best_match_score = similarity
        best_match_level = level
        if similarity >= 1.0:
            break

For each Title element, it computes SequenceMatcher.ratio() against every outline entry. With M titles and N outline entries, this is O(M * N * max_string_len). The 0.8 threshold means a 5-word title could match a completely different 5-word outline entry with 80% character overlap. There is no disambiguation by page number, which is available in both the elements and outline entries.

LOW — heading_level consolidation strategy is DROP

"heading_level": cls.DROP,

This means that when elements are chunked, heading_level is dropped from the resulting chunk metadata. This seems counterproductive — heading level is structural information that consumers would want preserved through chunking. It should probably be FIRST (take the heading level of the first pre-chunk element in the chunk).

LOW — Comment/docstring inconsistency

# -- heading level (1-4) for hierarchical document structure (H1, H2, H3, H4) --
heading_level: Optional[int]

The comment says "1-4" and "H1, H2, H3, H4" but the feature supports H1-H6 (1-6). This was a leftover from the original commit and never updated.

Summary table

Hi, @PastelStorm , I checked your comments and fixed.
Could you review again please?
Thank you for your review