Move parsers.py into a parsers/ package - #172

Review: circular-import workaround in parsers/__init__.py

Clean relocation overall — mechanically solid, 537 tests + 87 snapshots pass. One thing worth deciding on: the __getattr__ in parsers/__init__.py.

The cycle is real (verified)

Reproduced in a worktree: swapping the lazy init for an eager from .parsers import parse_serp fails on any entry, even plain import WebSearcher:

ImportError: cannot import name 'ClassifyFooter' from partially
initialized module 'WebSearcher.classifiers' (circular import)

Root cause is a layering inversion, not the move itself. The package now mixes two layers:

• Leaf layer — component.py, component_list.py, component_types.py, components/ — depended on by the lower layers extractors (from ..parsers.component_list import ComponentList) and classifiers (from ..parsers.component_types import ...).
• Orchestrator — parsers.py (parse_serp) — depends on the higher layer extractors.

Importing any leaf runs parsers/__init__.py first, so an eager init drags the orchestrator → extractors → classifiers in while they're mid-init. The __getattr__ defers that edge. It works and mirrors the existing SearchEngine lazy-load.

What it's actually buying

Not just WebSearcher.parse_serp. searchers/searchers.py:149 reaches it as a package attribute (from .. import parsers → parsers.parse_serp(...)), which resolved naturally on dev because parsers.py was a module. The __getattr__ is preserving the parsers.parse_serp attribute back-compat — that's the real reason it's there.

Better solution (verified, recommended)

Don't surface parse_serp through the package __init__ at all — then there's no edge to defer and no magic. Empty parsers/__init__.py (docstring only) + import by real path at the two internal call sites:

# WebSearcher/__init__.py
from .parsers.parsers import parse_serp        # was: from .parsers import parse_serp

# WebSearcher/searchers/searchers.py
from .. import logger, utils                   # drop `parsers`
from ..parsers.parsers import parse_serp
...
parsed = parse_serp(self.serp["html"], url=self.serp["url"])   # was: parsers.parse_serp(...)

Implemented and ran this: leaf-first import works (no cycle), ws.parse_serp (the only parse_serp in any __all__) intact, 537 tests + 87 snapshots pass. Sole trade-off: WebSearcher.parsers.parse_serp (the attribute) stops resolving — an implementation detail that worked incidentally because parsers.py used to be a module.

Decision rule:

• Fine to drop WebSearcher.parsers.parse_serp → Option A is strictly cleaner. (Quick check the downstream consumer doesn't import that attribute directly first.)
• Must keep it → keep the __getattr__, but reframe the comment as "preserve the parsers.parse_serp back-compat re-export" rather than purely cycle-avoidance.

Ideal fix (bigger, fine to defer)

The inversion exists only because the leaf trio lives beside the orchestrator despite being depended on below it. Moving them into their own leaf submodule (e.g. parsers/core/) removes it structurally — then __init__ could eager-export parse_serp with no hack and no lost attribute. Beyond a pure-relocation PR; reasonable as a follow-up. Would also relieve the "component" naming overload (component.py / component_list.py / component_types.py / components/ all in one namespace).