Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 5a98150

Browse files
committed
feat(ast): add owned visitor, fold, and printer API for lifetime-free AST
Adds complete owned-AST infrastructure so users of `parse()` can traverse, transform, and print results without touching the arena or managing lifetimes: - `php_ast::owned::visitor` — `OwnedVisitor`, `OwnedScopeVisitor`, `OwnedScopeWalker`, and `walk_owned_*` free functions mirroring the arena visitor - `php_ast::owned::fold` — `FoldOwned` trait and `fold_owned_*` free functions; identity defaults rebuild the tree identically; includes `fold_closure_use_var` hook that was silently bypassed before - `php_ast::owned` re-exports all `fold_owned_*` and `walk_owned_*` symbols so callers need only `use php_ast::owned::*` - `php_printer` — `pretty_print_owned`, `pretty_print_owned_file`, `pretty_print_owned_with_config`, `pretty_print_owned_with_comments`, and `pretty_print_owned_with_comments_and_config`; convert owned→arena via a short-lived bump arena internally - `php-parser` `ParserContext` — `reparse_owned` and `reparse_owned_versioned` return lifetime-free `ParseResult` - Splits `crates/php-ast/src/owned.rs` into a submodule directory (`owned/mod.rs`, `owned/visitor.rs`, `owned/fold.rs`) - README rewrite: `parse()` is now the default entry point; owned visitor/fold/printer examples precede arena variants; "when to use arena vs. owned" decision table added
1 parent fcb8d06 commit 5a98150

9 files changed

Lines changed: 3495 additions & 36 deletions

File tree

.github/workflows/ci.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ jobs:
4949
key: ${{ matrix.php-version }}
5050

5151
- name: Check PHP syntax in tests
52-
run: cargo test --test php_syntax
52+
run: cargo test --test integration -- php_syntax::
5353

5454
test:
5555
name: Test (PHP ${{ matrix.php-version }})

README.md

Lines changed: 132 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -9,13 +9,11 @@ A fast, fault-tolerant PHP parser written in Rust. Produces a full typed AST wit
99
```toml
1010
[dependencies]
1111
php-rs-parser = "*"
12-
php-ast = "*" # AST types and Visitor trait
13-
14-
# Needed when using parse_arena(), the Visitor/Fold traits, or the printer
15-
bumpalo = "*"
12+
php-ast = "*" # AST types and visitor/fold traits
1613

1714
# Optional
1815
php-printer = "*" # pretty-print AST back to PHP source
16+
bumpalo = "*" # only needed when using parse_arena() directly
1917
```
2018

2119
## Quick Start
@@ -41,8 +39,11 @@ let pos = result.source_map.offset_to_line_col(6);
4139

4240
- **`parse()` / `parse_versioned()`** — main entry points; return a fully-owned `ParseResult` with no lifetime parameters. See [`docs.rs/php-rs-parser`](https://docs.rs/php-rs-parser)
4341
- **`parse_arena()` / `parse_arena_versioned()`** — arena-allocated variants for LSP servers and hot paths; return `ArenaParseResult<'arena, 'src>`
44-
- **`ParserContext`** — reusable context for repeated re-parses; wraps the arena API
45-
- **`Visitor` / `ScopeVisitor`** — AST traversal traits; see [`docs.rs/php-ast`](https://docs.rs/php-ast) for the visitor infrastructure
42+
- **`ParserContext`** — reusable context for repeated re-parses; `reparse_owned()` returns a lifetime-free `ParseResult`, `reparse()` returns the arena form
43+
- **`OwnedVisitor` / `OwnedScopeVisitor`** — traverse a `ParseResult` AST with no arena involved; see `php_ast::owned::visitor`
44+
- **`FoldOwned`** — transform a `ParseResult` AST into a new owned AST; see `php_ast::owned::fold`
45+
- **`Visitor` / `ScopeVisitor`** — arena-form traversal traits; see [`docs.rs/php-ast`](https://docs.rs/php-ast)
46+
- **`Fold`** — arena-form transformation trait; reads one arena, writes another
4647
- **`ParseError` variants** — see [`crates/php-parser/src/diagnostics.rs`](crates/php-parser/src/diagnostics.rs) for all variants and recovery behavior
4748
- **AST node types** — see [`docs.rs/php-ast/ast`](https://docs.rs/php-ast/latest/php_ast/ast/index.html) for the full set of statement, expression, and declaration nodes
4849

@@ -104,11 +105,22 @@ assert!(!result.program.stmts.is_empty()); // AST still produced
104105

105106
### Re-parsing (LSP / editor use)
106107

107-
Use `ParserContext` when parsing the same document repeatedly (e.g. on every keystroke). It reuses the backing arena memory in O(1), avoiding allocator churn. The returned `ArenaParseResult` borrows from the context's arena — drop it before the next re-parse:
108+
Use `ParserContext` when parsing the same document repeatedly (e.g. on every keystroke). It reuses the backing arena memory in O(1), avoiding allocator churn.
109+
110+
`reparse_owned()` returns a fully-owned `ParseResult` — no lifetimes, no `drop` discipline required:
108111

109112
```rust
110113
let mut ctx = php_rs_parser::ParserContext::new();
111114

115+
let a = ctx.reparse_owned("<?php echo 1;");
116+
let b = ctx.reparse_owned("<?php echo 2;"); // a can stay alive
117+
assert!(a.errors.is_empty());
118+
assert!(b.errors.is_empty());
119+
```
120+
121+
`reparse()` returns an `ArenaParseResult` that borrows from the context arena. The borrow checker prevents calling `reparse` again while that result is alive — drop it first:
122+
123+
```rust
112124
let result = ctx.reparse("<?php echo 1;");
113125
assert!(result.errors.is_empty());
114126
drop(result); // must be dropped before the next reparse
@@ -117,22 +129,73 @@ let result = ctx.reparse("<?php echo 2;");
117129
assert!(result.errors.is_empty());
118130
```
119131

120-
`reparse_versioned` is also available for targeting a specific PHP version.
132+
`reparse_versioned` and `reparse_owned_versioned` are also available for targeting a specific PHP version.
121133

122-
### Arena API
134+
### Visitor API (owned)
123135

124-
When integrating with the printer or visitor (which take arena-allocated types), use `parse_arena` directly:
136+
`OwnedVisitor` works directly on a `ParseResult` — no arena, no lifetime parameters. Override only the node types you care about:
125137

126138
```rust
127-
let arena = bumpalo::Bump::new();
128-
let result = php_rs_parser::parse_arena(&arena, "<?php echo 1;");
129-
// result.program is Program<'_, '_> — works with the printer and visitor traits
130-
let output = php_printer::pretty_print(&result.program);
139+
use php_ast::owned::visitor::{OwnedVisitor, walk_owned_expr};
140+
use php_ast::owned::{Expr, ExprKind};
141+
use std::ops::ControlFlow;
142+
143+
struct VarCounter { count: usize }
144+
145+
impl OwnedVisitor for VarCounter {
146+
fn visit_expr(&mut self, expr: &Expr) -> ControlFlow<()> {
147+
if matches!(&expr.kind, ExprKind::Variable(_)) {
148+
self.count += 1;
149+
}
150+
walk_owned_expr(self, expr)
151+
}
152+
}
153+
154+
let result = php_rs_parser::parse("<?php $x = $y + $z;");
155+
let mut v = VarCounter { count: 0 };
156+
v.visit_program(&result.program);
157+
assert_eq!(v.count, 3);
131158
```
132159

133-
### Visitor API
160+
Return `ControlFlow::Break(())` to stop traversal early. Return `ControlFlow::Continue(())` without calling `walk_owned_*` to skip a subtree.
161+
162+
#### Scope-aware owned traversal
134163

135-
Implement `Visitor` to walk the AST depth-first. Override only the node types you care about; the default implementations recurse into children automatically.
164+
Use `OwnedScopeVisitor` + `OwnedScopeWalker` when you need to know **which namespace, class, or function** you are currently inside:
165+
166+
```rust
167+
use php_ast::owned::visitor::{OwnedScopeVisitor, OwnedScopeWalker, OwnedScope};
168+
use php_ast::owned::{ClassMember, ClassMemberKind};
169+
use std::ops::ControlFlow;
170+
171+
struct MethodCollector { methods: Vec<String> }
172+
173+
impl OwnedScopeVisitor for MethodCollector {
174+
fn visit_class_member(
175+
&mut self,
176+
member: &ClassMember,
177+
scope: &OwnedScope,
178+
) -> ControlFlow<()> {
179+
if let ClassMemberKind::Method(m) = &member.kind {
180+
self.methods.push(format!(
181+
"{}::{}",
182+
scope.class_name.as_deref().unwrap_or("<anon>"),
183+
m.name,
184+
));
185+
}
186+
ControlFlow::Continue(())
187+
}
188+
}
189+
190+
let result = php_rs_parser::parse("<?php class Foo { function bar() {} }");
191+
let mut walker = OwnedScopeWalker::new(MethodCollector { methods: vec![] });
192+
walker.walk(&result.program);
193+
// walker.into_inner().methods == ["Foo::bar"]
194+
```
195+
196+
### Visitor API (arena)
197+
198+
Use the arena `Visitor` when you need maximum throughput and manage the arena lifetime yourself:
136199

137200
```rust
138201
use php_ast::visitor::{Visitor, walk_expr};
@@ -151,11 +214,7 @@ impl<'arena, 'src> Visitor<'arena, 'src> for VarCounter {
151214
}
152215
```
153216

154-
Return `ControlFlow::Break(())` to stop traversal early. Return `ControlFlow::Continue(())` without calling `walk_*` to skip a subtree.
155-
156-
### Scope-aware traversal
157-
158-
Use `ScopeVisitor` + `ScopeWalker` when your visitor needs to know **which namespace, class, or function** it is currently inside. Every visit method receives a `Scope` with that context:
217+
Use `ScopeVisitor` + `ScopeWalker` for scope-aware arena traversal:
159218

160219
```rust
161220
use php_ast::visitor::{ScopeVisitor, ScopeWalker, Scope};
@@ -188,13 +247,33 @@ walker.walk(&result.program);
188247
// walker.into_inner().methods == ["Foo::bar"]
189248
```
190249

191-
Use plain `Visitor` when you don't need namespace/class/function context.
250+
### AST transformation — FoldOwned
251+
252+
`FoldOwned` transforms a `ParseResult` AST into a new owned AST. Override only the node types you want to change; all others are rebuilt identically:
192253

193-
### AST transformation (Fold)
254+
```rust
255+
use php_ast::owned::fold::{FoldOwned, fold_owned_expr};
256+
use php_ast::owned::{Expr, ExprKind};
194257

195-
`Fold` is the transformation counterpart of `Visitor`. Where `Visitor` reads a tree in place, `Fold` rebuilds it — reading from an input arena and writing into a new output arena. This is the correct design for arena-allocated ASTs: in-place mutation would break the arena lifetime invariant.
258+
struct NegateInts;
196259

197-
Implement `Fold` and override only the node types you want to change. All other nodes are rebuilt identically by the default implementations:
260+
impl FoldOwned for NegateInts {
261+
fn fold_expr(&mut self, expr: &Expr) -> Expr {
262+
if let ExprKind::Int(n) = &expr.kind {
263+
return Expr { kind: ExprKind::Int(-n), span: expr.span };
264+
}
265+
fold_owned_expr(self, expr)
266+
}
267+
}
268+
269+
let result = php_rs_parser::parse("<?php $x = 1;");
270+
let transformed = NegateInts.fold_program(&result.program);
271+
// transformed is a new owned::Program with all integers negated
272+
```
273+
274+
### AST transformation — Fold (arena)
275+
276+
`Fold` is the arena-form transformation trait. It reads from one arena and writes into a new output arena — the correct design for arena-allocated ASTs where in-place mutation would break lifetime invariants:
198277

199278
```rust
200279
use bumpalo::Bump;
@@ -236,18 +315,29 @@ for param in find_tags(&doc, "param") {
236315

237316
### Pretty printer
238317

318+
`pretty_print_owned` works directly on a `ParseResult` — no arena needed:
319+
320+
```rust
321+
let result = php_rs_parser::parse("<?php echo 1 + 2;");
322+
let output = php_printer::pretty_print_owned(&result.program);
323+
// output == "<?php\necho 1 + 2;"
324+
```
325+
326+
Use `pretty_print_owned_file` to append a trailing newline. Use `pretty_print_owned_with_config` for custom indentation.
327+
328+
When using the arena API (e.g. inside an LSP handler that already holds an `ArenaParseResult`), use the arena-form functions directly to avoid an extra conversion:
329+
239330
```rust
240331
let arena = bumpalo::Bump::new();
241332
let result = php_rs_parser::parse_arena(&arena, "<?php echo 1 + 2;");
242333
let output = php_printer::pretty_print(&result.program);
243-
// output == "<?php\necho 1 + 2;"
244334
```
245335

246-
Use `pretty_print_file` to produce a complete file with a `<?php\n\n` prefix and trailing newline.
247-
248336
To preserve comments in the output, use `pretty_print_with_comments`:
249337

250338
```rust
339+
let arena = bumpalo::Bump::new();
340+
let result = php_rs_parser::parse_arena(&arena, "<?php // comment\necho 1;");
251341
let output = php_printer::pretty_print_with_comments(
252342
&result.program,
253343
result.source,
@@ -261,7 +351,7 @@ To customise indentation or newlines, pass a `PrinterConfig`:
261351
use php_printer::{PrinterConfig, Indent};
262352

263353
let config = PrinterConfig { indent: Indent::Spaces(2), ..Default::default() };
264-
let output = php_printer::pretty_print_with_config(&result.program, &config);
354+
let output = php_printer::pretty_print_owned_with_config(&result.program, &config);
265355
```
266356

267357
## Architecture
@@ -271,11 +361,21 @@ Four crates, one workspace:
271361
| Crate | crates.io | Purpose |
272362
|-------|-----------|---------|
273363
| **php-lexer** | [![crates.io](https://img.shields.io/crates/v/php-lexer)](https://crates.io/crates/php-lexer) | Hand-written tokenizer with handling for strings, heredoc/nowdoc, and inline HTML |
274-
| **php-ast** | [![crates.io](https://img.shields.io/crates/v/php-ast)](https://crates.io/crates/php-ast) | AST type definitions, `Visitor` trait, `ScopeVisitor` trait, owned (lifetime-free) AST types |
364+
| **php-ast** | [![crates.io](https://img.shields.io/crates/v/php-ast)](https://crates.io/crates/php-ast) | AST type definitions; arena `Visitor`/`ScopeVisitor`/`Fold` traits; owned (lifetime-free) `OwnedVisitor`/`OwnedScopeVisitor`/`FoldOwned` traits |
275365
| **php-rs-parser** | [![crates.io](https://img.shields.io/crates/v/php-rs-parser)](https://crates.io/crates/php-rs-parser) | Pratt-based recursive descent parser with panic-mode error recovery, PHPDoc parser, source map |
276-
| **php-printer** | [![crates.io](https://img.shields.io/crates/v/php-printer)](https://crates.io/crates/php-printer) | Pretty printer — converts an AST back to PHP source |
366+
| **php-printer** | [![crates.io](https://img.shields.io/crates/v/php-printer)](https://crates.io/crates/php-printer) | Pretty printer — converts an AST back to PHP source; supports both arena and owned AST |
367+
368+
Source flows through `Lexer → Parser → arena-allocated AST nodes`. The lexer is lazy (tokens produced on demand with peeking slots); the parser is Pratt-based recursive descent with panic-mode error recovery. The owned AST (`php_ast::owned`) provides lifetime-free mirrors of every node type, enabling storage and manipulation without arena lifetime constraints.
369+
370+
**When to use the arena API vs. the owned API:**
277371

278-
Source flows through `Lexer → Parser → arena-allocated AST nodes`. The lexer is lazy (tokens produced on demand with peeking slots); the parser is Pratt-based recursive descent with panic-mode error recovery.
372+
| Use case | Recommended API |
373+
|---|---|
374+
| One-shot parsing, CLI tools, batch processing | `parse()``ParseResult` (owned) |
375+
| Store results in `HashMap`, send across threads | `parse()``ParseResult` (owned) |
376+
| Walk or transform a `ParseResult` | `OwnedVisitor` / `FoldOwned` |
377+
| LSP server, repeated re-parses | `ParserContext::reparse_owned()` or `reparse()` |
378+
| Maximum throughput, arena lifetime under your control | `parse_arena()` → arena `Visitor` / `Fold` |
279379

280380
## Performance
281381

crates/php-ast/Cargo.toml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,6 @@ readme = "../../README.md"
1414
[dependencies]
1515
serde = { workspace = true }
1616
bumpalo = { workspace = true }
17+
18+
[dev-dependencies]
19+
serde_json = { workspace = true }

0 commit comments

Comments
 (0)