We are currently implementing an incremental analysis system for JET and
JETLS by integrating them with Revise. We considered other approaches
such as implementing their own Revise-like system on the JET[LS] side
(e.g. #938), but concluded that implementing
incremental analysis by making Revise itself extensible would be better
in terms of implementation reliability and future maintenance costs.
This commit implements the first step towards this goal by extending
Revise's most important data structure `pkgdatas` to hold custom data
and providing utilities for interacting with it. Specifically, this
commit extends the leaf nodes of the `pkgdatas` data structure,
`MethodInfoKey(method_table => signature)`, to a new data structure
`SigInfo(method_table, signature, ext_data::ExtendedData)`.
`SigInfo` is defined as follows:
```julia
struct SigInfo
mt::Union{Nothing,MethodTable}
sig::Type
ext::ExtendedData
end
struct ExtendedData
owner::Symbol
data::Any
next::ExtendedData
ExtendedData(owner::Symbol, data, next::ExtendedData) = new(owner, data, next)
ExtendedData(owner::Symbol, data) = new(owner, data)
ExtendedData() = new(:Revise, nothing)
end
```
As seen from the type definition, `SigInfo` can hold `ExtendedData` in
addition to the information that the conventional `MethodInfoKey` holds.
`ExtendedData` is implemented as a linked list, enabling safe data
management per owner through the `owner` field (similar to
`Core.CodeInstance` design).
We also implement the following utility functions for getting or
replacing `ext_data`:
- `get_extended_data(ext::ExtendedData, owner::Symbol) -> ext::Union{ExtendedData,Nothing}`
- `get_extended_data(siginfo::SigInfo, owner::Symbol) -> ext::Union{ExtendedData,Nothing}`
- `replace_extended_data(ext::ExtendedData, owner::Symbol, data) -> new_ext::ExtendedData`
- `replace_extended_data(siginfo::SigInfo, owner::Symbol, data) -> new_siginfo::SigInfo`
This commit implements these simple extension setup only. To actually
use this extension system, external packages need to decide when to
inspect `Revise.pkgdatas` by themselves and use functions like
`replace_extended_data` to hold custom data. While packages using this
extension system and Revise's revision pipeline are not tightly
synchronized, stale `SigInfo` objects are automatically removed,
enabling minimal external incremental systems. Specifically, using this
commit, we can replace `JET.report_package` with a Revise-based
implementation and make it incremental (aviatesk/JET.jl#763).
In the future, we may need to set up some callbacks in Revise's revision
pipeline to allow external packages to customize in sync with Revise's
revision pipeline. However, since the necessary requirements are still
unclear at the current stage of extending `JET.report_package`, this
commit does not implement them.