Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit c8c5015

Browse files
bamzedevbamzedev
authored
Add arch doc (#542)
1 parent 98495c7 commit c8c5015

1 file changed

Lines changed: 278 additions & 0 deletions

File tree

‎docs/ARCHITECTURE.md‎

Lines changed: 278 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,278 @@
1+
# Strawberry Architecture Overview
2+
3+
Strawberry is a JAM (JOIN-ACCUMULATE MACHINE) client implementation for Polkadot, written in Go. It is developed by Eiger and follows the JAM graypaper specification (v0.7.2).
4+
5+
## Project Structure
6+
7+
```
8+
strawberry/
9+
├── cmd/ # Command-line executables
10+
│ ├── strawberry/ # Main node application (Milestone 2)
11+
├── pkg/ # Public packages (stable APIs)
12+
│ ├── network/ # Networking layer (QUIC, P2P)
13+
│ ├── conformance/ # JAM conformance testing node (Milestone 1)
14+
│ ├── db/ # Database abstraction (PebbleDB)
15+
│ ├── log/ # Logging utilities
16+
│ ├── serialization/ # JAM codec implementation
17+
├── internal/ # Internal packages (core implementation)
18+
│ ├── block/ # Block structures & operations (Extrinsics)
19+
│ ├── state/ # Chain state management (Merklization, Serialization, Block Sealing)
20+
│ ├── statetransition/ # State transition logic
21+
│ ├── store/ # State & data persistence
22+
│ ├── chain/ # Chain service (Finalization, Fork handling)
23+
│ ├── validator/ # Validator management (connections, keys, shards)
24+
│ ├── work/ # Work packages & bundles
25+
│ ├── service/ # Service accounts & state
26+
│ ├── crypto/ # Cryptographic operations (Including Bandersnatch via Rust FFI)
27+
│ ├── pvm/ # Polkadot Virtual Machine
28+
│ ├── guaranteeing/ # Work report guarantees (Guarantee Extrinsic validation and production)
29+
│ ├── assuring/ # Availability assurance (Assurance Extrinsic validation and production)
30+
│ ├── disputing/ # Dispute mechanisms (Dispute Extrinsic validation and production)
31+
│ ├── safrole/ # Block ticket algorithm
32+
│ ├── jamtime/ # Time slot management
33+
│ ├── merkle/ # Merkle tree structures
34+
│ ├── erasurecoding/ # Reed-Solomon erasure coding (Rust FFI)
35+
│ ├── d3l/ # Data Availability Layer (Segment fetching)
36+
│ ├── merkle/ # Merkle tree structures
37+
│ ├── refine/ # PVM Refinement operations
38+
│ ├── safemath/ # Safe arithmetic operations
39+
│ └── authorization/ # PVM Authorization mechanisms
40+
├── bandersnatch/ # Rust FFI: Bandersnatch cryptography
41+
├── erasurecoding/ # Rust FFI: Reed-Solomon encoding
42+
└── tests/ # Integration & simulation tests
43+
```
44+
45+
## Core Components
46+
47+
### 1. Network Layer (`pkg/network/`)
48+
49+
The networking layer provides peer-to-peer communication using QUIC protocol.
50+
51+
| Component | Purpose |
52+
|-----------|---------|
53+
| `node/` | Central manager for peer connections and protocol handling |
54+
| `transport/` | QUIC-based transport layer via `quic-go` |
55+
| `protocol/` | ALPN-based protocol negotiation and message routing |
56+
| `handlers/` | Message handlers (work packages, tickets, blocks, segments) |
57+
| `peer/` | Peer discovery and connection management |
58+
| `cert/` | TLS certificate generation with Ed25519 keys |
59+
60+
### 2. State Management (`internal/state/`, `internal/statetransition/`)
61+
62+
Manages the complete system state and state transitions.
63+
64+
**State Structure:**
65+
```
66+
State
67+
├── Services # Service accounts with code and storage
68+
├── ValidatorState # Current, archived, and queued validators
69+
├── CoreAssignments # Work-reports assigned to cores
70+
├── RecentHistory # Recent block information
71+
├── TimeslotIndex # Current time slot
72+
├── EntropyPool # Randomness for consensus
73+
├── ActivityStatistics # Validator performance metrics
74+
├── AccumulationQueue # Pending work-reports
75+
└── PastJudgements # Dispute history
76+
```
77+
78+
**State Transition Flow:**
79+
1. Verify block header (merkle proof validation)
80+
2. Parallel validation of extrinsics (preimages, guarantees, assurances, disputes)
81+
3. Update state components
82+
4. Persist new state root
83+
84+
### 3. Block Processing (`internal/block/`, `internal/chain/`)
85+
86+
Handles block structure, validation, and chain management.
87+
88+
**Block Structure:**
89+
```
90+
Block
91+
├── Header
92+
│ ├── ParentHash
93+
│ ├── TimeSlotIndex
94+
│ ├── BlockAuthorIndex
95+
│ ├── RecentHistoryRoot
96+
│ ├── EntropySource
97+
│ └── Seals
98+
└── Extrinsic
99+
├── TicketExtrinsic (ET) # Safrole tickets
100+
├── PreimageExtrinsic (EP) # Preimage data
101+
├── GuaranteesExtrinsic (EG) # Work report guarantees
102+
├── AssurancesExtrinsic (EA) # Availability assurances
103+
└── DisputeExtrinsic (ED) # Dispute verdicts
104+
```
105+
106+
### 4. Polkadot Virtual Machine (`internal/pvm/`)
107+
108+
Executes service code in a sandboxed environment.
109+
110+
| Component | Purpose |
111+
|-----------|---------|
112+
| `program.go` | Program blob parsing and validation |
113+
| `step.go` | Single-step instruction execution |
114+
| `mutations.go` | State mutation operations |
115+
| `host_call/` | Host function implementations |
116+
117+
**Memory Model:**
118+
- Read-only data section
119+
- Read-write data section
120+
- Heap (grows upward)
121+
- Stack (grows downward)
122+
123+
### 5. Cryptography (`internal/crypto/`)
124+
125+
Cryptographic operations supporting JAM's security requirements.
126+
127+
| Scheme | Implementation | Purpose |
128+
|--------|----------------|---------|
129+
| Bandersnatch | Rust FFI | Ring VRF signatures for anonymous validator selection |
130+
| Ed25519 | Go native | Standard digital signatures |
131+
| Blake2b | Go native | Hashing |
132+
133+
### 6. Validator & Consensus (`internal/validator/`, `internal/safrole/`)
134+
135+
Manages validator state and the Safrole block production algorithm.
136+
137+
- **ValidatorManager**: Key management and validator state
138+
- **ValidatorGrid**: Core-to-validator assignments
139+
- **Safrole**: Decentralized, fair block production
140+
141+
### 7. Work Execution Pipeline
142+
143+
```
144+
Work Package Submission
145+
│
146+
▼
147+
┌───────────────────┐
148+
│ Network Handler │
149+
└─────────┬─────────┘
150+
│
151+
▼
152+
┌───────────────────┐
153+
│ PVM Execution │
154+
│ (if refinement) │
155+
└─────────┬─────────┘
156+
│
157+
▼
158+
┌───────────────────┐
159+
│ Work Report │
160+
│ Generation │
161+
└─────────┬─────────┘
162+
│
163+
▼
164+
┌───────────────────┐
165+
│ Guarantor │
166+
│ Selection │
167+
└─────────┬─────────┘
168+
│
169+
▼
170+
┌───────────────────┐
171+
│ Signature │
172+
│ Aggregation │
173+
└─────────┬─────────┘
174+
│
175+
▼
176+
┌───────────────────┐
177+
│ Block Inclusion │
178+
│ & Accumulation │
179+
└───────────────────┘
180+
```
181+
182+
### 8. Data Availability (`internal/erasurecoding/`, `internal/d3l/`)
183+
184+
Ensures data availability through erasure coding.
185+
186+
- **Reed-Solomon Encoding**: Via Rust FFI for performance
187+
- **Segment Fetcher**: Retrieves segments from the D3L (Data Dissemination Layer)
188+
- **Shards Store**: Persists encoded data shards
189+
190+
### 9. Storage (`internal/store/`, `pkg/db/`)
191+
192+
Layered storage architecture using PebbleDB.
193+
194+
| Store | Purpose |
195+
|-------|---------|
196+
| Chain Store | Blocks and headers |
197+
| Trie Store | Merkle trie for state |
198+
| Ticket Store | Safrole ticket state |
199+
| Shards Store | Erasure-coded data shards |
200+
201+
## Key Dependencies
202+
203+
### Go Dependencies
204+
- `cockroachdb/pebble`: Key-value storage
205+
- `quic-go`: QUIC protocol implementation
206+
- `rs/zerolog`: Structured logging
207+
- `golang.org/x/crypto`: Cryptographic primitives
208+
- `ebitengine/purego`: FFI without CGO
209+
210+
### Rust FFI Components
211+
- **Bandersnatch** (`bandersnatch/`): Ring VRF signatures
212+
- **Erasure Coding** (`erasurecoding/`): Reed-Solomon encoding
213+
214+
## Design Principles
215+
216+
### 1. Modular Package Structure
217+
- `internal/` packages hide implementation details
218+
- `pkg/` packages expose stable, public APIs
219+
- Single responsibility per package
220+
221+
### 2. Concurrent Processing
222+
- Independent validation steps run in parallel via `errgroup`
223+
- Configurable concurrency for debugging
224+
225+
### 3. FFI via Pure Go
226+
- Uses `purego` to load Rust libraries without CGO overhead
227+
- Dynamic linking at runtime
228+
- Cross-platform support (macOS, Linux)
229+
230+
### 4. Performance Optimizations
231+
- Cached state roots to avoid re-merkleization
232+
- Fast-path header verification
233+
- Parallel extrinsic validation
234+
235+
## Build Configuration
236+
237+
### Build Tags
238+
| Tag | Purpose |
239+
|-----|---------|
240+
| `tiny` | Minimal configuration for quick tests |
241+
| `conformance` | Conformance test configuration |
242+
| `full` | Full protocol parameters |
243+
244+
### Prerequisites
245+
- Go 1.25.5+
246+
- Rust 1.81.1+
247+
- Make
248+
249+
### Common Commands
250+
```bash
251+
make build # Build main binary
252+
make test # Run unit tests
253+
make integration # Run integration tests
254+
make build-conformance # Build conformance runner
255+
make lint # Run linter
256+
```
257+
258+
## Testing
259+
260+
### Unit Tests
261+
Located alongside source files in `internal/` packages.
262+
263+
### Integration Tests (`tests/integration/`)
264+
- Trace execution (safrole, storage, preimages, fuzzy)
265+
- PVM execution
266+
- State transitions
267+
- Codec encoding/decoding
268+
- Merkle structures
269+
- Disputes and assurances
270+
271+
### Conformance Tests (`pkg/conformance/`)
272+
Formal JAM specification conformance testing with external fuzzer support.
273+
274+
## Protocol Compliance
275+
276+
- **JAM Graypaper Version**: 0.7.2
277+
- **Features**: Ancestry validation, fork handling
278+
- **Milestone 1**: State-transitioning conformance tests (completed)

0 commit comments

Comments
 (0)