Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@terwey
Copy link
Collaborator

@terwey terwey commented Oct 20, 2025

fix(metadata)!: remove CreatedAt from Metadata; switch checksum to CRC32 to avoid midnight order loss

Orders that crossed a day boundary produced different Metadata due to the embedded day-truncated CreatedAt. E.g. an order at 23:50 with a state change at 00:10 generated a new metadata value, causing the system to “lose” the order. The timestamp is no longer part of Metadata; it is now taken from the associated BotEvent when needed.

Metadata hex layout stays 16 bytes but changes as follows:

Offset Size Field
0 4 BotID
4 4 DealID
8 4 BotEventID
12 4 CRC32(IEEE)

Additional changes:

  • Replace crc16 with hash/crc32 (IEEE table).
  • Update AsHex/FromHex accordingly.
  • Drop Metadata.CreatedAt usage in engine, API, storage, and tests; use *row.BotEvent.CreatedAt where a created timestamp is required.
  • Test vectors updated to new checksum scheme.

BREAKING CHANGE: Metadata serialization format changed (removed 2-byte day counter; CRC16 → CRC32, different positions). Existing 16-byte metadata produced by prior versions will fail checksum validation and must be regenerated/migrated.

…C32 to avoid midnight order loss

Orders that crossed a day boundary produced different Metadata due to the
embedded day-truncated `CreatedAt`. E.g. an order at 23:50 with a state
change at 00:10 generated a new metadata value, causing the system to
“lose” the order. The timestamp is no longer part of Metadata; it is now
taken from the associated BotEvent when needed.

Metadata hex layout stays 16 bytes but changes as follows:

| Offset | Size | Field        |
|-------:|-----:|--------------|
| 0      | 4    | BotID        |
| 4      | 4    | DealID       |
| 8      | 4    | BotEventID   |
| 12     | 4    | CRC32(IEEE)  |

Additional changes:
- Replace `crc16` with `hash/crc32` (IEEE table).
- Update `AsHex`/`FromHex` accordingly.
- Drop `Metadata.CreatedAt` usage in engine, API, storage, and tests; use
  `*row.BotEvent.CreatedAt` where a created timestamp is required.
- Test vectors updated to new checksum scheme.

BREAKING CHANGE: Metadata serialization format changed (removed 2-byte
day counter; CRC16 → CRC32, different positions). Existing 16-byte
metadata produced by prior versions will fail checksum validation and
must be regenerated/migrated.
@terwey terwey merged commit 12a656c into main Oct 20, 2025
1 check passed
@terwey terwey deleted the metadata branch October 31, 2025 10:49
terwey pushed a commit that referenced this pull request Nov 5, 2025
Updates MULTI_VENUE_BUGS.md to reflect:
- Resolved: SQL duplicate clauses (commit cd763cf)
- Clarified: OrderId is not redundant (tracks 3commas orders)
- Renumbered remaining bugs #1-8 (from #1-12)

Remaining critical issues:
- Bug #1: OrderWork comparability violation
- Bugs #2-6: Scaled orders multi-venue architecture
terwey pushed a commit that referenced this pull request Nov 5, 2025
…rable

The Action struct had pointer fields (Create, Modify, Cancel) which made
it non-comparable in Go. Since OrderWork embeds Action and the typed queue
requires comparable structs, this violated Go's comparability requirements.

The fix removes the pointers and uses value types instead. The ActionType
field already indicates which action is active, making nil checks redundant.

This change:
- Changes Action fields from pointers to values
- Updates all action creation sites to use values instead of pointers
- Removes all nil pointer checks, replacing them with ActionType checks
- Updates all dereferences to use values directly
- Updates test assertions to remove NotNil checks

Resolves Bug #1: OrderWork Comparability Violation as described in
specs/multi_venue_emission.adoc
terwey pushed a commit that referenced this pull request Nov 5, 2025
Updates MULTI_VENUE_BUGS.md to reflect:
- Resolved: SQL duplicate clauses (commit cd763cf)
- Clarified: OrderId is not redundant (tracks 3commas orders)
- Renumbered remaining bugs #1-8 (from #1-12)

Remaining critical issues:
- Bug #1: OrderWork comparability violation
- Bugs #2-6: Scaled orders multi-venue architecture
terwey pushed a commit that referenced this pull request Nov 5, 2025
Updated MULTI_VENUE_BUGS.md to reflect resolved issues:

Resolved (PRs #75 and #76):
- Bug #1: OrderWork comparability violation
- Bug #2-4: Scaled orders multi-venue architecture

Remaining issues (renumbered #1-4):
- Bug #1: Missing scaled_orders in wallet migration (High)
- Bug #2: Take profit reconciliation fan-out (Medium)
- Bug #3: Fill tracker memory growth (Medium)
- Bug #4: Replay logic edge case (Low)

PRs:
- #75: Removed pointer fields from Action struct
- #76: Fixed multi-venue scaled order handling
terwey pushed a commit that referenced this pull request Nov 5, 2025
Add scaled_orders table migration to migrateDefaultVenueWalletLocked:
- Clone scaled_orders from old wallet to new wallet
- Delete scaled_orders associated with old wallet

This ensures the scaled order audit trail is preserved when a user
changes their Hyperliquid wallet address, matching the behavior of
hyperliquid_submissions and hyperliquid_status_history tables.

Fixes Bug #1: Missing scaled_orders in Wallet Migration
terwey pushed a commit that referenced this pull request Nov 5, 2025
Critical fixes for multi-venue take-profit reconciliation:

Issue #1 (P1): Each venue's take-profit was being sized to the GLOBAL net
position instead of that specific venue's position. This caused oversized
exit orders (e.g., if venue A had 60 and venue B had 40, both would try
to exit 100, totaling 200 and overshooting inventory).

Fix: Added calculateVenuePositions() to compute net position per venue
by summing filled buy/sell quantities for each venue identifier. Each
venue's take-profit is now correctly sized to match only its position.

Issue #2 (P1): ensureTakeProfit was only called when NO active take-profits
existed. If one venue had an active TP, other venues without TPs would
never get them created, leaving the multi-venue fan-out incomplete.

Fix: Changed logic to iterate through ALL venue positions and reconcile
or create take-profits for each venue independently. Now checks per-venue:
- If venue has position but no TP: create one
- If venue has position and TP: reconcile size
- If venue has no position but TP exists: cancel it

Location: filltracker/service.go:155-215
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants