fix(redis): Preserve millisecond timestamp precision for Redis online store #5807
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it
This PR fixes an issue in the Redis online store where Feast drops sub-second
precision when comparing and storing event timestamps through the
/pushendpoint.
Problem
Feast currently serializes event timestamps using only the seconds field of
the protobuf
Timestamp(Timestamp.seconds), discardingTimestamp.nanos.As a result:
47.500Zand47.700Z)are treated as having the same timestamp.
/pushendpoint rejects later events as “older”, even when they occurmilliseconds later.
high-frequency sources (streaming pipelines, event collectors, etc.).
What this PR changes
✔ Store full protobuf
Timestampincluding nanosonline_write_batchnow constructs timestamps using:which preserves both
secondsandnanos.✔ Compare event freshness using nanosecond precision
Old behavior:
New behavior:
This ensures that sub-second updates are correctly accepted.
✔ Reconstruct full timestamp in
online_readFeast now returns proper millisecond-aware timestamps by combining
seconds + nanos / 1e9.Result
Feast users can now push multiple updates that occur within the same second and
Redis will correctly:
get_online_features.This enables correct behavior for any use case emitting dense real-time data.
Which issue(s) this PR fixes
Fixes a long-standing limitation in the Redis online store where timestamp
precision was effectively second-level, causing incorrect deduplication and
event rejection when using the
/pushAPI.(If maintainers want an issue link, I can open one in feast-dev.)
Misc
work as before.
Timestampin otheronline store implementations.