Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@jatin5251
Copy link

What this PR does / why we need it

This PR fixes an issue in the Redis online store where Feast drops sub-second
precision when comparing and storing event timestamps through the /push
endpoint.

Problem

Feast currently serializes event timestamps using only the seconds field of
the protobuf Timestamp (Timestamp.seconds), discarding Timestamp.nanos.
As a result:

  • Two events occurring within the same second (e.g. 47.500Z and 47.700Z)
    are treated as having the same timestamp.
  • The /push endpoint rejects later events as “older”, even when they occur
    milliseconds later.
  • The online store cannot guarantee correct last-write-wins semantics for
    high-frequency sources (streaming pipelines, event collectors, etc.).

What this PR changes

✔ Store full protobuf Timestamp including nanos

online_write_batch now constructs timestamps using:

ts = Timestamp()
ts.FromDatetime(timestamp)

which preserves both seconds and nanos.

✔ Compare event freshness using nanosecond precision

Old behavior:

if new_seconds <= prev_seconds: skip

New behavior:

if new_total_nanos <= prev_total_nanos: skip

This ensures that sub-second updates are correctly accepted.

✔ Reconstruct full timestamp in online_read

Feast now returns proper millisecond-aware timestamps by combining
seconds + nanos / 1e9.

Result

Feast users can now push multiple updates that occur within the same second and
Redis will correctly:

  • accept the newer event,
  • store its full timestamp,
  • and return precise timestamps via get_online_features.

This enables correct behavior for any use case emitting dense real-time data.


Which issue(s) this PR fixes

Fixes a long-standing limitation in the Redis online store where timestamp
precision was effectively second-level, causing incorrect deduplication and
event rejection when using the /push API.

(If maintainers want an issue link, I can open one in feast-dev.)


Misc

  • Changes are fully backward compatible: existing keys without nanos continue to
    work as before.
  • No API changes.
  • Behavior is now aligned with Feast’s use of protobuf Timestamp in other
    online store implementations.

@jatin5251 jatin5251 requested a review from a team as a code owner January 1, 2026 22:31
Add millisecond-precision timestamp support to Redis online store

Signed-off-by: Jatin Kumar <[email protected]>
sub-second precision when returning timestamps to client

Signed-off-by: Jatin Kumar <[email protected]>
fix(redis): preserve millisecond timestamp precision

Signed-off-by: Jatin Kumar <[email protected]>
@jatin5251 jatin5251 changed the title fix(redis): preserve millisecond timestamp precision in online_write_batch and online_read fix(redis): Preserve millisecond timestamp precision for Redis online store Jan 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants