Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@gibber9809
Copy link
Contributor

@gibber9809 gibber9809 commented Feb 21, 2025

Description

This PR adds support for specifying authoritative timestamps in the kv-ir ingestion flow while refactoring authoritative timestamp extraction in the JSON ingestion flow. Logic to match against authoritative timestamps has been moved into ArchiveWriter so that each ingestion pathway wrapping ArchiveWriter doesn't have to implement as much timestamp matching logic on its own. In particular add_node now executes some hooks to progressively match the authoritative timestamp column as the SchemaTree is constructed and a new matches_timestamp function has been added to check if a leaf node matches the authoritative timestamp column.

This implementation is slightly slower than the previous one. Results for all of the open source datasets with timestamp keys are as follows:

Dataset (ingestion time new / ingestion time old)
cockroach 1.044
mongodb 1.011
elasticsearch 1.041
postgresql 0.974

Checklist

  • The PR satisfies the contribution guidelines.
  • This is a breaking change and that has been indicated in the PR title, OR this isn't a
    breaking change.
  • Necessary docs have been updated, OR no docs need to be updated.

Validation performed

  • Manually validated that timestamp column matching functions as expected for a variety of nested columns and types for both the kv-ir and JSON ingestion flows

Summary by CodeRabbit

Summary by CodeRabbit

  • New Features
    • Archival operations now feature enhanced timestamp management for improved tracking and data integrity.
    • JSON data processing has been refined with more accurate timestamp matching and node adjustment for consistent results.
  • Bug Fixes
    • Removed restrictions on using both --file-type and --timestamp-key command-line arguments, simplifying user input.
  • Refactor
    • Command-line argument parsing has been streamlined, allowing a more flexible combination of options to better suit user requirements.

@gibber9809 gibber9809 requested a review from wraymo as a code owner February 21, 2025 16:31
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 21, 2025

Walkthrough

The changes introduce enhanced timestamp management across several components. In the ArchiveWriter, new member variables and logic in the open, close, and add_node methods have been added to manage authoritative timestamps. Adjustments in the JSON parser update the timestamp matching logic, including renaming methods to return both node IDs and timestamp match statuses, as well as updating internal mappings. Additionally, a conditional check in the command-line arguments parser has been removed to allow simultaneous use of certain arguments.

Changes

File(s) Change Summary
components/core/src/clp_s/ArchiveWriter.{cpp, hpp} - Added member variables: m_authoritative_timestamp, m_matched_timestamp_prefix_length, and m_matched_timestamp_prefix_node_id in ArchiveWriter and a corresponding vector in ArchiveWriterOption.
- Modified the open, close, and add_node methods to include timestamp management logic.
- Added new method matches_timestamp.
components/core/src/clp_s/JsonParser.{cpp, hpp} - Replaced several boolean flags with direct calls to matches_timestamp.
- Renamed get_archive_node_id to get_archive_node_id_and_check_timestamp, now returning a pair (node ID and timestamp flag).
- Updated add_node_to_archive_and_translations signature to include a matches_timestamp parameter.
- Added static method adjust_archive_node_type_for_timestamp and updated mapping types.
components/core/src/clp_s/CommandLineArguments.cpp Removed the conditional check that prevented the simultaneous use of --file-type and --timestamp-key, thus eliminating the corresponding error logging and failure return path.

Sequence Diagram(s)

sequenceDiagram
    participant Main
    participant ArchiveWriter
    participant JSONParser

    Main->>ArchiveWriter: open(options with authoritative_timestamp)
    ArchiveWriter-->>ArchiveWriter: Initialize m_authoritative_timestamp
    JSONParser->>ArchiveWriter: matches_timestamp(parent_node_id, key)
    ArchiveWriter-->>JSONParser: Return timestamp match flag
    JSONParser->>ArchiveWriter: add_node(..., matches_timestamp)
    ArchiveWriter-->>ArchiveWriter: Update internal timestamp matching variables
    Main->>ArchiveWriter: close()
    ArchiveWriter-->>ArchiveWriter: Reset timestamp variables
Loading

Possibly related PRs

Suggested reviewers

  • wraymo

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d415ba9 and 8348687.

📒 Files selected for processing (1)
  • components/core/src/clp_s/JsonParser.cpp (8 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.{cpp,hpp,java,js,jsx,tpp,ts,tsx}`: - Prefer `false == ...

**/*.{cpp,hpp,java,js,jsx,tpp,ts,tsx}: - Prefer false == <expression> rather than !<expression>.

  • components/core/src/clp_s/JsonParser.cpp
⏰ Context from checks skipped due to timeout of 90000ms (11)
  • GitHub Check: ubuntu-focal-static-linked-bins
  • GitHub Check: ubuntu-jammy-static-linked-bins
  • GitHub Check: centos-stream-9-static-linked-bins
  • GitHub Check: ubuntu-focal-dynamic-linked-bins
  • GitHub Check: centos-stream-9-dynamic-linked-bins
  • GitHub Check: ubuntu-jammy-dynamic-linked-bins
  • GitHub Check: build-macos (macos-14, false)
  • GitHub Check: build-macos (macos-14, true)
  • GitHub Check: lint-check (ubuntu-latest)
  • GitHub Check: build-macos (macos-13, false)
  • GitHub Check: build-macos (macos-13, true)
🔇 Additional comments (13)
components/core/src/clp_s/JsonParser.cpp (13)

106-106: Initialization of authoritative_timestamp aligned with PR objectives

The new field authoritative_timestamp is initialized with the m_timestamp_column value, correctly implementing the requirement to support authoritative timestamps in the ingestion flow.


360-361: Good refactoring of timestamp matching logic

The timestamp matching logic has been moved to the ArchiveWriter component as intended, simplifying the code in this method and ensuring consistent timestamp handling across ingestion pathways.


397-398: Consistent implementation of timestamp matching

Similar to the previous instance, this code correctly delegates timestamp matching to the ArchiveWriter component, maintaining consistency in the implementation.


456-456: Code style improvement

The use of false == object_it_stack.empty() rather than !object_it_stack.empty() aligns with the coding guidelines.


459-459: Risk of infinite loop

The empty body while (false == object_stack.empty()); may never exit if the stack is not cleared within the flow. Consider breaking early or removing this loop if it is not necessary.

- while (false == object_stack.empty());
+ // Ensure object_stack is purged or remove this loop if redundant.

634-647: Well-structured node type adjustment logic

The new helper method adjust_archive_node_type_for_timestamp cleanly encapsulates the logic for adjusting node types based on whether they match the timestamp. The function follows good practices with early returns and a clear switch statement.


649-668: Updated function signature to support timestamp matching

The add_node_to_archive_and_translations function has been updated to accept the matches_timestamp parameter and properly adjust the node type using the new helper function. This change supports the PR objective of refactoring timestamp extraction.


670-737: Improved function interface for timestamp information

The function was renamed from get_archive_node_id to get_archive_node_id_and_check_timestamp and now returns both the node ID and whether it matches a timestamp. This change clearly communicates the expanded functionality and is consistent with the overall refactoring approach.


743-744: Clear variable naming in destructuring assignment

The use of structured binding to capture both the node ID and timestamp match status improves code readability and clearly shows the intention of the code.


750-752: Conditional timestamp processing for integer values

The code now properly checks if the node matches the timestamp column before attempting to ingest a timestamp entry, ensuring consistent timestamp handling.


758-760: Conditional timestamp processing for float values

Similar to the integer case, the code correctly checks for timestamp matches before processing, maintaining consistency across data types.


769-780: Enhanced string timestamp handling with clear conditional flow

The code now properly branches based on whether the string value matches a timestamp, with dedicated handling for each case. This improves the clarity of the code and ensures correct timestamp processing.


796-807: Consistent timestamp handling for ClpString values

This change ensures that ClpString values are processed the same way as VarString values with respect to timestamp matching, maintaining consistency in the codebase.

✨ Finishing Touches
  • 📝 Generate Docstrings (Beta)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (9)
components/core/src/clp_s/ArchiveWriter.cpp (1)

118-120: Consider extracting cleanup logic into a separate method.

The cleanup of timestamp-related member variables is correct, but could be extracted into a dedicated method like reset_timestamp_state() to improve maintainability and reusability.

 void ArchiveWriter::close() {
     // ... existing code ...
-    m_authoritative_timestamp.clear();
-    m_matched_timestamp_prefix_length = 0ULL;
-    m_matched_timestamp_prefix_node_id = constants::cRootNodeId;
+    reset_timestamp_state();
 }
+
+void ArchiveWriter::reset_timestamp_state() {
+    m_authoritative_timestamp.clear();
+    m_matched_timestamp_prefix_length = 0ULL;
+    m_matched_timestamp_prefix_node_id = constants::cRootNodeId;
+}
components/core/src/clp_s/JsonParser.hpp (3)

124-139: Documentation needs update for new parameter.

The documentation is missing the @param matches_timestamp entry and the return value description could be more specific about returning a node ID.

Add the following to the documentation:

     * @param parent_node_id ID of the parent of the IR node
+    * @param matches_timestamp Whether the node matches the timestamp column
     * @return The ID of the node added to the archive's Schema Tree

142-153: Documentation could be clearer about return value components.

The return value description could be more specific about the components of the returned pair.

Suggest updating the return description to:

-    * @return The ID of the corresponding node in the archive's schema tree and a flag indicating
-    * whether the field should be treated as a timestamp.
+    * @return A pair containing:
+    *         - first: The ID of the corresponding node in the archive's schema tree
+    *         - second: A flag indicating whether the field should be treated as a timestamp

216-217: Consider adding member variable documentation.

The member variable m_ir_node_to_archive_node_id_mapping lacks documentation explaining its purpose and the meaning of its components.

Add documentation above the member variable:

    /**
     * Maps IR node ID and type to archive node ID and timestamp flag.
     * The key is a pair of (IR node ID, node type).
     * The value is a pair of (archive node ID, is timestamp flag).
     */
    absl::flat_hash_map<std::pair<uint32_t, NodeType>, std::pair<int32_t, bool>>
            m_ir_node_to_archive_node_id_mapping;
components/core/src/clp_s/ArchiveWriter.hpp (4)

28-28: Consider clarifying the usage of multiple timestamps.

Storing multiple strings in authoritative_timestamp is perfectly valid. However, if only a single authoritative timestamp is typically required, you may wish to document whether multiple entries are supported or expected.


100-119: Validate prefix matching logic and handle mismatches carefully.

The incremental matching approach is logical. However, consider ensuring that m_matched_timestamp_prefix_length is reset or handled explicitly if a mismatch occurs. Additionally, you could use m_authoritative_timestamp.empty() instead of m_authoritative_timestamp.size() > 0 for clarity.


121-136: Mark the method as const and simplify the difference check.

The implementation looks correct. You can declare matches_timestamp(...) as a const method since it does not modify member variables. Also, consider storing the difference (m_authoritative_timestamp.size() - m_matched_timestamp_prefix_length) in a local variable for readability.


273-275: Add explanatory comments for new fields.

These new member variables support authoritative timestamp tracking. Adding doxygen or inline comments describing their purpose and usage would improve maintainability.

components/core/src/clp_s/JsonParser.cpp (1)

634-647: Adjust numeric timestamps if applicable
The logic correctly converts string-based node types to DateString whenever matches_timestamp is true. If numeric timestamps are intended to be stored in a specialized type (e.g., Integer vs. DateInteger), consider extending this function’s logic.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8fce672 and 7d1765d.

📒 Files selected for processing (5)
  • components/core/src/clp_s/ArchiveWriter.cpp (2 hunks)
  • components/core/src/clp_s/ArchiveWriter.hpp (4 hunks)
  • components/core/src/clp_s/CommandLineArguments.cpp (0 hunks)
  • components/core/src/clp_s/JsonParser.cpp (8 hunks)
  • components/core/src/clp_s/JsonParser.hpp (2 hunks)
💤 Files with no reviewable changes (1)
  • components/core/src/clp_s/CommandLineArguments.cpp
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.{cpp,hpp,java,js,jsx,tpp,ts,tsx}`: - Prefer `false == ...

**/*.{cpp,hpp,java,js,jsx,tpp,ts,tsx}: - Prefer false == <expression> rather than !<expression>.

  • components/core/src/clp_s/ArchiveWriter.hpp
  • components/core/src/clp_s/JsonParser.cpp
  • components/core/src/clp_s/JsonParser.hpp
  • components/core/src/clp_s/ArchiveWriter.cpp
⏰ Context from checks skipped due to timeout of 90000ms (11)
  • GitHub Check: ubuntu-focal-static-linked-bins
  • GitHub Check: centos-stream-9-static-linked-bins
  • GitHub Check: ubuntu-focal-dynamic-linked-bins
  • GitHub Check: ubuntu-jammy-static-linked-bins
  • GitHub Check: centos-stream-9-dynamic-linked-bins
  • GitHub Check: ubuntu-jammy-dynamic-linked-bins
  • GitHub Check: build-macos (macos-14, false)
  • GitHub Check: build-macos (macos-14, true)
  • GitHub Check: build-macos (macos-13, false)
  • GitHub Check: lint-check (macos-latest)
  • GitHub Check: build-macos (macos-13, true)
🔇 Additional comments (10)
components/core/src/clp_s/ArchiveWriter.cpp (1)

21-21: LGTM! Initialization of authoritative timestamp.

The initialization of m_authoritative_timestamp from the option parameter aligns with the PR's objective to support authoritative timestamps in the ingestion flow.

components/core/src/clp_s/JsonParser.hpp (1)

114-122: LGTM! Well-documented static method addition.

The new static method is well-documented and appropriately designed for adjusting node types based on timestamp matching.

components/core/src/clp_s/ArchiveWriter.hpp (1)

11-11: No concerns with this added include.

The addition of archive_constants.hpp appears aligned with the new references (e.g., constants::cRootNodeId), and there are no conflicts with existing imports.

components/core/src/clp_s/JsonParser.cpp (7)

106-106: Ensure non-empty timestamp key before assignment
If m_timestamp_key is blank, authoritative_timestamp will be set to an empty descriptor. Confirm that this assignment is only done when a valid timestamp key has been parsed.


360-361: Looks good
The usage of matches_timestamp here is clear and consistent.


397-398: No issues detected
This timestamp check aligns with the approach used for numeric values.


456-456: Double-check do-while control flow
This loop’s condition suggests you are relying on hit_end to break out. Confirm that hit_end is correctly set to avoid logical missteps.


649-668: Implementation looks solid
The additional parameter and updated logic for timestamp-aware node creation are well integrated.


670-737: Good extension for timestamp checks
Returning both node ID and matches_timestamp together simplifies further logic.


742-832: Timestamp ingestion is properly woven into parse logic
Integrating matches_timestamp ensures that relevant fields are recognized and processed. The flow looks consistent and correct.

Copy link
Contributor

@wraymo wraymo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! I reviewed the sections unrelated to IR first

*/
int32_t add_node(int parent_node_id, NodeType type, std::string_view const key) {
return m_schema_tree.add_node(parent_node_id, type, key);
int32_t add_node(int parent_node_id, NodeType type, std::string_view key) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you putting the implementation of those two functions here (not source file) for better performance?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, just thought they were short enough. I'll move them to the .cpp file.

*/
int32_t add_node(int parent_node_id, NodeType type, std::string_view const key) {
return m_schema_tree.add_node(parent_node_id, type, key);
int32_t add_node(int parent_node_id, NodeType type, std::string_view key) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And can you add more description of this method?

if (NodeType::Object == type) {
m_matched_timestamp_prefix_node_id = node_id;
}
} else if ((m_authoritative_timestamp.size() - m_matched_timestamp_prefix_length) > 1) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably remove a pair of parentheses?

if (NodeType::Object == type) {
m_matched_timestamp_prefix_node_id = node_id;
}
} else if ((m_authoritative_timestamp.size() - m_matched_timestamp_prefix_length) > 1) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether the compiler will optimize repeated calls to m_authoritative_timestamp.size(). Since the value remains constant, would it be more efficient to store it in a variable and reuse it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any numbers of the ingestion performance before and after this timestamp matching rewriting?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll get some numbers quickly this morning and add them to the PR description.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like it is many times slower -- probably made an obvious mistake. Will profile/fix the performance bug and benchmark again after.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nevermind -- was accidentally benchmarking a debug build. Looks like it's generally very slightly slower (~0.7%-2%); will put up full results in a bit.

return m_schema_tree.add_node(parent_node_id, type, key);
int32_t add_node(int parent_node_id, NodeType type, std::string_view key) {
auto const node_id{m_schema_tree.add_node(parent_node_id, type, key)};
if (m_matched_timestamp_prefix_node_id == parent_node_id
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we only perform operations when NodeType::Object == type, can we put it here?

Copy link
Contributor Author

@gibber9809 gibber9809 Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, but only because I'm going to change this in a follow up pr to interact with the NodeType::Metadata subtree in order to handle timestamps that are part of the auto-generated keys in kv-ir.

Copy link
Contributor

@wraymo wraymo Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. After the next PR, we can check for opportunities to rewrite it since it seems redundant to me for now.

Comment on lines 102 to 117
if (m_matched_timestamp_prefix_node_id == parent_node_id
&& m_authoritative_timestamp.size() > 0)
{
if (constants::cRootNodeId == parent_node_id) {
if (NodeType::Object == type) {
m_matched_timestamp_prefix_node_id = node_id;
}
} else if ((m_authoritative_timestamp.size() - m_matched_timestamp_prefix_length) > 1) {
if (NodeType::Object == type
&& m_authoritative_timestamp.at(m_matched_timestamp_prefix_length) == key)
{
m_matched_timestamp_prefix_length += 1;
m_matched_timestamp_prefix_node_id = node_id;
}
}
}
Copy link
Contributor

@wraymo wraymo Feb 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we can write it in this way so that we can get rid of a lot of checks for irrelevant nodes.

Suggested change
if (m_matched_timestamp_prefix_node_id == parent_node_id
&& m_authoritative_timestamp.size() > 0)
{
if (constants::cRootNodeId == parent_node_id) {
if (NodeType::Object == type) {
m_matched_timestamp_prefix_node_id = node_id;
}
} else if ((m_authoritative_timestamp.size() - m_matched_timestamp_prefix_length) > 1) {
if (NodeType::Object == type
&& m_authoritative_timestamp.at(m_matched_timestamp_prefix_length) == key)
{
m_matched_timestamp_prefix_length += 1;
m_matched_timestamp_prefix_node_id = node_id;
}
}
}
if (NodeType::Object == type &&
m_matched_timestamp_prefix_node_id == parent_node_id
&& m_authoritative_timestamp.size() > m_matched_timestamp_prefix_length + 1)
{
if (constants::cRootNodeId == parent_node_id) {
m_matched_timestamp_prefix_node_id = node_id;
} else if (m_authoritative_timestamp.at(m_matched_timestamp_prefix_length) == key) {
m_matched_timestamp_prefix_length += 1;
m_matched_timestamp_prefix_node_id = node_id;
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess per discussion in the other comment are you fine leaving this as is for now and considering refactoring in a follow-up PR?

@gibber9809 gibber9809 requested a review from wraymo February 24, 2025 20:02
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
components/core/src/clp_s/ArchiveWriter.cpp (2)

240-259: Refactor the node addition logic for better readability and maintainability.

The timestamp matching logic in add_node could be simplified and made more maintainable:

  1. Use !m_authoritative_timestamp.empty() instead of size comparison
  2. Consider restructuring with early returns to reduce nesting
  3. Add comments explaining the timestamp matching algorithm
 int32_t ArchiveWriter::add_node(int parent_node_id, NodeType type, std::string_view key) {
     auto const node_id{m_schema_tree.add_node(parent_node_id, type, key)};
-    if (m_matched_timestamp_prefix_node_id == parent_node_id
-        && m_authoritative_timestamp.size() > 0)
+    // Skip timestamp matching if no authoritative timestamp is set
+    if (m_authoritative_timestamp.empty()) {
+        return node_id;
+    }
+
+    // Only process nodes that extend the current matched prefix
+    if (m_matched_timestamp_prefix_node_id != parent_node_id) {
+        return node_id;
+    }
+
+    // Match root object node
+    if (constants::cRootNodeId == parent_node_id && NodeType::Object == type) {
+        m_matched_timestamp_prefix_node_id = node_id;
+        return node_id;
+    }
+
+    // Match intermediate object nodes in the timestamp path
+    if (m_authoritative_timestamp.size() - m_matched_timestamp_prefix_length > 1
+        && NodeType::Object == type
+        && m_authoritative_timestamp.at(m_matched_timestamp_prefix_length) == key)
     {
-        if (constants::cRootNodeId == parent_node_id) {
-            if (NodeType::Object == type) {
-                m_matched_timestamp_prefix_node_id = node_id;
-            }
-        } else if (m_authoritative_timestamp.size() - m_matched_timestamp_prefix_length > 1) {
-            if (NodeType::Object == type
-                && m_authoritative_timestamp.at(m_matched_timestamp_prefix_length) == key)
-            {
-                m_matched_timestamp_prefix_length += 1;
-                m_matched_timestamp_prefix_node_id = node_id;
-            }
-        }
+        m_matched_timestamp_prefix_length += 1;
+        m_matched_timestamp_prefix_node_id = node_id;
     }
     return node_id;

261-270: Add validation and documentation for timestamp matching logic.

The matches_timestamp method could benefit from:

  1. Input validation for parent_node_id
  2. Documentation explaining the timestamp matching algorithm
  3. Early return for empty authoritative timestamp
+/**
+ * Checks if the given key at the specified parent node matches the last component
+ * of the authoritative timestamp path.
+ *
+ * @param parent_node_id The ID of the parent node to check
+ * @param key The key to match against the last component
+ * @return true if the key matches the last component of the authoritative timestamp
+ */
 bool ArchiveWriter::matches_timestamp(int parent_node_id, std::string_view key) {
+    if (m_authoritative_timestamp.empty()) {
+        return false;
+    }
+
     if (m_matched_timestamp_prefix_node_id == parent_node_id) {
         if (1 == (m_authoritative_timestamp.size() - m_matched_timestamp_prefix_length)
             && m_authoritative_timestamp.back() == key)
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7d1765d and d415ba9.

📒 Files selected for processing (2)
  • components/core/src/clp_s/ArchiveWriter.cpp (3 hunks)
  • components/core/src/clp_s/ArchiveWriter.hpp (4 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • components/core/src/clp_s/ArchiveWriter.hpp
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.{cpp,hpp,java,js,jsx,tpp,ts,tsx}`: - Prefer `false == ...

**/*.{cpp,hpp,java,js,jsx,tpp,ts,tsx}: - Prefer false == <expression> rather than !<expression>.

  • components/core/src/clp_s/ArchiveWriter.cpp
⏰ Context from checks skipped due to timeout of 90000ms (12)
  • GitHub Check: ubuntu-jammy-static-linked-bins
  • GitHub Check: ubuntu-focal-static-linked-bins
  • GitHub Check: ubuntu-jammy-dynamic-linked-bins
  • GitHub Check: centos-stream-9-static-linked-bins
  • GitHub Check: ubuntu-focal-dynamic-linked-bins
  • GitHub Check: centos-stream-9-dynamic-linked-bins
  • GitHub Check: build-macos (macos-14, false)
  • GitHub Check: build-macos (macos-14, true)
  • GitHub Check: lint-check (ubuntu-latest)
  • GitHub Check: build-macos (macos-13, false)
  • GitHub Check: build (macos-latest)
  • GitHub Check: build-macos (macos-13, true)
🔇 Additional comments (2)
components/core/src/clp_s/ArchiveWriter.cpp (2)

21-21: LGTM! Clean initialization and cleanup of timestamp-related members.

The initialization of m_authoritative_timestamp and cleanup of all timestamp-related member variables follow a consistent pattern with other class members.

Also applies to: 118-120


35-35: LGTM! Consistent coding style.

The code consistently follows the guideline to prefer false == <expression> rather than !<expression>.

Also applies to: 136-136, 201-201

Copy link
Contributor

@wraymo wraymo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other part looks good to me

@gibber9809 gibber9809 requested a review from wraymo February 25, 2025 15:11
@wraymo
Copy link
Contributor

wraymo commented Feb 25, 2025

For the PR title, do you want to mention refactoring timestamp matching?

@gibber9809 gibber9809 changed the title feat(clp-s): Add support for specifying authoritative timestamp in kv-ir ingestion flow feat(clp-s): Add support for specifying authoritative timestamp in kv-ir ingestion flow; Refactor timestamp matching for JSON ingestion flow. Feb 25, 2025
@gibber9809
Copy link
Contributor Author

For the PR title, do you want to mention refactoring timestamp matching?

Sure. Is the new title okay?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants