Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

Lokeshchand33
Copy link
Contributor

@Lokeshchand33 Lokeshchand33 commented Sep 4, 2025

Summary
This PR adds support for Databricks Vector Search Index APIs by implementing 9 new actions.

Changes

  • Added create index action
  • Added get index action
  • Added list indexes action
  • Added delete index action
  • Added query index action
  • Added sync index action
  • Added scan index action
  • Added upsert index data action
  • Added delete index data action

Implements 9 Vector Search Index endpoints as discussed in the enhancement issue.

Resolves #18126"

Summary by CodeRabbit

  • New Features

    • Full Vector Search support: create, list, get, delete, sync, query, scan indexes; upsert/delete index rows and delete index data; per-index selector and related API actions.
    • Databricks SQL Warehouse management surfaced (create, edit, retrieve, list, start, stop, config get/set, permissions).
  • Chores

    • Numerous action metadata version bumps across Databricks actions; Databricks component version bumped to 0.5.0.

Lokeshchand33 and others added 22 commits August 22, 2025 17:59
- Added create SQL warehouse action
- Added list SQL warehouses action
- Added get SQL warehouse action
- Added edit SQL warehouse action
- Added delete SQL warehouse action
- Added start/stop SQL warehouse actions
- Added get/set SQL warehouse config actions
- Added get/set SQL warehouse permissions actions

Implements 11 SQL Warehouses endpoints as discussed in the enhancement issue.
Summary
This PR adds support for Databricks Vector Search Index APIs by implementing 9 new actions.

Changes
- Added create index action
- Added get index action
- Added list indexes action
- Added delete index action
- Added query index action
- Added sync index action
- Added scan index action
- Added upsert index data action
- Added delete index data action

Resolves PipedreamHQ#18126
Copy link

vercel bot commented Sep 4, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Preview Comments Updated (UTC)
pipedream-docs Ignored Ignored Sep 23, 2025 5:47pm
pipedream-docs-redirect-do-not-edit Ignored Ignored Sep 23, 2025 5:47pm

Copy link
Contributor

coderabbitai bot commented Sep 4, 2025

Walkthrough

Adds Databricks Vector Search support (endpoints and index APIs: create/get/list/delete/query/sync/upsert/scan/delete-data), new actions for vector index management and data operations, app propDefinition for indexName, multiple SQL Warehouse action version bumps, numerous action version updates, and package version bumped to 0.5.0.

Changes

Cohort / File(s) Summary
Databricks app: APIs & dynamic props
components/databricks/databricks.app.mjs
Adds indexName propDefinition and ensures endpoints = [] default; adds vector-search methods (createVectorSearchIndex, getVectorSearchIndex, listVectorSearchIndexes({ params, ... }), deleteVectorSearchIndex, queryVectorSearchIndex, syncVectorSearchIndex, deleteVectorSearchData, upsertVectorSearchIndexData, scanVectorSearchIndex).
Vector Search index management actions
components/databricks/actions/create-vector-search-index/create-vector-search-index.mjs, components/databricks/actions/get-vector-search-index/get-vector-search-index.mjs, components/databricks/actions/list-vector-search-indexes/list-vector-search-indexes.mjs, components/databricks/actions/delete-vector-search-index/delete-vector-search-index.mjs, components/databricks/actions/sync-vector-search-index/sync-vector-search-index.mjs
New actions to create/get/list/delete/sync vector search indexes; create implements DELTA_SYNC vs DIRECT_ACCESS branching with validations and payload composition.
Vector Search data operations
components/databricks/actions/upsert-vector-search-index-data/upsert-vector-search-index-data.mjs, components/databricks/actions/delete-vector-search-index-data/delete-vector-search-index-data.mjs, components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs, components/databricks/actions/scan-vector-search-index/scan-vector-search-index.mjs
New actions for upsert/delete/query/scan index data: JSON parsing/validation (rows, primary keys, query vectors), payload assembly, API calls, and runtime summaries.
SQL Warehouse management actions (metadata bumps)
components/databricks/actions/create-sql-warehouse/create-sql-warehouse.mjs, components/databricks/actions/edit-sql-warehouse/edit-sql-warehouse.mjs, components/databricks/actions/delete-sql-warehouse/delete-sql-warehouse.mjs, components/databricks/actions/start-sql-warehouse/start-sql-warehouse.mjs, components/databricks/actions/stop-sql-warehouse/stop-sql-warehouse.mjs, components/databricks/actions/list-sql-warehouses/list-sql-warehouses.mjs, components/databricks/actions/get-sql-warehouse/get-sql-warehouse.mjs, components/databricks/actions/get-sql-warehouse-config/get-sql-warehouse-config.mjs, components/databricks/actions/set-sql-warehouse-config/set-sql-warehouse-config.mjs, components/databricks/actions/get-sql-warehouse-permissions/get-sql-warehouse-permissions.mjs, components/databricks/actions/set-sql-warehouse-permissions/set-sql-warehouse-permissions.mjs
SQL Warehouse action modules updated (mostly version bumps); behavior unchanged except metadata version updates.
Jobs / Runs / Endpoints actions (metadata bumps)
components/databricks/actions/* (many files: create-endpoint, delete-endpoint, get-endpoint, list-endpoints, create-job, delete-job, get-job, list-jobs, get-run, get-run-output, list-runs, run-job-now, cancel-run, cancel-all-runs, export-run, repair-run, reset-job, set-job-permissions, update-job, delete-run, etc.)
Many existing actions had version bumps (e.g., 0.0.1→0.0.2, 0.0.2→0.0.3, 0.0.3→0.0.4/0.0.5) with no functional changes.
Package metadata
components/databricks/package.json
Bumps package version from 0.4.0 to 0.5.0.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor User
  participant Action as Databricks Action
  participant App as Databricks App
  participant API as Databricks REST API

  User->>Action: invoke vector/index action (create/get/list/query/sync/upsert/scan/delete/delete-data)
  Action->>Action: parse & validate inputs (JSON, vectors, keys, options)
  Note right of Action #DFF2E1: compose request payload\n(e.g., DELTA_SYNC vs DIRECT_ACCESS)
  Action->>App: call corresponding app method (createVectorSearchIndex / queryVectorSearchIndex / ...)
  App->>API: HTTP request to Databricks vector-search endpoint
  API-->>App: response
  App-->>Action: returns response
  Action-->>User: return result and export "$summary"
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested reviewers

  • jcortes
  • lcaresia

Poem

I hop through indexes, payloads neat,
Parsing vectors with tiny feet,
Warehouses wake and indexes sing,
Rows upserted with a joyful spring,
A rabbit ships this Databricks treat. 🥕🐇

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Title Check ⚠️ Warning The PR title "Databricks sql warehouses" is misleading and does not summarize the primary changes in this changeset, which primarily add Databricks Vector Search Index support (nine new actions and matching app methods) alongside several metadata version bumps; the title is also not a clear single-sentence summary teammates can scan to understand the main change. Rename the PR to a concise, accurate title that reflects the main work, for example: "Databricks: add Vector Search Index actions and app methods" or "Add Databricks Vector Search Index support (create/get/list/delete/query/sync/scan/upsert/delete-data)" so reviewers can immediately see the primary change.
Description Check ⚠️ Warning The repository's required PR template expects a "## WHY" section but the submitted description does not follow the template and omits that required rationale and structured template fields; it also lacks explicit test evidence references or a clear mapping of which files implement which endpoints beyond a short list. Update the PR description to use the repository template by adding a "## WHY" section explaining the rationale, expand the Changes section to map implemented endpoints to files, and include testing notes or links to the passing test report (also remove the stray trailing quote after "Resolves #18126").
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues Check ✅ Passed The changes implement the nine Vector Search Index endpoints (create/get/list/delete/query/sync/scan/upsert/delete-data) and add corresponding methods and an indexName propDefinition in components/databricks/databricks.app.mjs (including listVectorSearchIndexes updated to accept params.endpoint_name); action modules wire endpointName into indexName propDefinitions and call the new app methods, and SQL Warehouse updates are metadata version bumps only. Based on the provided summaries and the PR discussion noting test fixes and a final passing test report, the coding objectives in linked issue #18126 for the Vector Search Index subset appear satisfied.
Out of Scope Changes Check ✅ Passed All modified files are within the Databricks component and relate to the linked issue scope: newly added Vector Search Index actions and app methods, plus routine version bumps for existing Databricks actions and a package version bump; there are no changes to unrelated components or repositories.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
✨ Finishing touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@adolfo-pd adolfo-pd added the User submitted Submitted by a user label Sep 4, 2025
@pipedream-component-development
Copy link
Collaborator

Thank you so much for submitting this! We've added it to our backlog to review, and our team has been notified.

@pipedream-component-development
Copy link
Collaborator

Thanks for submitting this PR! When we review PRs, we follow the Pipedream component guidelines. If you're not familiar, here's a quick checklist:

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 13

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
components/databricks/actions/list-runs/list-runs.mjs (2)

53-53: Bug: array compared to number in loop condition.

allRuns < this.maxResults always evaluates unexpectedly; should be allRuns.length.

-    } while (total === params.limit && allRuns < this.maxResults);
+    } while (total === params.limit && allRuns.length < this.maxResults);

1-66: Fix array-to-number comparison in loop condition Change allRuns < this.maxResults to allRuns.length < this.maxResults in components/databricks/actions/list-runs/list-runs.mjs around line 53.

🧹 Nitpick comments (32)
components/databricks/actions/run-job-now/run-job-now.mjs (1)

49-57: Guard against mutually exclusive params (jarParams vs notebookParams).

The API rejects requests specifying both. Add a quick check to fail fast with a clear error.

   async run({ $ }) {
-    const response = await this.databricks.runJobNow({
+    const hasJar = Array.isArray(this.jarParams) && this.jarParams.length > 0;
+    const hasNotebook = this.notebookParams && Object.keys(this.notebookParams).length > 0;
+    if (hasJar && hasNotebook) {
+      throw new Error("jarParams and notebookParams are mutually exclusive. Provide only one.");
+    }
+
+    const response = await this.databricks.runJobNow({
       data: {
         job_id: this.jobId,
         jar_params: this.jarParams,
         notebook_params: this.notebookParams,
         python_params: this.pythonParams,
         spark_submit_params: this.sparkSubmitParams,
         idempotency_token: this.idempotencyToken,
       },
       $,
     });
components/databricks/actions/list-runs/list-runs.mjs (2)

33-38: Respect maxResults during pagination to avoid over-fetching.

Adjust limit to the remaining budget and stop early when the cap is reached.

   async run({ $ }) {
-    const params = {
+    const params = {
       job_id: this.jobId,
       active_only: this.activeOnly,
-      limit: 100,
+      limit: 100,
       offset: 0,
     };
+    const max = this.maxResults ?? 100;

42-47: Cap each page by remaining results.

Prevents extra API calls and trimming later.

-    do {
-      const { runs } = await this.databricks.listRuns({
+    do {
+      // Stop if we've already reached the cap
+      if (allRuns.length >= max) break;
+      // Limit this page to the remaining budget
+      params.limit = Math.min(params.limit, Math.max(1, max - allRuns.length));
+      const { runs } = await this.databricks.listRuns({
         params,
         $,
       });
components/databricks/package.json (1)

3-3: Version bump — LGTM. Please ensure release notes.

Given the sizable feature set, add/update CHANGELOG and fill in the PR description before merge.

components/databricks/common/constants.mjs (1)

1-11: Prevent accidental mutation of shared constants.

Freeze the array to avoid runtime edits.

-export const CLUSTER_SIZES = [
+export const CLUSTER_SIZES = Object.freeze([
   "2X-Small",
   "X-Small",
   "Small",
   "Medium",
   "Large",
   "X-Large",
   "2X-Large",
   "3X-Large",
   "4X-Large",
-];
+]);
components/databricks/common/utils.mjs (3)

16-24: Limit recursion to plain objects to avoid mangling Dates/Maps/Sets/etc.

Current typeof obj === "object" will traverse non-POJOs and strip their contents. Guard for plain objects.

Apply:

-  if (typeof obj === "object") {
+  if (typeof obj === "object" && Object.getPrototypeOf(obj) === Object.prototype) {
     return Object.fromEntries(Object.entries(obj).map(([
       key,
       value,
     ]) => [
       key,
       parseObject(value),
     ]));
   }

1-27: Protect against cyclic structures to prevent infinite recursion.

Add a WeakSet to track seen objects.

One way:

-const parseObject = (obj) => {
+const parseObject = (obj, _seen = new WeakSet()) => {
@@
-  if (Array.isArray(obj)) {
-    return obj.map((item) => parseObject(item));
+  if (Array.isArray(obj)) {
+    return obj.map((item) => parseObject(item, _seen));
   }
@@
-  if (typeof obj === "object" && Object.getPrototypeOf(obj) === Object.prototype) {
+  if (typeof obj === "object" && Object.getPrototypeOf(obj) === Object.prototype) {
+    if (_seen.has(obj)) return obj;
+    _seen.add(obj);
     return Object.fromEntries(Object.entries(obj).map(([
       key,
       value,
-    ]) => [
-      key,
-      parseObject(value),
+    ]) => [
+      key,
+      parseObject(value, _seen),
     ]));
   }

29-31: Export named helper for ergonomics and tree-shaking.

Apply:

-export default {
-  parseObject,
-};
+export { parseObject };
+export default { parseObject };
components/databricks/actions/list-vector-search-indexes/list-vector-search-indexes.mjs (1)

16-17: Prefer nullish coalescing for count and keep concise pluralization.

Apply:

-    const count = response?.vector_indexes?.length || 0;
-    $.export("$summary", `Found ${count} vector search index${count === 1 ? "" : "es"}`);
+    const count = response?.vector_indexes?.length ?? 0;
+    $.export("$summary", `Found ${count} vector search index${count === 1 ? "" : "es"}`);
components/databricks/actions/delete-vector-search-index/delete-vector-search-index.mjs (1)

6-6: Clarify that deletion is irreversible in the description.

Apply:

-  description: "Deletes a vector search index in Databricks. [See the documentation](https://docs.databricks.com/api/workspace/vectorsearchindexes/deleteindex)",
+  description: "Irreversibly deletes a vector search index in Databricks. [See the documentation](https://docs.databricks.com/api/workspace/vectorsearchindexes/deleteindex)",
components/databricks/actions/start-sql-warehouse/start-sql-warehouse.mjs (1)

19-27: Optionally support waiting until the warehouse is ready.

Starting is async; add a waitForReady flag to poll status via getSQLWarehouse until RUNNING (with timeout).

Apply:

   props: {
     databricks,
     warehouseId: {
       description: "The ID of the SQL Warehouse to start",
       propDefinition: [
         databricks,
         "warehouseId",
       ],
     },
+    waitForReady: {
+      type: "boolean",
+      label: "Wait Until Ready",
+      description: "Polls the warehouse state until it's running (or times out).",
+      optional: true,
+      default: false,
+    },
+    pollIntervalMs: {
+      type: "integer",
+      label: "Poll Interval (ms)",
+      optional: true,
+      default: 5000,
+    },
+    timeoutMs: {
+      type: "integer",
+      label: "Timeout (ms)",
+      optional: true,
+      default: 300000,
+    },
   },
   async run({ $ }) {
     const response = await this.databricks.startSQLWarehouse({
       warehouseId: this.warehouseId,
       $,
     });
 
     $.export("$summary", `Successfully started SQL Warehouse with ID ${this.warehouseId}`);
-    return response;
+    if (!this.waitForReady) return response;
+
+    const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+    const start = Date.now();
+    while (Date.now() - start < this.timeoutMs) {
+      const { state } = await this.databricks.getSQLWarehouse({ warehouseId: this.warehouseId, $ });
+      if (state === "RUNNING") return { ...response, state };
+      await sleep(this.pollIntervalMs);
+    }
+    throw new Error(`Timed out waiting for SQL Warehouse ${this.warehouseId} to be RUNNING`);
   },
components/databricks/actions/delete-sql-warehouse/delete-sql-warehouse.mjs (2)

9-18: Add an optional “idempotent delete” switch

Allow users to treat 404 as success to make deletes safe to re-run.

Apply this diff to introduce an ignoreMissing prop:

   props: {
     databricks,
     warehouseId: {
       description: "The ID of the SQL Warehouse to delete",
       propDefinition: [
         databricks,
         "warehouseId",
       ],
     },
+    ignoreMissing: {
+      type: "boolean",
+      label: "Ignore Missing",
+      description: "Treat 404 (not found) as success.",
+      optional: true,
+    },
   },

19-29: Make deletion idempotent and improve error messaging

Catch 404s when ignoreMissing is enabled; keep other errors bubbling.

Apply this diff:

-  async run({ $ }) {
-    await this.databricks.deleteSQLWarehouse({
-      warehouseId: this.warehouseId,
-      $,
-    });
-
-    $.export("$summary", `Successfully deleted SQL Warehouse with ID ${this.warehouseId}`);
-    return {
-      success: true,
-    };
-  },
+  async run({ $ }) {
+    try {
+      await this.databricks.deleteSQLWarehouse({
+        warehouseId: this.warehouseId,
+        $,
+      });
+      $.export("$summary", `Successfully deleted SQL Warehouse with ID ${this.warehouseId}`);
+      return { success: true };
+    } catch (err) {
+      const status = err?.response?.status ?? err?.status;
+      if (this.ignoreMissing && status === 404) {
+        $.export("$summary", `Warehouse ${this.warehouseId} not found. Treated as success (ignored).`);
+        return { success: true, ignored: true };
+      }
+      throw err;
+    }
+  },
components/databricks/actions/set-sql-warehouse-config/set-sql-warehouse-config.mjs (1)

131-152: Validate configuration pair items before sending

Guard against malformed entries to fail fast with actionable errors.

Apply this minimal validation:

-    const configParam = utils.parseObject(this.configParam);
+    const configParam = utils.parseObject(this.configParam);
     if (Array.isArray(configParam) && configParam.length) {
+      configParam.forEach((p, i) => {
+        if (!p || typeof p !== "object" || typeof p.key !== "string") {
+          throw new ConfigurationError(`configParam[${i}] must be an object like { "key": string, "value": any }`);
+        }
+      });
       payload.config_param = {
         configuration_pairs: configParam,
       };
     }
-    const globalParam = utils.parseObject(this.globalParam);
+    const globalParam = utils.parseObject(this.globalParam);
     if (Array.isArray(globalParam) && globalParam.length) {
+      globalParam.forEach((p, i) => {
+        if (!p || typeof p !== "object" || typeof p.key !== "string") {
+          throw new ConfigurationError(`globalParam[${i}] must be an object like { "key": string, "value": any }`);
+        }
+      });
       payload.global_param = {
         configuration_pairs: globalParam,
       };
     }
-    const sqlConfigurationParameters = utils.parseObject(this.sqlConfigurationParameters);
+    const sqlConfigurationParameters = utils.parseObject(this.sqlConfigurationParameters);
     if (Array.isArray(sqlConfigurationParameters) && sqlConfigurationParameters.length) {
+      sqlConfigurationParameters.forEach((p, i) => {
+        if (!p || typeof p !== "object" || typeof p.key !== "string") {
+          throw new ConfigurationError(`sqlConfigurationParameters[${i}] must be an object like { "key": string, "value": any }`);
+        }
+      });
       payload.sql_configuration_parameters = {
         configuration_pairs: sqlConfigurationParameters,
       };
     }
-    const dataAccessConfig = utils.parseObject(this.dataAccessConfig);
+    const dataAccessConfig = utils.parseObject(this.dataAccessConfig);
     if (Array.isArray(dataAccessConfig) && dataAccessConfig.length) {
       payload.data_access_config = dataAccessConfig;
     }
components/databricks/actions/get-sql-warehouse-permissions/get-sql-warehouse-permissions.mjs (1)

19-27: Add user-friendly summaries for 404/403 (optional).

Export clearer summaries on not found / forbidden without hiding the error.

-  async run({ $ }) {
-    const response = await this.databricks.getSQLWarehousePermissions({
-      warehouseId: this.warehouseId,
-      $,
-    });
-
-    $.export("$summary", `Retrieved permissions for SQL Warehouse ID ${this.warehouseId}`);
-    return response;
-  },
+  async run({ $ }) {
+    try {
+      const response = await this.databricks.getSQLWarehousePermissions({
+        warehouseId: this.warehouseId,
+        $,
+      });
+      $.export("$summary", `Retrieved permissions for SQL Warehouse ID ${this.warehouseId}`);
+      return response;
+    } catch (err) {
+      if (err?.response?.status === 404) {
+        $.export("$summary", `SQL Warehouse ${this.warehouseId} not found.`);
+      } else if (err?.response?.status === 403) {
+        $.export("$summary", `Access denied to SQL Warehouse ${this.warehouseId}.`);
+      }
+      throw err;
+    }
+  },
components/databricks/actions/get-sql-warehouse/get-sql-warehouse.mjs (1)

25-26: Use resource name in summary when available (nit).

Improves UX in runs list.

-    $.export("$summary", `Retrieved details for SQL Warehouse ID ${this.warehouseId}`);
+    $.export("$summary", `Retrieved SQL Warehouse ${response?.name ?? this.warehouseId}`);
components/databricks/actions/stop-sql-warehouse/stop-sql-warehouse.mjs (2)

9-18: Optional: add wait-for-stop toggle.

Stopping is often async; offer a wait flag for convenience.

   props: {
     databricks,
     warehouseId: {
       description: "The ID of the SQL Warehouse to stop",
       propDefinition: [
         databricks,
         "warehouseId",
       ],
     },
+    waitForStop: {
+      type: "boolean",
+      label: "Wait Until Stopped",
+      description: "Polls the warehouse status until STOPPED (or timeout).",
+      optional: true,
+      default: false,
+    },
+    pollIntervalMs: {
+      type: "integer",
+      label: "Poll Interval (ms)",
+      optional: true,
+      default: 5000,
+    },
   },

19-27: Optional: implement simple polling when wait is enabled.

Poll status until STOPPED with a bounded loop.

   async run({ $ }) {
     const response = await this.databricks.stopSQLWarehouse({
       warehouseId: this.warehouseId,
       $,
     });
 
-    $.export("$summary", `Successfully stopped SQL Warehouse with ID ${this.warehouseId}`);
-    return response;
+    if (!this.waitForStop) {
+      $.export("$summary", `Stop requested for SQL Warehouse ${this.warehouseId}`);
+      return response;
+    }
+    // Poll for terminal state
+    const maxTries = 60; // ~5 min with 5s default interval
+    for (let i = 0; i < maxTries; i++) {
+      const { state } = await this.databricks.getSQLWarehouse({ warehouseId: this.warehouseId, $ });
+      if (state === "STOPPED") {
+        $.export("$summary", `SQL Warehouse ${this.warehouseId} is STOPPED.`);
+        return response;
+      }
+      await new Promise((r) => setTimeout(r, this.pollIntervalMs ?? 5000));
+    }
+    $.export("$summary", `Timed out waiting for SQL Warehouse ${this.warehouseId} to stop.`);
+    return response;
   },
components/databricks/actions/scan-vector-search-index/scan-vector-search-index.mjs (2)

39-41: Validate numResults > 0 (defensive).

Prevent accidental zero/negative values.

-    if (this.numResults !== undefined) {
-      body.num_results = this.numResults;
-    }
+    if (this.numResults !== undefined) {
+      if (!Number.isInteger(this.numResults) || this.numResults <= 0) {
+        throw new Error("numResults must be a positive integer.");
+      }
+      body.num_results = this.numResults;
+    }

35-47: Minor: rename body -> payload for consistency.

Matches other actions’ naming.

-    const body = {};
+    const payload = {};
-    if (typeof this.lastPrimaryKey === "string" && this.lastPrimaryKey.trim().length > 0) {
-      body.last_primary_key = this.lastPrimaryKey.trim();
+    if (typeof this.lastPrimaryKey === "string" && this.lastPrimaryKey.trim().length > 0) {
+      payload.last_primary_key = this.lastPrimaryKey.trim();
     }
-    if (this.numResults !== undefined) {
-      if (!Number.isInteger(this.numResults) || this.numResults <= 0) {
-        throw new Error("numResults must be a positive integer.");
-      }
-      body.num_results = this.numResults;
+    if (this.numResults !== undefined) {
+      if (!Number.isInteger(this.numResults) || this.numResults <= 0) {
+        throw new Error("numResults must be a positive integer.");
+      }
+      payload.num_results = this.numResults;
     }
@@
-      data: body,
+      data: payload,
components/databricks/actions/sync-vector-search-index/sync-vector-search-index.mjs (1)

18-30: LGTM with a minor enhancement suggestion

Looks good. Consider an optional boolean prop like waitForCompletion to poll until the sync job reaches a terminal state and surface failures earlier, defaulting to fire-and-forget to preserve current behavior.

components/databricks/actions/upsert-vector-search-index-data/upsert-vector-search-index-data.mjs (1)

42-53: Potential large-payload risk: consider chunking large upserts

inputs_json can get big; APIs often have size limits. Consider batching (e.g., chunks of N rows or bytes) and invoking the endpoint per chunk, aggregating results.

components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs (2)

64-70: Stricter validation for queryVector content (optional)

After JSON.parse, ensure it’s a non-empty array of numbers to catch silent shape issues early.

   if (this.queryVector) {
     try {
       payload.query_vector = JSON.parse(this.queryVector);
+      if (!Array.isArray(payload.query_vector) || !payload.query_vector.length || payload.query_vector.some((v) => typeof v !== "number")) {
+        throw new Error("`queryVector` must be a non-empty JSON array of numbers.");
+      }
     } catch (err) {
       throw new Error(`Invalid queryVector JSON: ${err.message}`);
     }
   }

86-88: Result count path may vary across API versions

If responses use results instead of result.data_array, consider a fallback.

-  const count = response?.result?.data_array?.length || 0;
+  const count = response?.result?.data_array?.length
+    ?? response?.results?.length
+    ?? 0;
components/databricks/actions/set-sql-warehouse-permissions/set-sql-warehouse-permissions.mjs (1)

33-41: Optional: deduplicate ACL entries to avoid redundant updates

You can collapse duplicates by principal + permission_level before sending.

components/databricks/actions/create-sql-warehouse/create-sql-warehouse.mjs (4)

117-133: Validation is solid; consider aligning defaults and cross-field constraints

  • You already enforce maxNumClusters ≥ minNumClusters. Consider also failing fast when maxNumClusters is omitted but minNumClusters > 1 and the backend requires both for autoscaling.
  • Optionally cap num_clusters at backend-specific limits if they differ by edition.

149-158: Enforce PRO + serverless pairing (optional but user-friendly)

When enableServerlessCompute === true, validate warehouseType === "PRO" to avoid backend errors.

   if (this.enableServerlessCompute !== undefined)
     payload.enable_serverless_compute = this.enableServerlessCompute;
+  if (this.enableServerlessCompute && this.warehouseType && this.warehouseType !== "PRO") {
+    throw new ConfigurationError("Serverless compute requires warehouseType = PRO.");
+  }

135-147: Tags parsing: guard undefined to avoid surprises

If this.tags is undefined and utils.parseObject expects a string, default to {}.

-  const parsedTags = utils.parseObject(this.tags);
+  const parsedTags = utils.parseObject(this.tags || {});

156-158: Channel is already an object; parsing is optional

If the prop type enforces an object, utils.parseObject is unnecessary. Harmless but can be dropped.

components/databricks/actions/edit-sql-warehouse/edit-sql-warehouse.mjs (2)

140-146: Serverless + CLASSIC guard is good; consider auto-upgrading type (optional)

Optional: if enableServerlessCompute = true and warehouseType not provided, consider defaulting warehouse_type to PRO to reduce user error, or add a message prompting users to set it.


148-160: Harden tags parsing and enforce tag count limit (< 45)

parseObject on undefined relies on helper behavior; also enforce documented tag-count constraint and type-safety.

Apply this diff:

-    const parsedTags = utils.parseObject(this.tags);
-    const tagArray = Object.entries(parsedTags).map(([
-      key,
-      value,
-    ]) => ({
-      key,
-      value,
-    }));
-    if (tagArray.length) {
-      payload.tags = {
-        custom_tags: tagArray,
-      };
-    }
+    const parsedTags = this.tags ? utils.parseObject(this.tags) : {};
+    if (parsedTags && (typeof parsedTags !== "object" || Array.isArray(parsedTags))) {
+      throw new ConfigurationError("tags must be an object of key-value pairs.");
+    }
+    const tagEntries = Object.entries(parsedTags);
+    if (tagEntries.length > 45) {
+      throw new ConfigurationError("tags must contain fewer than 45 entries.");
+    }
+    const tagArray = tagEntries.map(([key, value]) => ({ key, value }));
+    if (tagArray.length) {
+      payload.tags = { custom_tags: tagArray };
+    }
components/databricks/databricks.app.mjs (1)

47-61: Nit: clarify prop description

This looks copy-pasted from runs; it’s about Warehouses.

Apply this diff:

-      description: "The ID of the SQL Warehouse to get runs from",
+      description: "The ID of the SQL Warehouse",
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 6aeaea6 and 62287c7.

📒 Files selected for processing (27)
  • components/databricks/actions/create-sql-warehouse/create-sql-warehouse.mjs (1 hunks)
  • components/databricks/actions/create-vector-search-index/create-vector-search-index.mjs (1 hunks)
  • components/databricks/actions/delete-sql-warehouse/delete-sql-warehouse.mjs (1 hunks)
  • components/databricks/actions/delete-vector-search-index-data/delete-vector-search-index-data.mjs (1 hunks)
  • components/databricks/actions/delete-vector-search-index/delete-vector-search-index.mjs (1 hunks)
  • components/databricks/actions/edit-sql-warehouse/edit-sql-warehouse.mjs (1 hunks)
  • components/databricks/actions/get-run-output/get-run-output.mjs (1 hunks)
  • components/databricks/actions/get-sql-warehouse-config/get-sql-warehouse-config.mjs (1 hunks)
  • components/databricks/actions/get-sql-warehouse-permissions/get-sql-warehouse-permissions.mjs (1 hunks)
  • components/databricks/actions/get-sql-warehouse/get-sql-warehouse.mjs (1 hunks)
  • components/databricks/actions/get-vector-search-index/get-vector-search-index.mjs (1 hunks)
  • components/databricks/actions/list-runs/list-runs.mjs (1 hunks)
  • components/databricks/actions/list-sql-warehouses/list-sql-warehouses.mjs (1 hunks)
  • components/databricks/actions/list-vector-search-indexes/list-vector-search-indexes.mjs (1 hunks)
  • components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs (1 hunks)
  • components/databricks/actions/run-job-now/run-job-now.mjs (1 hunks)
  • components/databricks/actions/scan-vector-search-index/scan-vector-search-index.mjs (1 hunks)
  • components/databricks/actions/set-sql-warehouse-config/set-sql-warehouse-config.mjs (1 hunks)
  • components/databricks/actions/set-sql-warehouse-permissions/set-sql-warehouse-permissions.mjs (1 hunks)
  • components/databricks/actions/start-sql-warehouse/start-sql-warehouse.mjs (1 hunks)
  • components/databricks/actions/stop-sql-warehouse/stop-sql-warehouse.mjs (1 hunks)
  • components/databricks/actions/sync-vector-search-index/sync-vector-search-index.mjs (1 hunks)
  • components/databricks/actions/upsert-vector-search-index-data/upsert-vector-search-index-data.mjs (1 hunks)
  • components/databricks/common/constants.mjs (1 hunks)
  • components/databricks/common/utils.mjs (1 hunks)
  • components/databricks/databricks.app.mjs (2 hunks)
  • components/databricks/package.json (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (18)
components/databricks/actions/delete-vector-search-index-data/delete-vector-search-index-data.mjs (3)
components/databricks/actions/delete-vector-search-index/delete-vector-search-index.mjs (1)
  • response (19-22)
components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs (1)
  • response (80-84)
components/databricks/actions/scan-vector-search-index/scan-vector-search-index.mjs (1)
  • response (43-47)
components/databricks/actions/get-vector-search-index/get-vector-search-index.mjs (6)
components/databricks/actions/create-vector-search-index/create-vector-search-index.mjs (1)
  • response (109-112)
components/databricks/actions/delete-vector-search-index-data/delete-vector-search-index-data.mjs (1)
  • response (34-38)
components/databricks/actions/delete-vector-search-index/delete-vector-search-index.mjs (1)
  • response (19-22)
components/databricks/actions/list-vector-search-indexes/list-vector-search-indexes.mjs (1)
  • response (14-14)
components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs (1)
  • response (80-84)
components/databricks/actions/scan-vector-search-index/scan-vector-search-index.mjs (1)
  • response (43-47)
components/databricks/actions/get-sql-warehouse/get-sql-warehouse.mjs (4)
components/databricks/actions/edit-sql-warehouse/edit-sql-warehouse.mjs (1)
  • response (174-178)
components/databricks/actions/get-sql-warehouse-permissions/get-sql-warehouse-permissions.mjs (1)
  • response (20-23)
components/databricks/actions/start-sql-warehouse/start-sql-warehouse.mjs (1)
  • response (20-23)
components/databricks/actions/stop-sql-warehouse/stop-sql-warehouse.mjs (1)
  • response (20-23)
components/databricks/actions/set-sql-warehouse-config/set-sql-warehouse-config.mjs (1)
components/databricks/actions/get-sql-warehouse-config/get-sql-warehouse-config.mjs (1)
  • response (13-15)
components/databricks/actions/stop-sql-warehouse/stop-sql-warehouse.mjs (2)
components/databricks/actions/get-sql-warehouse/get-sql-warehouse.mjs (1)
  • response (20-23)
components/databricks/actions/start-sql-warehouse/start-sql-warehouse.mjs (1)
  • response (20-23)
components/databricks/actions/start-sql-warehouse/start-sql-warehouse.mjs (3)
components/databricks/actions/edit-sql-warehouse/edit-sql-warehouse.mjs (1)
  • response (174-178)
components/databricks/actions/get-sql-warehouse/get-sql-warehouse.mjs (1)
  • response (20-23)
components/databricks/actions/stop-sql-warehouse/stop-sql-warehouse.mjs (1)
  • response (20-23)
components/databricks/actions/delete-vector-search-index/delete-vector-search-index.mjs (1)
components/databricks/actions/get-vector-search-index/get-vector-search-index.mjs (1)
  • response (20-23)
components/databricks/common/utils.mjs (1)
components/databricks/actions/set-sql-warehouse-config/set-sql-warehouse-config.mjs (2)
  • obj (110-110)
  • obj (119-121)
components/databricks/actions/scan-vector-search-index/scan-vector-search-index.mjs (4)
components/databricks/actions/delete-vector-search-index-data/delete-vector-search-index-data.mjs (1)
  • response (34-38)
components/databricks/actions/delete-vector-search-index/delete-vector-search-index.mjs (1)
  • response (19-22)
components/databricks/actions/get-vector-search-index/get-vector-search-index.mjs (1)
  • response (20-23)
components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs (1)
  • response (80-84)
components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs (5)
components/databricks/actions/upsert-vector-search-index-data/upsert-vector-search-index-data.mjs (1)
  • payload (42-45)
components/databricks/actions/delete-vector-search-index-data/delete-vector-search-index-data.mjs (1)
  • response (34-38)
components/databricks/actions/get-vector-search-index/get-vector-search-index.mjs (1)
  • response (20-23)
components/databricks/actions/list-vector-search-indexes/list-vector-search-indexes.mjs (1)
  • response (14-14)
components/databricks/actions/scan-vector-search-index/scan-vector-search-index.mjs (1)
  • response (43-47)
components/databricks/actions/create-vector-search-index/create-vector-search-index.mjs (7)
components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs (2)
  • payload (57-60)
  • response (80-84)
components/databricks/actions/upsert-vector-search-index-data/upsert-vector-search-index-data.mjs (1)
  • payload (42-45)
components/databricks/actions/delete-vector-search-index-data/delete-vector-search-index-data.mjs (1)
  • response (34-38)
components/databricks/actions/delete-vector-search-index/delete-vector-search-index.mjs (1)
  • response (19-22)
components/databricks/actions/get-vector-search-index/get-vector-search-index.mjs (1)
  • response (20-23)
components/databricks/actions/list-vector-search-indexes/list-vector-search-indexes.mjs (1)
  • response (14-14)
components/databricks/actions/scan-vector-search-index/scan-vector-search-index.mjs (1)
  • response (43-47)
components/databricks/actions/edit-sql-warehouse/edit-sql-warehouse.mjs (1)
components/databricks/actions/create-sql-warehouse/create-sql-warehouse.mjs (4)
  • payload (105-108)
  • parsedTags (135-135)
  • tagArray (136-142)
  • response (160-163)
components/databricks/actions/create-sql-warehouse/create-sql-warehouse.mjs (2)
components/databricks/actions/create-vector-search-index/create-vector-search-index.mjs (2)
  • payload (68-73)
  • response (109-112)
components/databricks/actions/edit-sql-warehouse/edit-sql-warehouse.mjs (4)
  • payload (106-106)
  • parsedTags (148-148)
  • tagArray (149-155)
  • response (174-178)
components/databricks/actions/get-sql-warehouse-permissions/get-sql-warehouse-permissions.mjs (3)
components/databricks/actions/get-sql-warehouse-config/get-sql-warehouse-config.mjs (1)
  • response (13-15)
components/databricks/actions/get-sql-warehouse/get-sql-warehouse.mjs (1)
  • response (20-23)
components/databricks/actions/set-sql-warehouse-permissions/set-sql-warehouse-permissions.mjs (1)
  • response (33-39)
components/databricks/actions/sync-vector-search-index/sync-vector-search-index.mjs (5)
components/databricks/actions/delete-vector-search-index-data/delete-vector-search-index-data.mjs (1)
  • response (34-38)
components/databricks/actions/delete-vector-search-index/delete-vector-search-index.mjs (1)
  • response (19-22)
components/databricks/actions/get-vector-search-index/get-vector-search-index.mjs (1)
  • response (20-23)
components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs (1)
  • response (80-84)
components/databricks/actions/scan-vector-search-index/scan-vector-search-index.mjs (1)
  • response (43-47)
components/databricks/actions/get-sql-warehouse-config/get-sql-warehouse-config.mjs (1)
components/databricks/actions/list-vector-search-indexes/list-vector-search-indexes.mjs (1)
  • response (14-14)
components/databricks/actions/upsert-vector-search-index-data/upsert-vector-search-index-data.mjs (4)
components/databricks/actions/create-vector-search-index/create-vector-search-index.mjs (2)
  • payload (68-73)
  • response (109-112)
components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs (2)
  • payload (57-60)
  • response (80-84)
components/databricks/actions/delete-vector-search-index-data/delete-vector-search-index-data.mjs (1)
  • response (34-38)
components/databricks/actions/delete-vector-search-index/delete-vector-search-index.mjs (1)
  • response (19-22)
components/databricks/actions/set-sql-warehouse-permissions/set-sql-warehouse-permissions.mjs (2)
components/databricks/actions/edit-sql-warehouse/edit-sql-warehouse.mjs (1)
  • response (174-178)
components/databricks/actions/get-sql-warehouse-permissions/get-sql-warehouse-permissions.mjs (1)
  • response (20-23)
🔇 Additional comments (25)
components/databricks/actions/run-job-now/run-job-now.mjs (1)

7-7: Version bump only — LGTM.

components/databricks/actions/list-runs/list-runs.mjs (1)

55-57: Trimming fallback still fine — keep as a safety net.

components/databricks/actions/get-run-output/get-run-output.mjs (1)

7-7: Version bump only — LGTM.

components/databricks/common/constants.mjs (1)

13-15: Default export — LGTM.

components/databricks/actions/list-vector-search-indexes/list-vector-search-indexes.mjs (1)

13-20: LGTM.

Straightforward call/summary/return flow is correct.

components/databricks/actions/delete-vector-search-index/delete-vector-search-index.mjs (1)

18-26: LGTM.

Uses propDefinition, invokes app method, and exports a clear summary.

components/databricks/actions/get-vector-search-index/get-vector-search-index.mjs (1)

19-30: LGTM.

Consistent with other actions; summary is helpful.

components/databricks/actions/start-sql-warehouse/start-sql-warehouse.mjs (1)

7-7: Version bump sanity check.

New action files elsewhere use 0.0.1; confirm 0.0.3 is intentional for this action.

components/databricks/actions/get-sql-warehouse-config/get-sql-warehouse-config.mjs (1)

12-18: LGTM — action wiring and summary are correct

Straightforward call-through to the app method with a clear summary. Versioning and metadata look consistent with the suite.

components/databricks/actions/set-sql-warehouse-config/set-sql-warehouse-config.mjs (1)

77-88: Sanity-check allowed keys against Databricks SQL Warehouse API contract
No JSDoc or schema for setSQLWarehouseConfig in databricks.app.mjs; manually compare your allowed list with the official /sql/config/warehouses PUT request schema in the Databricks REST API docs or OpenAPI spec to ensure you’re not dropping any supported fields.

components/databricks/actions/list-sql-warehouses/list-sql-warehouses.mjs (2)

12-16: LGTM on the listing call + shape.

The call pattern and destructuring look good.


13-16: No action needed: listSQLWarehouses returns { warehouses } as expected
GET /2.0/sql/warehouses returns JSON of the form { "warehouses": [ … ] } (learn.microsoft.com), and listSQLWarehouses() simply returns that raw JSON.

components/databricks/actions/get-sql-warehouse/get-sql-warehouse.mjs (1)

20-23: getSQLWarehouse signature validated
Method getSQLWarehouse in components/databricks/databricks.app.mjs:138 destructures { warehouseId, ...args }, so invoking it with { warehouseId: this.warehouseId, $ } is correct.

components/databricks/actions/stop-sql-warehouse/stop-sql-warehouse.mjs (1)

6-6: Confirm and align Stop API reference & states

  • Update description link in components/databricks/actions/stop-sql-warehouse/stop-sql-warehouse.mjs (line 6) to the canonical Stop API docs: https://docs.databricks.com/sql/api/warehouses/stop
  • Verify polling or summary logic uses response.state === "STOPPED" as a terminal state alongside STARTING, RUNNING, and STOPPING
components/databricks/actions/scan-vector-search-index/scan-vector-search-index.mjs (1)

49-53: No changes needed for response shape
The ScanVectorIndexResponse schema returns the entries in response.data as a list, so the existing summary and response.data.length check are correct.

components/databricks/actions/upsert-vector-search-index-data/upsert-vector-search-index-data.mjs (1)

26-41: Confirmed utils.parseObject handles arrays—no changes needed
parseObject uses Array.isArray to recursively map arrays, so parsedRows will be an array when rows is an array or a JSON string thereof. Keep the existing array check.

components/databricks/actions/set-sql-warehouse-permissions/set-sql-warehouse-permissions.mjs (1)

26-33: No action required: utils.parseObject already performs recursive array mapping, so the subsequent filter covers empty or falsy entries correctly.

components/databricks/actions/create-sql-warehouse/create-sql-warehouse.mjs (1)

110-115: Good guard on autoStopMinutes

Clear validation and mapping to auto_stop_mins looks correct.

components/databricks/actions/create-vector-search-index/create-vector-search-index.mjs (3)

75-81: Good: required field enforcement for DELTA_SYNC

Clear, early validation of sourceTable for DELTA_SYNC.


114-118: Good: informative success summary

Exports a concise, useful $summary.


50-56: Fix prop type: embeddingSourceColumns should be object[] (not string[])

The API expects an array of objects, but the prop is declared as string[] and later treated as objects. This will yield a malformed payload at runtime.

Apply this diff:

-    embeddingSourceColumns: {
-      type: "string[]",
+    embeddingSourceColumns: {
+      type: "object[]",
       label: "Embedding Source Columns",
       description:
-        "List of embedding source column configs. Each entry should be a JSON object string like `{ \"embedding_model_endpoint_name\": \"e5-small-v2\", \"name\": \"text\" }`",
+        "List of embedding source column configs. Example: `{ \"embedding_model_endpoint_name\": \"e5-small-v2\", \"name\": \"text\" }`",
       optional: true,
     },
⛔ Skipped due to learnings
Learnt from: GTFalcao
PR: PipedreamHQ/pipedream#13961
File: components/fakturoid/actions/create-invoice/create-invoice.mjs:95-99
Timestamp: 2024-10-15T15:07:38.361Z
Learning: An "object array" type is not yet supported for Pipedream props.
components/databricks/actions/edit-sql-warehouse/edit-sql-warehouse.mjs (3)

108-121: Validation reads well

Name length and autoStopMins constraints are clear and enforceable.


123-138: Cluster bounds validation looks correct

Range checks and max >= min constraint are accurate.


180-181: Good: succinct success summary

Clear confirmation of the edited warehouse.

components/databricks/databricks.app.mjs (1)

62-75: LGTM: dynamic index options

Simple, correct mapping of index names to option labels.

@vunguyenhung vunguyenhung moved this from Ready for QA to Changes Required in Component (Source and Action) Backlog Sep 18, 2025
@vunguyenhung
Copy link
Collaborator

Hello everyone, I have tested this PR and there're some test cases failed or needed improvement.

Please check the test report below for more information
https://vunguyenhung.notion.site/Databricks-sql-warehouses-268bf548bb5e8103acc1c7d5a2474495

@Lokeshchand33
Copy link
Contributor Author

Hii everyone i have sent the fixes

@Lokeshchand33
Copy link
Contributor Author

hey everyone can anyone help me ..i m getting following error from my indexName propDefinitions :

"Field 'endpoint_name' must be specified for the request.",

Hi @jcortes this issue is fixed in all components except the 'Delete Vector Search Index' component. I reviewed the code but couldn’t find any issue. However, the test case above still shows this error for that component.

@jcortes jcortes self-requested a review September 18, 2025 12:37
Copy link
Collaborator

@jcortes jcortes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Lokeshchand33 can you please first resolve the conflicts in databricks.app.mjs? Also can you please make sure this change is applied to the endpoints that you tested so we can avoid potential issues in other endpoints

@Lokeshchand33
Copy link
Contributor Author

Hi @Lokeshchand33 can you please first resolve the conflicts in databricks.app.mjs? Also can you please make sure this change is applied to the endpoints that you tested so we can avoid potential issues in other endpoints

Hi @jcortes , I’ve resolved the conflicts in databricks.app.mjs.
yes these changes have only been applied to the components I tested,

Copy link
Collaborator

@jcortes jcortes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Lokeshchand33 please remove the paramsSerializer from here and put it inside the methods you have tested

Comment on lines 137 to 139
paramsSerializer: {
indexes: null,
},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is going to apply to other endpoints that were not tested, so please put this parameter inside the endpoints that you have tested only! Thanks

@Lokeshchand33
Copy link
Contributor Author

Hi @Lokeshchand33 please remove the paramsSerializer from here and put it inside the methods you have tested

Hi @jcortes .I’ve made the changes

@jcortes jcortes moved this from Changes Required to In Review in Component (Source and Action) Backlog Sep 19, 2025
@jcortes jcortes moved this from In Review to Ready for QA in Component (Source and Action) Backlog Sep 19, 2025
@vunguyenhung vunguyenhung moved this from Ready for QA to Changes Required in Component (Source and Action) Backlog Sep 21, 2025
@vunguyenhung
Copy link
Collaborator

Hello everyone, I have tested this PR and there're some test cases failed or needed improvement.

Please check the test report below for more information
https://vunguyenhung.notion.site/Databricks-sql-warehouses-268bf548bb5e8103acc1c7d5a2474495

@Lokeshchand33
Copy link
Contributor Author

Hey everyone , I’ve pushed the fixes.

@michelle0927 michelle0927 moved this from Changes Required to Ready for QA in Component (Source and Action) Backlog Sep 22, 2025
@vunguyenhung vunguyenhung moved this from Ready for QA to Ready for Release in Component (Source and Action) Backlog Sep 23, 2025
@vunguyenhung
Copy link
Collaborator

Hi everyone, all test cases are passed! Ready for release!

Test report
https://vunguyenhung.notion.site/Databricks-sql-warehouses-268bf548bb5e8103acc1c7d5a2474495

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
components/databricks/actions/repair-run/repair-run.mjs (1)

23-34: Enforce “exactly one” of rerunTasks or rerunAllFailedTasks; build minimal payload

Currently both can be omitted or provided together, and an empty pipeline_params object may be sent. Add validation and only include populated fields in the request.

Apply this diff:

   async run({ $ }) {
     const {
       app,
       runId,
       rerunTasks,
       rerunAllFailedTasks,
       pipelineParamsFullRefresh,
     } = this;

-    const response = await app.repairRun({
-      $,
-      data: {
-        run_id: runId,
-        rerun_tasks: rerunTasks,
-        rerun_all_failed_tasks: rerunAllFailedTasks,
-        pipeline_params: {
-          full_refresh: pipelineParamsFullRefresh,
-        },
-      },
-    });
+    // Validate: exactly one of rerunTasks or rerunAllFailedTasks
+    if (rerunAllFailedTasks && (rerunTasks?.length)) {
+      throw new Error("Specify either Rerun Tasks or Rerun All Failed Tasks, not both.");
+    }
+    if (!rerunAllFailedTasks && (!rerunTasks || rerunTasks.length === 0)) {
+      throw new Error("Provide either Rerun Tasks or enable Rerun All Failed Tasks.");
+    }
+
+    const data = { run_id: runId };
+    if (rerunAllFailedTasks) data.rerun_all_failed_tasks = true;
+    if (rerunTasks?.length) data.rerun_tasks = rerunTasks;
+    if (typeof pipelineParamsFullRefresh === "boolean") {
+      data.pipeline_params = { full_refresh: pipelineParamsFullRefresh };
+    }
+
+    const response = await app.repairRun({ $, data });

Also applies to: 42-61

components/databricks/databricks.app.mjs (1)

68-76: Guard prevContext to avoid crash in async options

Accessing prevContext.page_token can throw when prevContext is undefined.

Apply:

-      async options({ prevContext }) {
+      async options({ prevContext } = {}) {
         const {
           endpoints = [], next_page_token,
         } = await this.listEndpoints({
           params: {
-            page_token: prevContext.page_token,
+            page_token: prevContext?.page_token,
           },
         });
🧹 Nitpick comments (3)
components/databricks/actions/repair-run/repair-run.mjs (1)

63-65: Harden summary against response shape differences

If the API returns a different field, the summary may print “undefined”. Use a safe fallback.

-    $.export("$summary", `Successfully initiated repair of run with ID \`${response.repair_id}\`.`);
+    $.export("$summary", `Successfully initiated repair of run with ID \`${response?.repair_id ?? runId}\`.`);
components/databricks/databricks.app.mjs (1)

285-306: URL‑encode endpointName in paths

Encode dynamic segments to avoid broken URLs and path confusion.

       return this._makeRequest({
-        path: `/vector-search/endpoints/${endpointName}`,
+        path: `/vector-search/endpoints/${encodeURIComponent(endpointName)}`,
         ...opts,
       });
       return this._makeRequest({
-        path: `/vector-search/endpoints/${endpointName}`,
+        path: `/vector-search/endpoints/${encodeURIComponent(endpointName)}`,
         method: "DELETE",
         ...opts,
       });
components/databricks/actions/upsert-vector-search-index-data/upsert-vector-search-index-data.mjs (1)

35-40: Friendlier validation on JSON parse

Wrap parse to surface a clear ConfigurationError on invalid JSON.

-    const parsedRows = utils.parseObject(this.rows);
-
-    if (!Array.isArray(parsedRows) || !parsedRows.length) {
+    let parsedRows;
+    try {
+      parsedRows = utils.parseObject(this.rows);
+    } catch (e) {
+      throw new ConfigurationError("rows must be a valid JSON array string.");
+    }
+    if (!Array.isArray(parsedRows) || parsedRows.length === 0) {
       throw new ConfigurationError("rows must be a non-empty JSON array.");
     }
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 008e23b and 6b54d05.

📒 Files selected for processing (36)
  • components/databricks/actions/cancel-all-runs/cancel-all-runs.mjs (1 hunks)
  • components/databricks/actions/cancel-run/cancel-run.mjs (1 hunks)
  • components/databricks/actions/create-endpoint/create-endpoint.mjs (1 hunks)
  • components/databricks/actions/create-job/create-job.mjs (1 hunks)
  • components/databricks/actions/create-sql-warehouse/create-sql-warehouse.mjs (1 hunks)
  • components/databricks/actions/delete-endpoint/delete-endpoint.mjs (1 hunks)
  • components/databricks/actions/delete-job/delete-job.mjs (1 hunks)
  • components/databricks/actions/delete-run/delete-run.mjs (1 hunks)
  • components/databricks/actions/delete-sql-warehouse/delete-sql-warehouse.mjs (1 hunks)
  • components/databricks/actions/edit-sql-warehouse/edit-sql-warehouse.mjs (1 hunks)
  • components/databricks/actions/export-run/export-run.mjs (1 hunks)
  • components/databricks/actions/get-endpoint/get-endpoint.mjs (1 hunks)
  • components/databricks/actions/get-job-permissions/get-job-permissions.mjs (1 hunks)
  • components/databricks/actions/get-job/get-job.mjs (1 hunks)
  • components/databricks/actions/get-run-output/get-run-output.mjs (1 hunks)
  • components/databricks/actions/get-run/get-run.mjs (1 hunks)
  • components/databricks/actions/get-sql-warehouse-config/get-sql-warehouse-config.mjs (1 hunks)
  • components/databricks/actions/get-sql-warehouse-permissions/get-sql-warehouse-permissions.mjs (1 hunks)
  • components/databricks/actions/get-sql-warehouse/get-sql-warehouse.mjs (1 hunks)
  • components/databricks/actions/list-endpoints/list-endpoints.mjs (1 hunks)
  • components/databricks/actions/list-jobs/list-jobs.mjs (1 hunks)
  • components/databricks/actions/list-runs/list-runs.mjs (1 hunks)
  • components/databricks/actions/list-sql-warehouses/list-sql-warehouses.mjs (1 hunks)
  • components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs (1 hunks)
  • components/databricks/actions/repair-run/repair-run.mjs (1 hunks)
  • components/databricks/actions/reset-job/reset-job.mjs (1 hunks)
  • components/databricks/actions/run-job-now/run-job-now.mjs (1 hunks)
  • components/databricks/actions/set-job-permissions/set-job-permissions.mjs (1 hunks)
  • components/databricks/actions/set-sql-warehouse-config/set-sql-warehouse-config.mjs (1 hunks)
  • components/databricks/actions/set-sql-warehouse-permissions/set-sql-warehouse-permissions.mjs (1 hunks)
  • components/databricks/actions/start-sql-warehouse/start-sql-warehouse.mjs (1 hunks)
  • components/databricks/actions/stop-sql-warehouse/stop-sql-warehouse.mjs (1 hunks)
  • components/databricks/actions/update-job/update-job.mjs (1 hunks)
  • components/databricks/actions/upsert-vector-search-index-data/upsert-vector-search-index-data.mjs (1 hunks)
  • components/databricks/databricks.app.mjs (3 hunks)
  • components/databricks/package.json (1 hunks)
✅ Files skipped from review due to trivial changes (15)
  • components/databricks/actions/update-job/update-job.mjs
  • components/databricks/actions/get-run/get-run.mjs
  • components/databricks/actions/create-job/create-job.mjs
  • components/databricks/actions/delete-job/delete-job.mjs
  • components/databricks/actions/reset-job/reset-job.mjs
  • components/databricks/actions/cancel-all-runs/cancel-all-runs.mjs
  • components/databricks/actions/get-job/get-job.mjs
  • components/databricks/actions/delete-run/delete-run.mjs
  • components/databricks/actions/run-job-now/run-job-now.mjs
  • components/databricks/actions/export-run/export-run.mjs
  • components/databricks/actions/set-job-permissions/set-job-permissions.mjs
  • components/databricks/actions/list-jobs/list-jobs.mjs
  • components/databricks/actions/get-job-permissions/get-job-permissions.mjs
  • components/databricks/actions/stop-sql-warehouse/stop-sql-warehouse.mjs
  • components/databricks/actions/list-endpoints/list-endpoints.mjs
🚧 Files skipped from review as they are similar to previous changes (15)
  • components/databricks/actions/list-sql-warehouses/list-sql-warehouses.mjs
  • components/databricks/actions/edit-sql-warehouse/edit-sql-warehouse.mjs
  • components/databricks/actions/create-sql-warehouse/create-sql-warehouse.mjs
  • components/databricks/actions/get-sql-warehouse-config/get-sql-warehouse-config.mjs
  • components/databricks/actions/delete-endpoint/delete-endpoint.mjs
  • components/databricks/actions/get-run-output/get-run-output.mjs
  • components/databricks/actions/set-sql-warehouse-config/set-sql-warehouse-config.mjs
  • components/databricks/actions/get-sql-warehouse/get-sql-warehouse.mjs
  • components/databricks/actions/get-endpoint/get-endpoint.mjs
  • components/databricks/actions/create-endpoint/create-endpoint.mjs
  • components/databricks/actions/list-runs/list-runs.mjs
  • components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs
  • components/databricks/actions/set-sql-warehouse-permissions/set-sql-warehouse-permissions.mjs
  • components/databricks/actions/get-sql-warehouse-permissions/get-sql-warehouse-permissions.mjs
  • components/databricks/package.json
🧰 Additional context used
🧬 Code graph analysis (1)
components/databricks/actions/upsert-vector-search-index-data/upsert-vector-search-index-data.mjs (5)
components/databricks/actions/query-vector-search-index/query-vector-search-index.mjs (1)
  • response (95-99)
components/databricks/actions/delete-vector-search-index/delete-vector-search-index.mjs (1)
  • response (28-31)
components/databricks/actions/delete-vector-search-index-data/delete-vector-search-index-data.mjs (1)
  • response (50-56)
components/databricks/actions/get-vector-search-index/get-vector-search-index.mjs (1)
  • response (29-32)
components/databricks/actions/scan-vector-search-index/scan-vector-search-index.mjs (1)
  • response (52-56)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: pnpm publish
  • GitHub Check: Lint Code Base
  • GitHub Check: Publish TypeScript components
  • GitHub Check: Verify TypeScript components
🔇 Additional comments (11)
components/databricks/actions/cancel-run/cancel-run.mjs (1)

7-7: Version bump only — LGTM

Matches the broader PR pattern; no behavior change.

components/databricks/actions/repair-run/repair-run.mjs (1)

7-7: Version bump — LGTM

No runtime changes; metadata-only update looks good.

components/databricks/actions/delete-sql-warehouse/delete-sql-warehouse.mjs (1)

7-7: LGTM: version bump only — verify SQL Warehouse actions & component package version

No behavior change. I scanned components/databricks/actions/-sql-warehouse/.mjs and found no files. Confirm all SQL Warehouse action files are set to version "0.0.4" and components/databricks/package.json (or equivalent) reflects the intended package version; if files were moved/renamed, provide the correct paths.

components/databricks/actions/start-sql-warehouse/start-sql-warehouse.mjs (1)

7-7: Version bump to 0.0.4 — LGTM

  • startSQLWarehouse is implemented in components/databricks/databricks.app.mjs (≈line 341).
  • Search found no version: "x.y.z" entries under components/databricks/actions/-sql-warehouse/.mjs — confirm all SQL Warehouse action files were bumped to 0.0.4 and that startSQLWarehouse targets the correct Databricks endpoint.
components/databricks/databricks.app.mjs (6)

86-89: Fix misleading Warehouse ID description

Warehouses don’t have “runs”. Keep it concise and accurate.

-      description: "The ID of the SQL Warehouse to get runs from",
+      description: "The SQL Warehouse ID",

100-119: Support paginated async options for indexName

Return { options, context } and page using prevContext to handle large lists.

-      async options({ endpointName }) {
-        if (!endpointName) {
-          return [];
-        }
-        const { vector_indexes = [] } = await this.listVectorSearchIndexes({
-          params: {
-            endpoint_name: endpointName,
-          },
-        });
-
-        return vector_indexes.map(({ name }) => ({
-          value: name,
-          label: name,
-        }));
-      },
+      async options({ endpointName, prevContext } = {}) {
+        if (!endpointName) {
+          return { options: [], context: {} };
+        }
+        const {
+          vector_indexes = [],
+          next_page_token,
+        } = await this.listVectorSearchIndexes({
+          params: {
+            endpoint_name: endpointName,
+            page_token: prevContext?.page_token,
+          },
+        });
+        return {
+          options: vector_indexes.map(({ name }) => ({
+            value: name,
+            label: name,
+          })),
+          context: { page_token: next_page_token },
+        };
+      },

308-359: URL‑encode warehouseId in all SQL Warehouse endpoints

Defensive encoding for IDs containing reserved characters.

-        path: `/sql/warehouses/${warehouseId}`,
+        path: `/sql/warehouses/${encodeURIComponent(warehouseId)}`,
-        path: `/sql/warehouses/${warehouseId}`,
+        path: `/sql/warehouses/${encodeURIComponent(warehouseId)}`,
-        path: `/sql/warehouses/${warehouseId}/edit`,
+        path: `/sql/warehouses/${encodeURIComponent(warehouseId)}/edit`,
-        path: `/sql/warehouses/${warehouseId}/start`,
+        path: `/sql/warehouses/${encodeURIComponent(warehouseId)}/start`,
-        path: `/sql/warehouses/${warehouseId}/stop`,
+        path: `/sql/warehouses/${encodeURIComponent(warehouseId)}/stop`,

376-393: Also encode warehouseId in permissions paths

Consistency with other warehouse endpoints.

-        path: `/permissions/warehouses/${warehouseId}`,
+        path: `/permissions/warehouses/${encodeURIComponent(warehouseId)}`,
-        path: `/permissions/warehouses/${warehouseId}`,
+        path: `/permissions/warehouses/${encodeURIComponent(warehouseId)}`,

402-486: URL‑encode indexName in all vector index paths

Index names often include dots and other reserved chars.

-        path: `/vector-search/indexes/${indexName}`,
+        path: `/vector-search/indexes/${encodeURIComponent(indexName)}`,
-        path: `/vector-search/indexes/${indexName}`,
+        path: `/vector-search/indexes/${encodeURIComponent(indexName)}`,
-        path: `/vector-search/indexes/${indexName}/query`,
+        path: `/vector-search/indexes/${encodeURIComponent(indexName)}/query`,
-        path: `/vector-search/indexes/${indexName}/sync`,
+        path: `/vector-search/indexes/${encodeURIComponent(indexName)}/sync`,
-        path: `/vector-search/indexes/${indexName}/delete-data`,
+        path: `/vector-search/indexes/${encodeURIComponent(indexName)}/delete-data`,
-        path: `/vector-search/indexes/${indexName}/upsert-data`,
+        path: `/vector-search/indexes/${encodeURIComponent(indexName)}/upsert-data`,
-        path: `/vector-search/indexes/${indexName}/scan`,
+        path: `/vector-search/indexes/${encodeURIComponent(indexName)}/scan`,

412-421: Make params optional and safe

Default params to {} to avoid undefined access and allow extensible queries.

-    listVectorSearchIndexes({
-      params, ...args
-    }) {
+    listVectorSearchIndexes({ params = {}, ...args } = {}) {
       return this._makeRequest({
         path: "/vector-search/indexes",
         method: "GET",
         params,
         ...args,
       });
     },
components/databricks/actions/upsert-vector-search-index-data/upsert-vector-search-index-data.mjs (1)

42-51: LGTM: upsert call and summary are correct

Correct endpoint usage and payload shape (inputs_json).

Copy link
Collaborator

@michelle0927 michelle0927 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving and merging since @jcortes is out of office.

@michelle0927 michelle0927 merged commit 946cafd into PipedreamHQ:master Sep 23, 2025
10 checks passed
@github-project-automation github-project-automation bot moved this from Ready for Release to Done in Component (Source and Action) Backlog Sep 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
User submitted Submitted by a user
Development

Successfully merging this pull request may close these issues.

[ACTION] Databricks API extension
7 participants