Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: Add search-jobs CLI commands for managing search jobs #1145

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
May 5, 2025

Conversation

trly
Copy link
Contributor

@trly trly commented Feb 18, 2025

Add new src search-jobs command with subcommands to manage search jobs:

  • create: Create new search jobs with query validation
  • get: Retrieve search job details by ID
  • list: List search jobs with sorting and pagination
  • cancel: Cancel running search jobs
  • restart: Create a new search jobs with the same query as an existing job
  • delete: Remove search jobs
  • logs: download search job operational logs
  • results: download search job results as JSONL

The command provides functionality to:

  • Format output as JSON, or TSV
  • Sort and filter search jobs
  • Track job status

Test plan

Manual testing:

Tested all new command functionality locally to ensure UX and functionality matched the new feature documentation.
Tested parsing TSV output with awk

go run ./cmd/src search-jobs get U2VhcmNoSm9iOjg3 | awk -F'\t' '{print $1}'
U2VhcmNoSm9iOjg3

go run ./cmd/src search-jobs get U2VhcmNoSm9iOjg3 | awk -F'\t' '{print $2}'
jon.galindo

Handled error conditions tested:

  • invalid search syntax when creating a new search-job
  • cancelling, deleting, getting a non-existing job id
  • cancelling, deleting, getting a job without providing an id
  • failing to create or write to a job log or results file
  • restarting a non-existing job
  • attempting to retrieve a non-existing job
  • attempting to access logs of a non-existing job

@trly trly requested a review from a team as a code owner February 18, 2025 19:05
@trly
Copy link
Contributor Author

trly commented Feb 18, 2025

I'm also working on sourcegraph-docs updates for this, how are those normally synced to release when a new cli-version is released?

@trly
Copy link
Contributor Author

trly commented Feb 18, 2025

This is also my first foray into Go. I'm pretty certain that the tests i generated/built aren't necessarily testing the CLI functionality directly, but rather just mocking request/responses to the mock API. If these aren't necessarily correct or needed, I'm happy to remove them :)

@peterguy
Copy link
Contributor

What do you think about adding logs and download sub-commands?

Downloading the log file or the output file doesn't use the GQL API - it's just $SRC_ENDPOINT/.api/search/export/<id>.log or .jsonl, but might be nice to include as a sub command so the user doesn't have to separately automate a curl call to grab the logs or output.

@trly
Copy link
Contributor Author

trly commented Feb 18, 2025

I can definitely add those. I was debating between that or just leaving it to the user utilizing to tool to make use of the JSON response. Having the sub-command will definitely make it easier for users that aren't integrating with an external tool.

@trly
Copy link
Contributor Author

trly commented Feb 26, 2025

Removed flakey tests that were not functionally testing production code, but rather mocking and re-implementing prod code in tests. functionality for search-jobs has been manually tested

@trly
Copy link
Contributor Author

trly commented Feb 28, 2025

@peterguy those additional functions have been added, I also removed the tests as they weren't actually doing a good job of testing functionality here.

@peterguy
Copy link
Contributor

peterguy commented Mar 4, 2025

I'm also working on sourcegraph-docs updates for this, how are those normally synced to release when a new cli-version is released?

In sourcegraph/docs? I think we manually manage syncing those. CC @MaedahBatool for confirmation.

@trly trly force-pushed the feat/search_jobs branch from 52ec689 to c8acc6d Compare March 12, 2025 14:19
trly and others added 8 commits March 12, 2025 10:44
Add new `src search-jobs` command with subcommands to manage search jobs:
- create: Create new search jobs with query validation
- get: Retrieve search job details by ID
- list: List search jobs with sorting and pagination
- cancel: Cancel running search jobs
- delete: Remove search jobs

The command provides functionality to:
- Format output using Go templates
- Sort and filter search jobs
- Track job status
search_jobs_logs:
Implement a new search-jobs subcommand to retrieve jobs logs from the configured Sourcegraph instance.

search_jobs_get.go:
Extract the GraphQL query logic for fetching a search job into a separate helper function to improve code organization and reusability. This change:

- Creates new getSearchJob helper function that encapsulates the GraphQL query logic
- Simplifies the main handler function by delegating job fetching
- Maintains existing functionality while improving code structure
Add new command to retrieve search job results in JSONL format with the following capabilities:
- Fetch results using search job ID
- Optional file output with -out flag
This commit introduces the `search-jobs restart` command, enabling users to restart a search job by its ID. The command retrieves the query from the original job and creates a new search job with the same query.
@trly trly force-pushed the feat/search_jobs branch from c8acc6d to 50165f4 Compare March 12, 2025 14:45
@jtibshirani jtibshirani requested a review from a team March 12, 2025 19:06
Copy link
Member

@stefanhengl stefanhengl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a first pass and left a couple of comments.

Great job so far! I recommend to first update Sourcegraph to return the database ID in the GQL API and then circle back here to simplify the code.

Other than that I think we can unexport probably most of the exported variables here. It's always best to limit the API surface.

Left some comments about CLI ergonomics, too.

Copy link
Contributor

@jasonhawkharris jasonhawkharris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool tool. Glad to finally have this as part of src-cli. I would echo @stefanhengl's comments regarding some of the encapsulation stuff but other than that LGTM. I'm going to stop short of approving and let Stefan do that but from Codex point of view we're all good here. Cheers!

@trly
Copy link
Contributor Author

trly commented Mar 14, 2025

Thanks for all the feedback. I'll clean up the ID usage next week when I'm back at work.

This commit refactors the search-jobs commands to:
- Implement a builder pattern for consistent command creation
- Add column-based output format with customizable columns
- Support JSON output format for programmatic access
- Improve argument handling by using positional arguments for IDs
- Separate command logic from presentation for better testability
- Extract common functionality into reusable helper functions
- Enhance usage documentation with better examples
- Remove SearchJobID parsing functions in favor of direct ID handling
@trly
Copy link
Contributor Author

trly commented Mar 18, 2025

@stefanhengl, recommended changes and a few refactors have been put in place. Let me know how it looks

@trly
Copy link
Contributor Author

trly commented Apr 9, 2025

@peterguy @stefanhengl, can this be reviewed please?

Copy link
Contributor

@peterguy peterguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great new feature; looking forward to using it! Works great locally.

@trly trly merged commit 29ccb97 into main May 5, 2025
8 checks passed
@trly trly deleted the feat/search_jobs branch May 5, 2025 18:17
trly added a commit to sourcegraph/docs that referenced this pull request May 6, 2025
Add documentation for src search-jobs feature
sourcegraph/src-cli#1145

## Pull Request approval

You will need to get your PR approved by at least one member of the
Sourcegraph team. For reviews of docs formatting, styles, and component
usage, please tag the docs team via the #docs Slack channel.

---------

Co-authored-by: Maedah Batool <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants