feat(generativeai): Create genai_sdk_supervised_checkpoints_test_exa… #13347
Conversation
…ple.py Add an example to test tuned checkpoints.
There was a problem hiding this comment.
Hello @yishan-pu, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
Summary of Changes
This pull request adds a new example, genai_sdk_supervised_checkpoints_test_example.py, to test tuned checkpoints using the genai SDK. The example demonstrates how to generate content using the default checkpoint and specific checkpoints from a tuning job.
Highlights
- New Example: A new example file
generative_ai/model_tuning/genai_sdk_supervised_checkpoints_test_example.pyhas been added to demonstrate testing of tuned checkpoints. - Checkpoint Testing: The example tests the default checkpoint and two specific checkpoints (index 0 and 1) of a tuned model endpoint.
- genai SDK Usage: The example uses the
google.genaiSDK to interact with the generative AI service, retrieving a tuning job and generating content.
Changelog
- generative_ai/model_tuning/genai_sdk_supervised_checkpoints_test_example.py
- Added a new file to demonstrate testing tuned checkpoints.
- The example retrieves a tuning job using
client.tunings.get(name=name). - The example generates content using the default checkpoint via
client.models.generate_content(model=tuning_job.tuned_model.endpoint, contents=contents). - The example generates content using checkpoint 1 via
client.models.generate_content(model=tuning_job.tuned_model.checkpoints[0].endpoint, contents=contents). - The example generates content using checkpoint 2 via
client.models.generate_content(model=tuning_job.tuned_model.checkpoints[1].endpoint, contents=contents).
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
A checkpoint's tale,
Models tuned, will not fail,
Answers now refined.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
The code adds a new example to test tuned checkpoints, which is a valuable addition to the generative AI SDK. The example covers testing the default checkpoint and two additional checkpoints. Overall, the code is clear and well-structured.
Merge Readiness
The code is well-structured and the example is clear. I am unable to directly approve the pull request, and recommend that others review and approve this code before merging. There are no critical or high severity issues, so it is up to the other reviewers to decide if the pull request is ready to be merged.
| # TODO(developer): Update and un-comment below lines | ||
| # PROJECT_ID = "your-project-id" |
There was a problem hiding this comment.
It's good practice to provide a more descriptive comment explaining why the developer needs to update and uncomment these lines. For example, specify what your-project-id refers to and how to obtain it.
| # TODO(developer): Update and un-comment below lines | |
| # PROJECT_ID = "your-project-id" | |
| # TODO(developer): Update with your project ID, which can be found in the Google Cloud Console, and uncomment below lines | |
| # PROJECT_ID = "your-project-id" |
| location="us-central1", | ||
| ) | ||
|
|
||
| name = "projects/12345678/locations/us-central1/tuningJobs/123456789012345" |
There was a problem hiding this comment.
The name variable is hardcoded. While this is an example, it would be beneficial to provide guidance on how a user would obtain the correct tuning job name. Consider adding a comment that explains where to find this information in the Google Cloud Console or via the API.
| name = "projects/12345678/locations/us-central1/tuningJobs/123456789012345" | |
| # TODO(developer): Replace with your actual tuning job name. | |
| # You can find the tuning job name in the Google Cloud Console or via the API. | |
| name = "projects/12345678/locations/us-central1/tuningJobs/123456789012345" |
| ) | ||
|
|
||
| name = "projects/12345678/locations/us-central1/tuningJobs/123456789012345" | ||
| tuning_job = client.tunings.get(name=name) |
There was a problem hiding this comment.
Consider adding error handling in case the tuning job is not found or if there are issues retrieving it. This would make the example more robust.
| tuning_job = client.tunings.get(name=name) | |
| try: | |
| tuning_job = client.tunings.get(name=name) | |
| except Exception as e: | |
| print(f"Error retrieving tuning job: {e}") | |
| return |
| model=tuning_job.tuned_model.endpoint, | ||
| contents=contents, | ||
| ) | ||
| print(response.text) |
There was a problem hiding this comment.
It would be beneficial to add a check to ensure that the response is valid before printing the text. This could prevent errors if the API call fails.
if response and response.text:
print(response.text)
else:
print("No response text")| model=tuning_job.tuned_model.checkpoints[0].endpoint, | ||
| contents=contents, | ||
| ) | ||
| print(checkpoint1_response.text) |
There was a problem hiding this comment.
Similar to the default checkpoint, add a check to ensure that the response is valid before printing the text for checkpoint 1.
if checkpoint1_response and checkpoint1_response.text:
print(checkpoint1_response.text)
else:
print("No response text for checkpoint 1")| model=tuning_job.tuned_model.checkpoints[1].endpoint, | ||
| contents=contents, | ||
| ) | ||
| print(checkpoint2_response.text) |
There was a problem hiding this comment.
Similar to the default checkpoint and checkpoint 1, add a check to ensure that the response is valid before printing the text for checkpoint 2.
if checkpoint2_response and checkpoint2_response.text:
print(checkpoint2_response.text)
else:
print("No response text for checkpoint 2")|
Here is the summary of possible violations 😱 There is a possible violation for not having product prefix.
The end of the violation section. All the stuff below is FYI purposes only. Here is the summary of changes. You are about to add 1 region tag.
This comment is generated by snippet-bot.
|
Add an example to test tuned checkpoints.