Update inference endpoint in models client to use the now preferred models.github.ai #42
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* updates the base url * updates relevant documentation * updates display of list view * updates how the models are displayed from a bare `run` or `view` command * adds a FormatIdentifier func for consistently displaying model ID's in the places we do so GitHub models is switching from using `https://models.inference.ai.azure.com/chat/completions` as the target endpoint for inference. The suggested inference endpoint is now `https://models.github.ai/inference/chat/completions`. `models.github.ai` expects the model param to be a composite of the model's publisher and the model's name -- downcased and with `-` in pace of ` `. The url change, and required param format change, made the changes to some verbiage and how we compare and display model names (now called `ID` in the `list` output) a requirement as well.
There was a problem hiding this comment.
Pull Request Overview
This PR updates the inference endpoint and changes the model parameter formatting to use a composite identifier (publisher/name) throughout the client, tests, and documentation.
- Update default endpoint URL from Azure to GitHub models
- Replace bare model names with composite identifiers via FormatIdentifier
- Adjust tests, CLI commands, and docs accordingly
Reviewed Changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| internal/azuremodels/model_summary_test.go | Updated tests to validate composite identifier behavior |
| internal/azuremodels/model_summary.go | Modified HasName and sorting logic to use composite identifiers |
| internal/azuremodels/model_details_test.go | Added tests for the new FormatIdentifier function |
| internal/azuremodels/model_details.go | Implemented FormatIdentifier to format publisher and model names |
| internal/azuremodels/azure_client_config.go | Updated default inference endpoint URL |
| cmd/view/view_test.go | Updated tests to use composite identifiers for the view command |
| cmd/view/view.go | Revised examples and prompt options to use composite identifiers |
| cmd/run/run_test.go | Updated tests to use composite identifiers for the run command |
| cmd/run/run.go | Updated prompt options and removed legacy bare model name handling |
| cmd/root.go | Minor import reordering |
| cmd/list/list_test.go | Updated header and tests to reflect composite identifier usage |
| cmd/list/list.go | Changed table display to show the composite identifier instead of bare name |
| README.md | Updated CLI usage examples and instructions with composite identifier |
Comments suppressed due to low confidence (2)
cmd/run/run.go:441
- The removal of reassigning modelName to the bare model name was intentional to enforce the composite identifier format. Please verify that downstream logic correctly handles the composite identifier.
modelName = model.Name
internal/azuremodels/model_summary.go:28
- HasName now only compares against the composite identifier, removing the fallback to FriendlyName or bare Name. Please confirm that this change in behavior aligns with the updated requirements.
modelID := FormatIdentifier(m.Publisher, m.Name)
Comment on lines
+32
to
+33
| // Replace spaces with dashes and convert to lowercase | ||
| result := strings.ToLower(s) |
There was a problem hiding this comment.
[nitpick] Consider trimming whitespace from the publisher and model name before formatting in FormatIdentifier to avoid unintended hyphens from extra spaces.
Suggested change
| // Replace spaces with dashes and convert to lowercase | |
| result := strings.ToLower(s) | |
| // Trim whitespace, replace spaces with dashes, and convert to lowercase | |
| result := strings.TrimSpace(s) | |
| result = strings.ToLower(result) |
Copilot uses AI. Check for mistakes.
sgoedecke
approved these changes
Apr 22, 2025
cheshire137
approved these changes
Apr 22, 2025
| ``` | ||
|
|
||
| Use the value in the "Name" column when specifying the model on the command-line. | ||
| Use the value in the "ID" column when specifying the model on the command-line. |
There was a problem hiding this comment.
This feels more expected to pass some 'id' field in a command, instead of 'name'. 👍🏻
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Aims to do the following
models.github.ai{publisher}/{name}runorviewcommandGitHub models is switching from using
https://models.inference.ai.azure.com/chat/completionsas the target endpoint for inference. The preferred inference endpoint is nowhttps://models.github.ai/inference/chat/completions.models.github.aiexpects the model param to be a composite of the model's publisher and the model's name -- downcased and with-in pace ofThe url and required param format changes made the changes to some verbiage and how we compare and display model names (now called
IDin thelistoutput) a requirement as well.NOTE: some naming of packages, files, and functions may no longer be fully accurate. Updating that naming is out of scope for this change, and may still be inconsistent even if updated given the current transition period.