Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 2ac36e3

Browse files
Sarah EdwardsSiaraMist
Sarah Edwards
and
SiaraMist
authored
Document GitHub Models (#51698)
Co-authored-by: SiaraMist <[email protected]> Co-authored-by: Siara <[email protected]>
1 parent e4bb133 commit 2ac36e3

8 files changed

Lines changed: 188 additions & 1 deletion

File tree

‎content/get-started/exploring-integrations/about-building-integrations.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,5 @@ Your integration can use {% data variables.product.company_short %}'s API to fet
2424
Your integration can use webhooks to learn when specific events happen on {% data variables.product.company_short %}. For more information, see "[AUTOTITLE](/webhooks/about-webhooks)."
2525

2626
{% ifversion fpt or ghec %} If your integration is a {% data variables.product.prodname_github_app %} or custom action, you can publish your integration on {% data variables.product.prodname_marketplace %}. For more information, see "[AUTOTITLE](/apps/github-marketplace/github-marketplace-overview/about-github-marketplace-for-apps)" and "[AUTOTITLE](/actions/creating-actions/publishing-actions-in-github-marketplace)."{% endif %}
27+
28+
If your integration uses generative AI, you can find and experiment with AI models for free on {% data variables.product.company_short %}. See "[AUTOTITLE](/github-models/prototyping-with-ai-models)."

‎content/github-models/index.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
title: GitHub Models
3+
intro: 'Find and experiment with AI models for free.'
4+
versions:
5+
fpt: '*'
6+
ghes: '*'
7+
ghec: '*'
8+
children:
9+
- /prototyping-with-ai-models
10+
- /responsible-use-of-github-models
11+
---

‎content/github-models/prototyping-with-ai-models.md‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
---
2+
title: Prototyping with AI models
3+
intro: 'Find and experiment with AI models for free.'
4+
versions:
5+
feature: github-models
6+
---
7+
8+
If you want to develop a generative AI application, you can use {% data variables.product.prodname_github_models %} to find and experiment with AI models for free. Once you are ready to bring your application to production, you can switch to a token from a paid Azure account. See the [Azure AI](https://ai.azure.com/github/model/docs) documentation.
9+
10+
See also "[AUTOTITLE](/github-models/responsible-use-of-github-models)."
11+
12+
## Finding AI models
13+
14+
To find AI models, go to [{% data variables.product.prodname_marketplace %}](https://github.com/marketplace/models), then click {% octicon "ai-model" aria-hidden="true" %} **Models** in the sidebar.
15+
16+
To view details about a model, click on the model's name.
17+
18+
## Experimenting with AI models in the playground
19+
20+
>[!NOTE]
21+
>
22+
> The playground is in limited public beta and subject to change. To request access, [join the waitlist](https://github.com/marketplace/models/waitlist).
23+
24+
{% data variables.product.prodname_marketplace %} provides a free playground where you can adjust model parameters and submit prompts to see how the model responds.
25+
26+
To open the playground, go to [{% data variables.product.prodname_marketplace %}](https://github.com/marketplace/models), then click {% octicon "ai-model" aria-hidden="true" %} **Models** in the sidebar. Click on a model's name, then click {% octicon "command-palette" aria-hidden="true" %} **Playground**.
27+
28+
To adjust parameters for the model, select the **Parameters** tab in the sidebar. To see code that corresponds to the parameters that you selected, switch from the **Chat** tab to the **Code** tab.
29+
30+
The playground is rate limited. See [Rate limits](#rate-limits) below.
31+
32+
## Experimenting with AI models using the API
33+
34+
>[!NOTE]
35+
>
36+
> The free API usage is in limited public beta and subject to change. To request access, [join the waitlist](https://github.com/marketplace/models/waitlist).
37+
38+
{% data variables.product.company_short %} provides free API usage so that you can experiment with AI models in your own application.
39+
40+
To learn how to use a model in your application, go to [{% data variables.product.prodname_marketplace %}](https://github.com/marketplace/models), then click {% octicon "ai-model" aria-hidden="true" %} **Models** in the sidebar. Click on a model's name, then click {% octicon "code" aria-hidden="true" %} **Code**.
41+
42+
The steps to use each model are similar. In general, you will need to:
43+
44+
1. Optionally, use the language dropdown to select the programming language.
45+
1. Optionally, use the SDK dropdown to select which SDK to use.
46+
47+
All models can be used with the Azure AI Inference SDK, and some models support additional SDKs. If you want to easily switch between models, you should select "Azure AI Inference SDK". If you selected "REST" as the language, you won't use an SDK. Instead, you will use the API endpoint directly.
48+
1. Either open a codespace, or set up your local environment:
49+
* To run in a codespace, click {% octicon "codespaces" aria-hidden="true" %} **Run codespace**, then click **Create new codespace**.
50+
* To run locally:
51+
* Create a {% data variables.product.company_short %} {% data variables.product.pat_generic %}. The token should not have any scopes or permissions. See "[AUTOTITLE](/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)."
52+
* Save your token as an environment variable.
53+
* Install the dependencies for the SDK, if required.
54+
1. Use the example code to make a request to the model.
55+
56+
The free API usage is rate limited. See [Rate limits](#rate-limits) below.
57+
58+
## Going to production
59+
60+
The rate limits for the playground and free API usage are intended to help you experiment with models and develop your AI application. Once you are ready to bring your application to production, you can use a token from a paid Azure account instead of your {% data variables.product.company_short %} {% data variables.product.pat_generic %}. You don't need to change anything else in your code. For more information, see the [Azure AI](https://ai.azure.com/github/model/docs) documentation.
61+
62+
## Rate limits
63+
64+
The playground and free API usage are rate limited by requests per minute, requests per day, tokens per request, and concurrent requests. If you get rate limited, you will need to wait for the rate limit that you hit to reset before you can make more requests.
65+
66+
Low, high, and embedding models have different rate limits. To see which type of model you are using, refer to the model's information in {% data variables.product.prodname_marketplace %}.
67+
68+
<table>
69+
<tr>
70+
<th scope="col" style="width:15%"><b>Rate limit tier</b></th>
71+
<th scope="col" style="width:25%"><b>Rate limits</b></th>
72+
<th scope="col" style="width:20%"><b>Free and Copilot Individual</b></th>
73+
<th scope="col" style="width:20%"><b>Copilot Business</b></th>
74+
<th scope="col" style="width:20%"><b>Copilot Enterprise</b></th>
75+
</tr>
76+
<tr>
77+
<th rowspan="4" scope="rowgroup"><b>Low</b></th>
78+
<th style="padding-left: 0"><b>Requests per minute</b></th>
79+
<td>15</td>
80+
<td>15</td>
81+
<td>20</td>
82+
</tr>
83+
<tr>
84+
<th><b>Requests per day</b></th>
85+
<td>150</td>
86+
<td>300</td>
87+
<td>450</td>
88+
</tr>
89+
<tr>
90+
<th><b>Tokens per request</b></th>
91+
<td>8000 in, 4000 out</td>
92+
<td>8000 in, 4000 out</td>
93+
<td>8000 in, 8000 out</td>
94+
</tr>
95+
<tr>
96+
<th><b>Concurrent requests</b></th>
97+
<td>5</td>
98+
<td>5</td>
99+
<td>8</td>
100+
</tr>
101+
<tr>
102+
<th rowspan="4" scope="rowgroup"><b>High</b></th>
103+
<th style="padding-left: 0"><b>Requests per minute</b></th>
104+
<td>10</td>
105+
<td>10</td>
106+
<td>15</td>
107+
</tr>
108+
<tr>
109+
<th><b>Requests per day</b></th>
110+
<td>50</td>
111+
<td>100</td>
112+
<td>150</td>
113+
</tr>
114+
<tr>
115+
<th><b>Tokens per request</b></th>
116+
<td>8000 in, 4000 out</td>
117+
<td>8000 in, 4000 out</td>
118+
<td>16000 in, 8000 out</td>
119+
</tr>
120+
<tr>
121+
<th><b>Concurrent requests</b></th>
122+
<td>2</td>
123+
<td>2</td>
124+
<td>4</td>
125+
</tr>
126+
<tr>
127+
<th rowspan="4" scope="rowgroup" style="box-shadow: none"><b>Embedding</b></th>
128+
<th style="padding-left: 0"><b>Requests per minute</b></th>
129+
<td>15</td>
130+
<td>15</td>
131+
<td>20</td>
132+
</tr>
133+
<tr>
134+
<th><b>Requests per day</b></th>
135+
<td>150</td>
136+
<td>300</td>
137+
<td>450</td>
138+
</tr>
139+
<tr>
140+
<th><b>Tokens per request</b></th>
141+
<td>64000</td>
142+
<td>64000</td>
143+
<td>64000</td>
144+
</tr>
145+
<tr>
146+
<th><b>Concurrent requests</b></th>
147+
<td>5</td>
148+
<td>5</td>
149+
<td>8</td>
150+
</tr>
151+
</table>
152+
153+
These limits are subject to change without notice.

‎content/github-models/responsible-use-of-github-models.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
title: Responsible use of GitHub Models
3+
shortTitle: Responsible use
4+
intro: 'Learn how to use {% data variables.product.prodname_github_models %} responsibly by understanding its purposes, capabilities, and limitations.'
5+
versions:
6+
feature: github-models
7+
type: rai
8+
---
9+
10+
With {% data variables.product.prodname_github_models %}, you build your understanding of AI model capabilities by experimenting with model settings and sending prompts through a chat interface. Additionally, you can directly interact with models through an SDK. Refer to a model's "Getting Started" tab for more information about how to use the SDK. Refer to a model’s "README" tab for more information on the model. Remember when interacting with a model you are experimenting with AI, so content mistakes are possible.
11+
12+
{% data variables.product.prodname_github_models %} is designed to allow for learning, experimentation and proof-of-concept activities. The feature is subject to various limits (including requests per minute, requests per day, tokens per request, and concurrent requests) and is not designed for production use cases. {% data variables.product.prodname_github_models %} employs a number of [content filters](https://azure.microsoft.com/en-us/products/ai-services/ai-content-safety). These filters cannot be turned off as part of the {% data variables.product.prodname_github_models %} experience. If you decide to employ models through [Azure AI](https://ai.azure.com/github/model/docs) or a paid service, please configure your content filters to meet your requirements.

‎content/index.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@ children:
8282
- support
8383
- video-transcripts
8484
- contributing
85+
- github-models
8586
childGroups:
8687
- name: Get started
8788
octicon: RocketIcon
@@ -133,6 +134,7 @@ childGroups:
133134
- rest
134135
- graphql
135136
- webhooks
137+
- github-models
136138
- name: Enterprise and Teams
137139
octicon: OrganizationIcon
138140
children:
@@ -169,4 +171,3 @@ externalProducts:
169171
href: 'https://docs.npmjs.com/'
170172
external: true
171173
---
172-

‎data/features/github-models.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# GitHub Models (AI models on GitHub Marketplace)
2+
versions:
3+
fpt: '*'
4+
ghec: '*'
5+
ghes: '*'

‎data/reusables/copilot/differences-cfi-cfb-table.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,11 @@
1515
| {% data variables.product.prodname_copilot_chat_short %} skills in IDEs[^3] | {% octicon "x" aria-label="Not included" %} | {% octicon "x" aria-label="Not included" %} | {% octicon "check" aria-label="Included" %} |
1616
| {% data variables.product.prodname_copilot_for_prs %} | {% octicon "x" aria-label="Not included" %} | {% octicon "x" aria-label="Not included" %} | {% octicon "check" aria-label="Included" %} |
1717
| {% data variables.product.prodname_copilot_short %} knowledge bases | {% octicon "x" aria-label="Not included" %} | {% octicon "x" aria-label="Not included" %} | {% octicon "check" aria-label="Included" %} |
18+
| Increased {% data variables.product.prodname_github_models %} rate limits[^4] | {% octicon "x" aria-label="Not included" %} | {% octicon "check" aria-label="Included" %} | {% octicon "check" aria-label="Included" %} |
1819

1920
{% endrowheaders %}
2021

2122
[^1]: Code completion in IDEs is available in {% data variables.product.prodname_vscode %}, {% data variables.product.prodname_vs %}, JetBrains IDEs, Azure Data Studio, and Vim/Neovim.
2223
[^2]: {% data variables.product.prodname_copilot_chat_short %} in IDEs is available in {% data variables.product.prodname_vscode %}, {% data variables.product.prodname_vs %}, and JetBrains IDEs.
2324
[^3]: {% data variables.product.prodname_copilot_chat_short %} skills in IDEs is available in {% data variables.product.prodname_vscode %} and {% data variables.product.prodname_vs %}.
25+
[^4]: For details about the increased rate limits, see "[AUTOTITLE](/github-models/prototyping-with-ai-models)."

‎data/variables/product.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,7 @@ prodname_github_app: 'GitHub App'
127127
prodname_github_apps: 'GitHub Apps'
128128
prodname_oauth_app: 'OAuth app'
129129
prodname_oauth_apps: 'OAuth apps'
130+
prodname_github_models: 'GitHub Models'
130131

131132
# API and developer docs
132133
prodname_enterprise_api: 'GitHub Enterprise Server APIs'

0 commit comments

Comments
 (0)