{"id":73497,"date":"2023-08-03T09:01:05","date_gmt":"2023-08-03T16:01:05","guid":{"rendered":"https:\/\/github.blog\/?p=73497"},"modified":"2024-02-07T18:06:16","modified_gmt":"2024-02-08T02:06:16","slug":"introducing-code-referencing-for-github-copilot","status":"publish","type":"post","link":"https:\/\/github.blog\/news-insights\/product-news\/introducing-code-referencing-for-github-copilot\/","title":{"rendered":"Introducing code referencing for GitHub Copilot"},"content":{"rendered":"

Make more informed decisions about the code you use. In the rare case where a GitHub Copilot suggestion matches public code, this update will show a list of repositories where that code appears\u00a0and their licenses. Sign up for the private beta today<\/a>.<\/p><\/blockquote>\n

Over the course of the last year, GitHub Copilot, the world\u2019s first at-scale AI pair programmer trained on billions of lines of public code<\/a>, has attracted more than 1 million developers and helped over 27,000 organizations build faster and more productively. During that time, many developers told us they want to see when GitHub Copilot\u2019s suggestions match public code.<\/p>\n

Today, we\u2019re announcing a private beta of GitHub Copilot with code referencing<\/a> that includes an updated filter which detects and shows context of code suggestions matching public code on GitHub. When the filter is enabled, GitHub Copilot checks code suggestions with surrounding code of about 150 characters and compares it against an index of all the public code on GitHub.com. Matches\u2014along with information about every repository in which they appear\u2014are displayed right in the editor. Developers can now choose whether to block suggestions containing matching code, or allow those suggestions with information about matches.<\/p>\n

Why? Some want to learn from others\u2019 work, others may want to take a dependency rather than introduce new app logic, and still others want to give or receive credit for similar work. Whatever the reason, it\u2019s nice to know when similar code is out there.<\/p>\n

Let\u2019s see how it works.<\/p>\n

How GitHub Copilot code referencing works<\/span><\/a><\/h2>\n

With billions of files to index and a latency budget of only 10-20ms, it\u2019s a miracle of engineering that this is even possible. Still, if there\u2019s a match, a notification appears in the editor showing: (1) the matching code, (2) the repositories where that code appears, and (3) the license governing each repository.<\/p>\n

https:\/\/github.blog\/wp-content\/uploads\/2023\/08\/16-9-Blog-Demo.mp4#t=0.001<\/a><\/video><\/div>\n

Why code referencing matters<\/span><\/a><\/h2>\n

In our journey to create a code referencing tool, we discovered a few interesting things:<\/p>\n

First, our previous research<\/a> suggests that matches occur in less than one percent of GitHub Copilot suggestions. But that one percent isn’t evenly distributed across all use cases. In the context of an existing application with surrounding code, we almost never see a match. But in an empty or nearly empty file, we see matches far more often.<\/p>\n

Suggestions are heavily biased toward the prompt so GitHub Copilot can provide suggestions tailor-made for your current task. That means, in an existing app with lots of context, you’ll get a suggestion customized for your code. But in an empty, or nearly empty file, there’s little to no context. So, you’re more likely to get a suggestion that matches public code.<\/p>\n

We’ve also found that when suggestions match public code, those matches frequently appear in dozens, if not hundreds of repositories. In some ways, this isn’t surprising because the models that power GitHub Copilot are akin to giant probability machines. A code fragment that appears in many repositories is more likely to be a \u201cpattern\u201d detected by the model\u2014similar to the patterns we see elsewhere in public code.<\/p>\n

For example, research on Java projects<\/a> finds that up to 11% of repositories may contain code that resembles solutions posted to Stack Overflow, and the vast majority of those snippets appear without attribution. Another study on Python<\/a> found that many matches are too generic to trace to an original usage. A smaller-scale study<\/a> found that Stack Overflow answers contain code from Android applications.<\/p>\n

Finally, the repositories with matching code are often governed by multiple, sometimes conflicting licenses, which makes attributing a match to its source more challenging. By consulting a list of references, developers can now:<\/p>\n