Replies: 6 comments 6 replies
-
|
such a great intiative |
Beta Was this translation helpful? Give feedback.
-
|
I know this is a pretty ambitious idea and not trivial to implement, but it would be really powerful to have an AI-detection mechanism with a configurable threshold at the repository or organization level. That way, teams could decide what percentage of AI-generated code is acceptable in pull requests. Another possible approach would be to define a set of rules or prompts and evaluate pull requests against them. PRs that don’t meet those rules could be automatically flagged or potentially even closed. |
Beta Was this translation helpful? Give feedback.
-
This is definitely something we’re exploring. One idea is to leverage a repository’s CONTRIBUTING.md file as a source of truth for project guidelines and then validate PRs against any defined rules. In regards to AI-generated code, have you seen cases where the code is AI-generated but still high-quality and actually solves the problem? Or is it always just something you want to close out immediately? I'm curious if an AI-detection mechanism would rule out PRs where AI is used constructively, but interested in investigating this more and understanding what sensible would thresholds look like. |
Beta Was this translation helpful? Give feedback.
This comment was marked as spam.
This comment was marked as spam.
-
|
As of today, I would say that 1 out of 10 PRs created with AI is legitimate and meets the standards required to open that PR.
On 28 Jan 2026, at 18:41, Camilla Moraes ***@***.***> wrote:
Another possible approach would be to define a set of rules or prompts and evaluate pull requests against them. PRs that don’t meet those rules could be automatically flagged or potentially even closed.
This is definitely something we’re exploring. One idea is to leverage a repository’s CONTRIBUTING.md file as a source of truth for project guidelines and then validate PRs against any defined rules.
In regards to AI-generated code, have you seen cases where the code is AI-generated but still high-quality and genuinely solves the problem? Or is it alwaays just something you want to close out immediately? I'm curious because I'm wondering if an AI-detection mechanism would rule out PRs where AI is used constructively, but that's where we'd want to test this thoroughly and understand what sensible thresholds look like.
—
Reply to this email directly, view it on GitHub<#185387 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABBWEYEKF6WLNDKE376L3GD4JDYFXAVCNFSM6AAAAACS7B7C7OVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKNRTGEZTMMI>.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as off-topic.
This comment was marked as off-topic.
-
PLEASE make this better than disabling issues: Imparticular, do not restrict access to previously opened PRs. Fine to hide the UI or even the ability to list them, but let people access them with a direct link. I'd also appreciate if that was done in repos where issues got disabled. Context: Someone realizing that they no longer want to allow issues (or PRs) should NOT make any existing content disappear for good. Especially in cases where this content is referenced elsewhere and suddenly becomes a dead link...
Similar to my suggestion above, this should be limited in time. It's amazing to delete spam PRs. Hacktoberfest-cheating, Indian-Youtube-nonsense, and the likes. But if there is a controversy around a PR, which has happened numerous times before e.g. due to highly unpopular decisions by repo maintainers (Minio comes to my mind), then they should NOT be able to permanently delete at PR just because it's inconvenient or shows their true face. My suggestion would be to have a (very) limited timeframe to delete a PR. Maybe a week in case of low activity (e.g. a spam PR the maintainers did not see), much less in case of high activity. Having an exception for "quiet" PRs (no meaningful amount of comments) and allowing to delete those for longer would be perfectly fine of course so someone who rarely checks their repo can delete spam even when they notice it late. PS: Yes, I'm aware that deleting individual comments on PRs and issues is already possible. But that's a hassle compared to being able to nuke the whole thing with two clicks. |
Beta Was this translation helpful? Give feedback.
-
Would you want to be able to navigate to the PR just for viewing purposes or to potentially keep making changes to it? I agree it makes sense for users to still be able to view their PRs - will see what we can do here.
Thoughts on having this be an org-level or repo-level setting so admins can determine when deletion can be done? I know there are also certain repos/orgs where PRs can't be deleted for legal reasons. |
Beta Was this translation helpful? Give feedback.
-
|
I think just being able to view would be more than enough. For private repos I would not limit this at all by default, or make it configurable. My comment was mostly about public repos where I don't think the maintainer should have the ability to make a PR and its associated discussion simply disappear for good, e.g. because they do not like how that discussion went. |
Beta Was this translation helpful? Give feedback.
-
|
Hey! I am from Azure Core Upstream and we have a lot of OSS maintainers who mainly maintain repositories on GitHub. We held an internal session to talk about copilot and there is a discussion on the topic where maintainers feel caught between today’s required review rigor (line-by-line understanding for anything shipped) and a future where agentic / AI-generated code makes that model increasingly unsustainable. below are some key maintainer's pain points:
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks for sharing @Mossaka ! Great to get more feedback. This looks along the lines of what we've been hearing from maintainers too, so validates what some of the key pain points are. Were there any feature requests mentioned during the session? |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hey everyone,
I wanted to provide an update on a critical issue affecting the open source community: the increasing volume of low-quality contributions that is creating significant operational challenges for maintainers.
We’ve been hearing from you that you’re dedicating substantial time to reviewing contributions that do not meet project quality standards for a number of reasons - they fail to follow project guidelines, are frequently abandoned shortly after submission, and are often AI-generated. As AI continues to reshape software development workflows and the nature of open source collaboration, I want you to know that we are actively investigating this problem and developing both immediate and longer-term strategic solutions.
What we're exploring
We’ve spent time reviewing feedback from community members, working directly with maintainers to explore various solutions, and looking through open source repositories to understand the nature of these contributions. Below is an overview of the solutions we’re currently evaluating.
Short-term solutions:
Long-term direction:
As AI adoption accelerates, we recognize the need to proactively address how it can potentially transform both contributor and maintainer workflows. We are exploring:
Next Steps
These are some starting points, and we’re continuing to explore both immediate improvements and long-term solutions. Please share your feedback, questions, or concerns in this thread. Your input is crucial to making sure we’re building the right things and tackling this challenge effectively. As always, thank you for being part of this conversation. Looking forward to hearing your thoughts and working together to address this problem.
Beta Was this translation helpful? Give feedback.
All reactions