Adding Third-Party Packages as Package Extras - #118
Conversation
|
Great work! Does "Package Extras" refer to optional dependencies or something more conceptual? |
Thanks! Yes, it refers to optional dependencies in pyproject.toml so people can run pip install django[abc]. |
tim-schilling
left a comment
There was a problem hiding this comment.
Thank you for the work on this @prafulgulani! I've got some questions around things we may want to include and suggestions on where things should live in the document. I think you're headed in the right direction defining the limits and requirements for packages. That will communicate things clearly which is always helpful 😊
| visibility for the users and also acts as encouragement for the developers. | ||
|
|
||
|
|
||
| Guidelines for third party packages |
There was a problem hiding this comment.
This is likely better off in the specification section.
|
|
||
| To safely list package extras without compromising the stability and performance | ||
| of Django we need to have a test suite that helps with isolation and bridging | ||
| of the packages. |
There was a problem hiding this comment.
Should we be running smoke tests with the extras in the Django CI to confirm when a problem may crop up based on a change to core?
| before being configured in ``settings.py`` | ||
| * Poorly specified or circular third-party dependencies can break installations | ||
| across package managers. | ||
| * Unconfigured extras executing early imports might lead to performance degradations. |
There was a problem hiding this comment.
I think a fair amount in the current specification section probably belongs in rationale. Or maybe I'm misunderstanding the purpose here?
Specifically, these two sentences make partial claims without evidence or examples which makes it hard for others to know what they should avoid explicitly. I feel like the specification section should be pretty clear on how extras are decided, implemented and maintained.
There was a problem hiding this comment.
Yes, it looks mixed as of now, what I had in mind was this section along with examples would serve as what should be avoided and I ended up writing these in specification.
I'll restructure both the sections accordingly.
| * The package must use a license compatible with Django's BSD-3 clause license. | ||
| * Top-level ``__init__.py`` files must not perform eager network calls, heavy | ||
| IO, or database access. | ||
| * The package must follow a deprecation schedule aligned with Django's releases. |
There was a problem hiding this comment.
I'd like to see more about what is an aligned release schedule.
There was a problem hiding this comment.
It's not necessary for a package to move at the same pace as Django. They might release more or fewer versions than Django does, but irrespective of their release cycle, they should maintain compatibility with older Django releases for as long as Django supports those versions. When Django deprecates an API the package depends on, the package shouldn't instantly drop support for it.
And they should also test against Django's alpha/beta/RC releases and ship a compatible release within a reasonable timeframe, like 30 days or so.
There was a problem hiding this comment.
It's not necessary for a package to move at the same pace as Django.
...
And they should also test against Django's alpha/beta/RC releases and ship a compatible release within a reasonable timeframe, like 30 days or so.
Yeah, this is the discussion I was hoping to extract out. I feel like a month is fast for OSS, but we could always loosen this too if it is problematic.
| * Packages must use lower-bound constraints (``package>=X.Y``) rather than exact | ||
| versions (``package==X.Y.Z``) for declaring Django dependency. | ||
| * Third-party packages that rely on private Django APIs must run CI matrix | ||
| checks against ``django/main``. |
There was a problem hiding this comment.
Probably worth us requiring that the tests should run weekly as well as on PR/push, just in case their development cycle is slower than Django's. Especially if we don't have a smoke test in core for the packages.
| =================================== | ||
|
|
||
| * The package must have a documented security policy and a responsive process | ||
| for reporting vulnerabilities. |
There was a problem hiding this comment.
Do we need anything about governance for the projects? A project like Mongo has a company behind it which is much different scenario than a project that has a single developer with the only commit bit.
There was a problem hiding this comment.
We definitely need to define governance criteria so that we can avoid removing package as much as possible.
Few requirements we can consider could be:
- We can mandate at least 2 maintainers.
- We can check if they support every stable version of Django.
- We can look at release track record to check for timely release for new versions of Django.
- We need to look at how useful a particular package is in order to avoid adding a lot of packages.
what else do you think we should look for in a package before we add them?
There was a problem hiding this comment.
I think there are individuals who have been maintaining packages for 5+ years that I would be fine adding as an extra. Not that the package is 5 years old, but that they've been a contributor for that long. I think we could carve out exceptions where needed. Otherwise, agreed on the multiple maintainers bit. Or some other proof of governance that shows it's unlikely to disappear.
Agreed on 2 and 3 entirely.
Regarding 4, we can use is @adamchainz well-maintained test.
I've also been working on a project to allow us to gather this type of metric based on several conversations. Not that it solves it now, but we could use this as a objective way to see what third-party packages are actually in use by the community.
| Backwards Compatibility | ||
| ======================= | ||
|
|
||
| This DEP doesn't introduce any backwards incompatibilities. It does not modify |
There was a problem hiding this comment.
Do we need to note that these extras are only available for certain versions of Django?
There was a problem hiding this comment.
I didn't get this! Do you mean that extras will be available from the versions that come after this DEP is accepted? or something else?
There was a problem hiding this comment.
If we have django[db-vendor] has been working for up to 2030 and then we have to remove it, is there anything we should consider up front now? For example, do we have a special deprecation process for removing an extra? Now that I'm writing it out, it seems like this concern should be handled elsewhere and not in the Backwards Compatibility section.
| To safely list package extras without compromising the stability and performance | ||
| of Django we need to have a test suite that helps with isolation and bridging | ||
| of the packages. | ||
|
|
There was a problem hiding this comment.
Should we include any information on why an extra may be removed?
There was a problem hiding this comment.
Yes, as much as we want to avoid removing them, we might need to remove a package someday, so a section outlining possible reasons would be an excellent addition.
|
Hi @prafulgulani and @nanorepublica, what about reducing scope on this to just those packages that we already define in the docs, such as psycopg and allowing for the option for the board to have a corporate sponsorship grant an extra? This isn't coming from the SC, but from me as a community member. I was in the DSF office hours meeting this week and it was brought up that having the ability to have an extra may be appealing to a theoretical large corporate sponsor. Reducing the scope would allow for us to get some extras out into the wild sooner and learn from that experience rather than trying to get it right for all cases immediately. I think this would table my concern about governance to a later date. It'd potentially remove a lot of the concerns and text around third-party packages being extensions of Django, simplifying the DEP. |
This DEP proposes adding third-party packages as package extras.
References: