Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@soapy1
Copy link
Contributor

@soapy1 soapy1 commented May 23, 2025

Description

This PR:

  • adds a cli arg --env-spec/--environment-specifier to conda env commands to users can explicitly select the environment spec plugin they want to use when creating/updating an environment using an environment file
  • removes references to the function env.specs.detect
  • does not substitue improving the environment spec plugin detection process

This PR suggests that environment spec plugin selection works as follows:

  • If no plugin is selected, conda should try to detect what environment spec plugin should be used against the file
  • If the user requests a specific plugin with the --env-spec cli arg or env_spec config parameter, conda will try to use the plugin with that name
    • if a environment spec plugin with that name is not registered, return an error
    • if the requested environment spec plugin can not handle the input file, return an error

Here are some examples of this change:

The environment spec plugin is automatically detected if no argument is provided

$ conda env create --file tests/env/support/simple.yml
Channels:
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done
#
# To activate this environment, use
#
#     $ conda activate nlp
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Return an error to the user if an unregistered plugin is requested

$ conda env create --file tests/env/support/simple.yml --env-spec idontextis

usage: conda env create [-h] [-f FILE] [-n ENVIRONMENT | -p PATH] [-C] [-k] [--offline]
                        [--env-spec-plugin {binstar,environment.yml,requirements.txt}] [--no-default-packages] [--json]
                        [--console CONSOLE] [-v] [-q] [-d] [-y] [--solver {classic,libmamba}] [--subdir SUBDIR]
                        [remote_definition]
conda env create: error: argument --env-spec-plugin: invalid choice: 'idontextis' (choose from 'binstar', 'environment.yml', 'requirements.txt')

Return an error to the user if a plugin that won't work is requested

$ conda env create --file tests/env/support/simple.yml --env-spec binstar

PluginError: Requested plugin 'binstar' is unable to handle environment spec 'tests/env/support/simple.yml'

Set the environment spec using env vars

$ CONDA_ENV_SPEC=idontexist conda env create --file tests/env/support/simple.yml

CondaValueError: You have chosen an unrecognized environment specifier type (idontexist). Choose one of: binstar, environment.yml, requirements.txt

Checklist - did you ...

  • Add a file to the news directory (using the template) for the next release's release notes?
  • Add / update necessary tests?
  • Add / update outdated documentation?

@github-project-automation github-project-automation bot moved this to 🆕 New in 🔎 Review May 23, 2025
@conda-bot conda-bot added the cla-signed [bot] added once the contributor has signed the CLA label May 23, 2025
@codspeed-hq
Copy link

codspeed-hq bot commented May 23, 2025

CodSpeed Performance Report

Merging #14877 will not alter performance

Comparing soapy1:explicitly-selected-env_spec-plugin (226e33d) with main (ba01096)

Summary

✅ 21 untouched benchmarks

@soapy1 soapy1 force-pushed the explicitly-selected-env_spec-plugin branch from 4b83356 to a979783 Compare May 23, 2025 22:13
@soapy1
Copy link
Contributor Author

soapy1 commented May 26, 2025

pre-commit.ci autofix

@soapy1 soapy1 marked this pull request as ready for review May 26, 2025 17:45
@soapy1 soapy1 requested a review from a team as a code owner May 26, 2025 17:45
Copy link
Member

@jezdez jezdez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs some regiggering of words I think, we should absolutely not call out the existence of "plugins" in the cli or config as that is not consistent how we've mapped plugin provided functionality to the user interface.

@github-project-automation github-project-automation bot moved this from 🆕 New to 🏗️ In Progress in 🔎 Review May 27, 2025
@soapy1 soapy1 force-pushed the explicitly-selected-env_spec-plugin branch 6 times, most recently from 5213276 to 1f933f0 Compare May 27, 2025 22:28
"25.9",
"26.3",
addendum="Use conda.base.context.plugin_manager.get_environment_specifiers.",
addendum="Use conda.base.context.plugin_manager.detect_environment_specifier.",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's maybe spicy to be changing this deprecation warning here? Not sure how conda's process for this is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, hmm, I'm not sure. @kenodegard did we have a case like this before? Could we keep a get_environment_specifiers stub around as an alias until this deprecation cycle is done?

@soapy1
Copy link
Contributor Author

soapy1 commented May 28, 2025

pre-commit.ci autofix

@soapy1 soapy1 force-pushed the explicitly-selected-env_spec-plugin branch 2 times, most recently from ae2f190 to e4f3543 Compare May 28, 2025 20:40
@soapy1 soapy1 requested a review from jezdez May 29, 2025 16:53
jezdez
jezdez previously approved these changes Jun 5, 2025
@github-project-automation github-project-automation bot moved this from 🏗️ In Progress to ✅ Approved in 🔎 Review Jun 5, 2025
@soapy1 soapy1 force-pushed the explicitly-selected-env_spec-plugin branch from 7280477 to bffd540 Compare June 5, 2025 18:09
@soapy1
Copy link
Contributor Author

soapy1 commented Jun 5, 2025

pre-commit.ci autofix

# prevent modifications to envs marked with conda-meta/frozen
),
"Plugin Configuration": ("no_plugins",),
"Environment Managment Configuration": ("environment_specifier",),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"Environment Managment Configuration": ("environment_specifier",),
"Environment Management Configuration": ("environment_specifier",),

Hm, do we need a new section for this? If it's experimental, we should maybe put it under "undocumented" for now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this config field really fits in any of the other categories. Open to suggestions.

I don't think this should go in the undocumented section. But maybe could add an "experimental" section?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I'm in favor of an experimental section!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Me too, let's move it to experimental.

p.add_argument(
"--environment-specifier",
"--env-spec", # for brevity
choices=context.plugin_manager.get_environment_specifiers(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we depend on context bis at CLI initialization? My understanding is that the context needs the parsed CLI args to finish initialization, so I'm not sure we have discovered anything about plugins yet here 🤔 Would we need to validate later?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Erm, ya this is not great, I'd say. This happens other places in the code, for example with the --solver cli argument https://github.com/conda/conda/blob/main/conda/cli/helpers.py#L423.

My understanding is that the context needs the parsed CLI args to finish initialization,

yes, that'll be for slurping up all the config from all the different places it comes from. I don't think this is changing anything for plugin loading though. Like, all the plugins get discovered when the plugin manager gets initialized with plugin.manager::get_plugin_manager.

So, I think setting choices is an ok thing to do here. It is weird that configuration and other things that are expected to be globally accessible (like the plugin manager) and shoved into the context object.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a fair point, I've used something like this before, should maybe used for the solver option as well

import argparse

class LazyChoicesAction(argparse.Action):
    def __init__(self, option_strings, dest, choices_func, **kwargs):
        self.choices_func = choices_func
        super().__init__(option_strings, dest, **kwargs)
    
    def __call__(self, parser, namespace, values, option_string=None):
        valid_choices = self.choices_func()
        if values not in valid_choices:
            raise argparse.ArgumentTypeError(
                f"invalid choice: {values!r} (choose from {valid_choices})"
            )
        setattr(namespace, self.dest, values)
Suggested change
choices=context.plugin_manager.get_environment_specifiers(),
action=LazyChoicesAction,
choices_func=context.plugin_manager.get_environment_specifiers,

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I had missed we do that with the solver already. I think we can apply Jannis' suggestion and fix the solver bits in a separate PR to aid with traceability.

Copy link
Member

@jezdez jezdez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lazy population of the choices is good to fix before merge, other than that this looks great

# prevent modifications to envs marked with conda-meta/frozen
),
"Plugin Configuration": ("no_plugins",),
"Environment Managment Configuration": ("environment_specifier",),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I'm in favor of an experimental section!

p.add_argument(
"--environment-specifier",
"--env-spec", # for brevity
choices=context.plugin_manager.get_environment_specifiers(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a fair point, I've used something like this before, should maybe used for the solver option as well

import argparse

class LazyChoicesAction(argparse.Action):
    def __init__(self, option_strings, dest, choices_func, **kwargs):
        self.choices_func = choices_func
        super().__init__(option_strings, dest, **kwargs)
    
    def __call__(self, parser, namespace, values, option_string=None):
        valid_choices = self.choices_func()
        if values not in valid_choices:
            raise argparse.ArgumentTypeError(
                f"invalid choice: {values!r} (choose from {valid_choices})"
            )
        setattr(namespace, self.dest, values)
Suggested change
choices=context.plugin_manager.get_environment_specifiers(),
action=LazyChoicesAction,
choices_func=context.plugin_manager.get_environment_specifiers,

name=filename, plugin_names=[hook.name for hook in hooks]
)

def get_environment_specifier(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this

jaimergp
jaimergp previously approved these changes Jun 10, 2025
Copy link
Contributor

@jaimergp jaimergp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with leaving choices= as is for now provided we check the Lazy* approach @jezdez suggested in a new PR. Let's move the documentation bits to a new Experimental section and merge here!

@soapy1
Copy link
Contributor Author

soapy1 commented Jun 10, 2025

Pushed up the category map change. Follow up PR with the Lazy approch coming soon

@jezdez jezdez merged commit 72d03c1 into conda:main Jun 11, 2025
75 checks passed
@github-project-automation github-project-automation bot moved this from ✅ Approved to 🏁 Done in 🔎 Review Jun 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed [bot] added once the contributor has signed the CLA

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

4 participants