Codestin Search App

edgarrmondragon · 2026-05-21T23:58:30Z

Description

Variables set in a project's .env (e.g. AWS_PROFILE, AZURE_TENANT_ID, GOOGLE_APPLICATION_CREDENTIALS) were invisible to cloud SDK clients (boto3, azure-storage-blob, google-cloud-storage) running inside the Meltano process.

Meltano reads .env via dotenv_values() into an internal dict that is passed to plugin subprocesses as part of their environment, but the values were never injected into os.environ. Cloud SDK clients read os.environ directly, so they never saw those values. The only workaround was to set the variable in the shell before invoking Meltano (e.g. AWS_PROFILE=arch-prod meltano state get ...).

Fix:

Call load_dotenv(override=False) in the CLI entry point right after the project is located. This injects any .env variable not already set in the shell into os.environ, so SDK clients pick it up. Shell variables always take precedence (override=False), preserving the existing priority order.
EnvStoreManager.env now subtracts project.dotenv_env keys before returning. This ensures that a variable in .env is still reported as coming from .env (not "the environment") in meltano config get, and that AutoStoreManager write-store decisions are unaffected.

Checklist

I have read the contribution guide

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 following the agentic coding guidelines

Related Issues

Closes feature: .env values not visible to in-process SDK clients (e.g. AWS_PROFILE for S3 state backend) #10037

Summary by Sourcery

Load project .env variables into the Meltano CLI process environment while preserving correct configuration source reporting between environment and .env stores.

Bug Fixes:

Ensure variables defined in a project .env file are visible to in-process SDK clients by loading them into os.environ when the project is located.
Maintain accurate attribution of settings to .env rather than the environment by filtering .env-sourced keys out of the environment settings store.

Enhancements:

Tighten type annotations and align SQLAlchemy exception handling with the sqlalchemy.exc.NoResultFound import.

…ect env vars Variables set in a project's `.env` (e.g. `AWS_PROFILE`, `AZURE_TENANT_ID`, `GOOGLE_APPLICATION_CREDENTIALS`) were invisible to cloud SDK clients (boto3, azure-storage-blob, google-cloud-storage) running inside the Meltano process. Meltano read `.env` via `dotenv_values()` into an internal dict that is passed to plugin *subprocesses*, but never injected into `os.environ`. SDK clients read `os.environ` directly, so they never saw these values. Fix: call `load_dotenv(override=False)` in the CLI entry point right after the project is located, so any `.env` variable that is not already set in the shell gets injected into `os.environ`. Shell variables always take precedence (`override=False`), preserving the existing priority order. To keep Meltano's own settings-store source attribution correct, `EnvStoreManager` now subtracts `project.dotenv_env` keys from the env dict it exposes. This ensures that a variable in `.env` is still reported as coming from `.env` (not "the environment") in `meltano config get`, and that `AutoStoreManager` write decisions are unaffected. Closes #10037 Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

netlify · 2026-05-21T23:58:37Z

✅ Deploy Preview for meltano canceled.

Name	Link
🔨 Latest commit	`f9dba05`
🔍 Latest deploy log	https://app.netlify.com/projects/meltano/deploys/6a0f9c293c180d0008cbd1d5

sourcery-ai · 2026-05-21T23:58:49Z

Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Ensures project .env variables are both available to in-process SDK clients via os.environ and still correctly attributed to .env in Meltano’s settings layer, while fixing a NoResultFound import.

Sequence diagram for CLI loading .env into os.environ

sequenceDiagram
    actor User
    participant MeltanoCLI as MeltanoCLI
    participant Project as Project
    participant DotEnv as load_dotenv

    User ->> MeltanoCLI: cli(env_file)
    MeltanoCLI ->> Project: find(dotenv_file)
    Project -->> MeltanoCLI: project
    MeltanoCLI ->> DotEnv: load_dotenv(dotenv_path=project.dotenv, override=False)
    MeltanoCLI ->> MeltanoCLI: setup_logging(project)

File-Level Changes

Change	Details	Files
Inject project .env variables into os.environ in the CLI entrypoint so in-process SDK clients can read them while preserving shell precedence.	Call load_dotenv immediately after Project.find using the project’s .env path. Use override=False so existing shell environment variables take precedence over .env values.	`src/meltano/cli/cli.py`
Adjust EnvStoreManager.env to hide .env-backed keys from the environment store so they remain attributed to .env, and tighten type hints.	Change BaseEnvStoreManager.env to return dict[str, str] and EnvStoreManager.env to match that signature. Filter EnvStoreManager.env to exclude keys present in project.dotenv_env, relying on settings_service.env as the source. Update the env property docstring to clarify the exclusion of .env-origin keys.	`src/meltano/core/settings_store.py`
Fix the exception type import used when a system database configuration key is missing.	Replace sqlalchemy.orm.exc.NoResultFound with sqlalchemy.exc.NoResultFound in the settings retrieval logic.	`src/meltano/core/settings_store.py`

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#10037	Make variables defined in a project's `.env` file available in the Meltano process environment (`os.environ`) so that in‑process SDK clients (e.g. boto3, Azure, GCS clients) can read them.	✅
#10037	Ensure that `.env` variables do not override variables already set in the shell environment when populating `os.environ`.	✅

Possibly linked issues

feature: .env values not visible to in-process SDK clients (e.g. AWS_PROFILE for S3 state backend) #10037: PR loads .env into os.environ and adjusts env handling so in-process SDK clients see project env vars.

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

codspeed-hq · 2026-05-22T00:02:12Z

Merging this PR will not alter performance

✅ 4 untouched benchmarks

_{Comparing fix/dotenv-env-for-in-process-sdk-clients (f9dba05) with main (7ae448a)}

codecov · 2026-05-22T00:05:47Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.68%. Comparing base (7ae448a) to head (f9dba05).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10038      +/-   ##
==========================================
- Coverage   96.74%   96.68%   -0.06%     
==========================================
  Files         286      286              
  Lines       25460    25463       +3     
  Branches     1451     1451              
==========================================
- Hits        24631    24619      -12     
- Misses        667      683      +16     
+ Partials      162      161       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

edgarrmondragon changed the title ~~fix(cli): Apply .env to os.environ so in-process SDK clients see project env vars~~ fix(cli): Apply .env to os.environ so in-process SDK clients see project env vars May 22, 2026

edgarrmondragon self-assigned this May 22, 2026

edgarrmondragon added the Settings Management of plugin and Meltano settings label May 22, 2026

edgarrmondragon added this to the v4.3 milestone May 22, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(cli): Apply `.env` to `os.environ` so in-process SDK clients see project env vars#10038

fix(cli): Apply `.env` to `os.environ` so in-process SDK clients see project env vars#10038
edgarrmondragon wants to merge 1 commit into
mainfrom
fix/dotenv-env-for-in-process-sdk-clients

edgarrmondragon commented May 21, 2026 •

edited by sourcery-ai Bot

Loading

Uh oh!

netlify Bot commented May 21, 2026 •

edited

Loading

Uh oh!

sourcery-ai Bot commented May 21, 2026

Reviewer's Guide

Sequence diagram for CLI loading .env into os.environ

File-Level Changes

Assessment against linked issues

Possibly linked issues

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

codspeed-hq Bot commented May 22, 2026

Uh oh!

codecov Bot commented May 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

edgarrmondragon commented May 21, 2026 • edited by sourcery-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Was generative AI tooling used to co-author this PR?

Related Issues

Summary by Sourcery

Uh oh!

netlify Bot commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for meltano canceled.

Uh oh!

sourcery-ai Bot commented May 21, 2026

Reviewer's Guide

Sequence diagram for CLI loading .env into os.environ

File-Level Changes

Assessment against linked issues

Possibly linked issues

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

codspeed-hq Bot commented May 22, 2026

Merging this PR will not alter performance

Uh oh!

codecov Bot commented May 22, 2026

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

edgarrmondragon commented May 21, 2026 •

edited by sourcery-ai Bot

Loading

netlify Bot commented May 21, 2026 •

edited

Loading