Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix config.dirs usage in the CLI #7906

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 20, 2023
Merged

fix config.dirs usage in the CLI #7906

merged 3 commits into from
Mar 20, 2023

Conversation

thrau
Copy link
Member

@thrau thrau commented Mar 19, 2023

This PR adds a band-aid to a long-standing issue that config.dirs is used when running CLI commands.

This caused localstack to create the .filesystem directory in the Python path, which was originally just designed for executing host mode in developer environments, even when simply using the CLI.

 % locate .filesystem     
/home/thomas/.pyenv/versions/3.10.9/lib/python3.10/site-packages/.filesystem
/home/thomas/workspace/localstack/localstack/.filesystem
/home/thomas/workspace/localstack/localstack-ext/.venv/lib/python3.10/site-packages/.filesystem

Because the CLI and the runtime share control paths, there are situations where values from config.dirs are used to create directories or files, even though they may not make sense in the context of the CLI.

  • dirs.cache is used by the analytics metadata (cache machine id file), and was used by the get_service_catalog call (now removed)
  • dirs.data is used by localstack_ext.config (this will be removed)
  • dirs.logs is used as target to put the logs of the docker run command when starting localstack
  • dirs.tmp should be removed now, but left it in there just in case

There is now a special control path that populates config.dirs deliberately with values that make the above examples work without the .filesystem directory.
To reduce the number of calls to config.dirs, I also removed some unused code from the startup procedure.

Once the old edge proxy logic is removed, we can more easily split up infra.py and therefore bootstrap.py, which will hopefuly also make it easier to separate out the CLI code.

@thrau thrau requested a review from dfangl March 19, 2023 11:17
@thrau thrau requested a review from alexrashed as a code owner March 19, 2023 11:17
@thrau thrau temporarily deployed to localstack-ext-tests March 19, 2023 11:17 — with GitHub Actions Inactive
@thrau thrau temporarily deployed to localstack-ext-tests March 19, 2023 11:22 — with GitHub Actions Inactive
@github-actions
Copy link

github-actions bot commented Mar 19, 2023

LocalStack integration with Pro

       1 files   -     2         1 suites   - 2   1h 45m 59s ⏱️ + 10m 14s
1 811 tests  -     2  1 413 ✔️  -   17  398 💤 +  15  0 ±0 
1 811 runs   - 720  1 413 ✔️  - 383  398 💤  - 337  0 ±0 

Results for commit 0496c40. ± Comparison against base commit 762339a.

♻️ This comment has been updated with latest results.

Copy link
Member

@alexrashed alexrashed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes are looking good, I only added a minor nitpick. It's great to see this behavior being fixed. I'm really looking forward to extracting the CLI from the runtime and properly isolate the two.
But it seems there is a small thing missing: The pipeline is red because the LocalstackContainer class expects the config.dirs.tmp to be present for writing the logs (which is now is not anymore). As soon as this is fixed, we're good to go from my perspective!

Comment on lines 3 to 4
# indicate to the environment we are starting from the CLI
os.environ["LOCALSTACK_CLI"] = "1"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This module should only be loaded if the CLI is started, but maybe this should still be moved to the main function body to ensure that this cannot be set accidentally by importing this module?

@thrau
Copy link
Member Author

thrau commented Mar 20, 2023

good catch @alexrashed, pushed a fix for your comments

@coveralls
Copy link

Coverage Status

Coverage: 85.204% (-0.001%) from 85.205% when pulling 0496c40 on v2-cli-directories into ba4125d on v2.

@thrau thrau merged commit fff0553 into v2 Mar 20, 2023
@thrau thrau deleted the v2-cli-directories branch March 20, 2023 18:49
thrau added a commit that referenced this pull request Mar 21, 2023
thrau added a commit that referenced this pull request Mar 22, 2023
dfangl pushed a commit that referenced this pull request Mar 22, 2023
alexrashed pushed a commit that referenced this pull request Mar 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants