feat : simplify plugin ecosystem (init-container) #609
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Pull Request Test Coverage Report for Build 3441208627
💛 - Coveralls |
6cee607 to
c9b3f6a
Compare
6b3d8fa to
c9fccd3
Compare
c9fccd3 to
b47ae4e
Compare
13bd35d to
c102b58
Compare
* update job boot process for transformer * fix : hooks, google cred env * fix hook compilation issue : job.name * fix hook launch issue : list * fix : unit test for dag compilation * add entrypoint for init_container * add dry_run for skpo * update job boot process for transformer * fix : hooks, google cred env * fix hook compilation issue : job.name * fix hook launch issue : list * fix : unit test for dag compilation * add entrypoint for init_container * add dry_run for skpo * fix init_container_imagename * fix tests and init_container image * fix: k8s secret migration * remove comments (secrets)
00f6ff8 to
0a91c93
Compare
sravankorumilli
approved these changes
Nov 17, 2022
entrypoint_init_container.sh
Outdated
| echo "" | ||
|
|
||
| echo "-- initializing optimus assets" | ||
| OPTIMUS_ADMIN_ENABLED=1 optimus job run-input "$JOB_NAME" --project-name \ |
Contributor
There was a problem hiding this comment.
I want to understand does job run-input fail if the run doesn't exist?
| k8s.V1EnvVar(name=key,value=val) for key, val in job_meta["context"]["secrets"].items() | ||
| ] | ||
|
|
||
| def _dry_run(self, pod): |
Contributor
There was a problem hiding this comment.
what is this dry run?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes :
init-container:
fetches config: airflow to load config onto init-container & executor containerfetches assets: init container to load assets onto shared volumejob run-inputapi : to compile + fetch assets; instead of admin build instance