Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@sabinem
Copy link
Contributor

@sabinem sabinem commented Feb 14, 2024

This PR adds validation to the mongo db creation of data:

  • for components and versions the git repo and the commit hash are checked: for private git repos a github token is needed and needs to be added to the .env file, see .env.dist

  • ports are parsed from the cli into lists and then again checked before they are entered into the mongodb

  • paramaters are parsed from files and also checked before they are entered into the database

schema changes have been added on components, versions and executions: versions now contain an extended reference to executions

@sabinem sabinem changed the base branch from main to develop February 14, 2024 13:16
@sabinem sabinem force-pushed the cli-run-commands branch 7 times, most recently from 423e4ad to e66e2bf Compare February 16, 2024 10:48
{sabine.maennel} added 2 commits February 17, 2024 07:22
the config is needed in several places of the application
therefor it makes sense to read it in a module that can be
imported from other modules

add github token and mongo db name

the mongo db name should be seen as a setting

the github token is needed so that the github api can be called
also for private repos to check for the repos existence or for
the existence of commit hashes in the repo
this function should be reusable not only in the context of
handeling the mongodb: therefore it is moved in with other helper
functions
{sabine.maennel} added 10 commits February 19, 2024 06:44
these helper functions validate input form the command line
interface:

`parse` contains functions to parse and transform
command line argumments such as ports or files

`git` accesses the github api to check on repo urls and commit
hashes

`utils` contains utils to transform between formats
schema for components, versions and executions is changed
to imporve usage

versions now have an extended reference on components

executions just reference versions and not anymore versions and
components
adapt cli interface to the mongodb schema changes
a helper to get the version of odtp was added since it is also
used when versions are added to the mongodb: by default they
get a property odtp_version that has the current version of
odtp
the component_version is used in the naming when the workflow
of executions is run. Therefore it should be set
the version and component information has to be picked up
differently now. It is used for naming in the workflow
@caviri
Copy link
Member

caviri commented Feb 20, 2024

Hello @sabinem,

I found the problem when storing the output.zip file. The internal folder structure was not generated as there was one missing method. I fixed it.

I found another bug in which the step_index=0 was being evaluate as None when using if not step_index. This was creating folders with the incorrect name convention. I solved it by evaluating directly to None.

So far I tested an execution with multiple components, and it works as expected.

@caviri
Copy link
Member

caviri commented Feb 20, 2024

Hi @sabinem,

I tried adding a new component version, and it works good. The new version is added without issues. However, I detected a problem. The --ports parameters is expecting a port mapping string (8501:8501). This is wrong, as mapping is defined when creating a new execution. A component port should be a single port (8501) in order to tell the user that a map should be added later. The main reason behind this is to allow the user to have multiple instances of the same interactive component running in parallel.

@sabinem
Copy link
Contributor Author

sabinem commented Feb 20, 2024

@caviri Thanks a lot for the feedback: I will adapt this. Just one question:
Can a component only have a single port such as 8501 or can it also be a list of ports 8501,8502?

@caviri
Copy link
Member

caviri commented Feb 20, 2024

Hello @sabinem, good point. Is a list of ports.

caviri and others added 3 commits February 20, 2024 11:24
component carry only single ports:

port mappings are defined only later for executions of the
components
this just some minor renaming of the functions, so that is
clearer where port mappings / ports are parsed and checked
@sabinem sabinem requested a review from caviri February 20, 2024 18:35
@sabinem
Copy link
Contributor Author

sabinem commented Feb 20, 2024

@caviri: I did the requested changes regarding the port for component entries. I tested again:

  • execution entry: port-mappings (example 8080:8081+6000:6001,5000:5001)
  • component entry: single port (example 8080,6000)
  • component run: port-mappings (example 8080:8081+6000:6001)

Please review again. If everything is fine, you may approve and merge.

@caviri caviri merged commit b0e6075 into develop Feb 21, 2024
@sabinem sabinem deleted the cli-run-commands branch April 4, 2024 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants