Release0.3.0 #130

sabinem · 2024-05-15T13:45:36Z

Relase 0.3.0:

Status of bug fixes and comments:

pymongo upgrade is needed for security

both code files are currently not used and are therefore removed from the code base

Package upgrades

make execution forms user friendly for the execution create form it is now possible to add form fields in a more flexible way on demand rather then knowing in advance how much steps, parameters and ports should be added it can be configured to add a JSON Editor for showing the progress in the execution create and run form for execution runs a better workflow is provided: all that belongs to the run of an execution is now stored in one run object that can also be viewed in a json editor if configured.

refactor(dashboard execution pages): execution form user friendly

chore: remove unused code barfi and models

caviri · 2024-05-16T09:38:18Z

Hello @sabinem, I'm building and testing the ODTP GUI on my Windows machine.

I'm going to list here my suggestions and bugs I'm catching:

Home.

Check out documentation appears misplaced to the right. I would be more coherent to put if below. Also. I would add a little and concise list of the steps that are needed based on the most common cases. But this last is not urgent.

Menu bar.

A run is one action related to execution, so it would be better to include it within executions. If we want a shortcut, we can add a link to the section within Executions.

Users

How feasible is adding a button to create a new folder when setting the working directory?

General Layout.

The sidebar with the working area and other information is on the right, which feels a bit strange. Did you test the feeling of placing this to the right?

Also, this sidebar disappears when the screen width is minimized.

Components.

In the tabs menu, Add version at the same level of Add component. Add version should be a submenu under component selection. I would rename Select Component to Component options or Component settings and add the option to add a new version in that tab.

Manage Executions > Select and Execution

Change Cancel execution selection to Unselect execution.

Manage Executions > Select Execution

Amazing way of adding steps to the workflow. Great work @sabinem!!

Regarding the parameters from files, is there any way to change the icon or the color once you pick the parameters?

Parameters overwriting: Change Add/Remove line by Add/Remove parameter.

Ports. Change Ports to Ports mapping (XXXX:XXXX)

Manage Executions > Sidebar (Work Area)

Missing work directory information.

Prepare and Run Executions

Add the possibility to add secret fields. This is fine if the users needs it. However, make them "star" coded? Like ***********

I'm currently running the whole corsica pipeline. But Amazing work Sabine!

caviri · 2024-05-16T10:09:24Z

Prepare and Run Executions

Change Check output in project folder to Open Logging.
When I clicked on Run execution the modal shows ....loading

sabinem · 2024-05-27T09:38:46Z

@caviri I want to discuss with you on

Prepare and Run Executions
Add the possibility to add secret fields. This is fine if the users needs it. However, make them "star" coded? Like ***********

I would like to postpone this, unless you help me solve the problem I have with this:

for the parameters there is no problem to overwrite them, as they are stored as variables in the mongodb and taken from there in CLI command odtp execution run --project-path /Users/smaennel/odtp/digital-twins/testagain --execution-id 665447b2d8e14289615ef5a7 --secrets-files file.txt

But the secrets are taken from file. And the GUI starts the CLI command. Otherwise I would have to call the WorkflowManager directly, but it a long running process and should rather run in the background, therefore I went with starting the CLI command.

To be able to overwrite secrets, the CLI command would have to be changed, so that is not only takes files but also variables.

I would suggest to leave it as it is for now, since it is possible to provide secrets, you just have to add them via files.
What do you think or suggest? It would be for sure a lot of work if we want to implement the feature that you suggest here.

it should be accessible via the execution page

align in a reliable way with markdown add more links on the homepage

integrate it into the page, so that it does not disappear when the page width is reduced leave out unnecessary information

add functionality that user can create a new directory as working directory

automatically select a user when it has been added

adding versiob is now part of options for components the workarea is integretd into the page

since the function is now only used here

use a common layout for all dashboard pages

automatically select a digital twin when it has been added

when parameters files are uploaded for components, then add a check mark, so that the users know that this has been done succcessfully

that way there can be more progress seen in the dashboard logging when the steps are run also seperate log level on runs from the general log level

…n-dockerfile Feature/make sources editable in dockerfile

sabinem · 2024-06-18T13:13:43Z

@caviri Can you please take another look at the PR and approve if possible. I have followed up on your suggestions and also added the following features:

dev set up of docker compose, where the project is installed editable
indicators on the dashboard to show the connections to S3, Mongodb and Git
additional log level for the run of components, so that the log level there can be higher than the log level on the dashboard
improved logging during the run of executions: have custom loggers and use log.exception rather then log.error

caviri · 2024-06-18T13:36:38Z

Hello @sabinem, thank you! Can we have a two docker compose? One for dev and the other for "production"? It can happen that some users might change the branch on local potentially affecting to the files located on the docker, so I think it's better to reduce the amount of volumes mounted. What do you think?

Make code base editable in docker compose depending on a variable in the `.env` file * feat: Add pip install args * fix(compose.yml): mount docker service --------- Co-authored-by: Gabriel Nützi <[email protected]> Co-authored-by: sabinem <[email protected]>

sabinem · 2024-06-19T04:36:17Z

@caviri We added a setting that decides whether the code base will be editable. Hope this is considered by you good enough to take care of your concern on not having the code base editable in production.

add log level settings and app path setting

caviri · 2024-06-19T05:17:08Z

Good morning @sabinem, not fully convinced. What would be the point of having an /app volume mounted, then? My concern wasn't about the --editable flag, which I think it can remains. But rather with the /app volume mounted which I think it's not necessary on normal conditions of use. The inclusion of the --editable flag solves the problem I described above, however I think that can be simpler. Why not considering a compose for development and another for production?

Also we can take the opportunity to simplify the "production" compose by uploading the image to dockerhub and making not necessary to clone the repo and build the image in order to deploy the platform.

caviri · 2024-06-19T09:22:44Z

Hi, no, I think that it’s a good idea for developers. I was just worried about non experts users.

…

On 19 Jun 2024, at 10:50, Sabine Maennel ***@***.***> wrote: @caviri <https://github.com/caviri> Okay I can do that: but even then you would be against mounting the the app folder in the dev setup? — Reply to this email directly, view it on GitHub <#130 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AK2SKEPCRMBRIB4PHJOAEN3ZIFA6HAVCNFSM6AAAAABHYGXFZ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZYGEZDKNJWGU>. You are receiving this because you were mentioned.

sabinem · 2024-06-19T09:54:35Z

@caviri I have spilt the compose.yml into a compose.dev.yml and compose.prod.yml as you suggested. Can you please take another look on the PR now?

sabinem and others added 7 commits May 15, 2024 10:22

chore: upgrade dependencied: pymongo, nicegui, black

eb6363f

pymongo upgrade is needed for security

chore: upgrade dependicies by using poetry update

437623f

chore: remove unused code barfi and models

df827db

both code files are currently not used and are therefore removed from the code base

Merge pull request #127 from odtp-org/package-upgrades

47db7c6

Package upgrades

Merge pull request #128 from odtp-org/improve-gui-execution-forms

dd4d40c

refactor(dashboard execution pages): execution form user friendly

Merge pull request #129 from odtp-org/remove-unused-code

4295c35

chore: remove unused code barfi and models

sabinem force-pushed the release0.3.0 branch from c8cb156 to b2fc87e Compare May 27, 2024 10:47

sabinem added 19 commits May 27, 2024 13:15

chore(config __init__.py): set log level to ERROR

d0ed998

refactor(dashboard ui_theme.py): remove run page from menu

980cb98

it should be accessible via the execution page

feat(dashboard ui_theme.py): add helper function for missing value

908c2d9

fix(dashboard helpers,py): fix bug in setting execution versions

3eeb714

style(dashboard parameters.py): rename new line in parameter form

27af980

style(dashboard ports.py): rename ports mapping, improve layout

1319aef

style(dashboard page_about.py) improve about page

e68cc7c

align in a reliable way with markdown add more links on the homepage

style(dashbaord page_about.py): prettify page

22df96e

feat(dashboard page_about.py): reset storage on starting

7b16d9e

style(dashboard page_user.py): improve workarea location

013b7fe

integrate it into the page, so that it does not disappear when the page width is reduced leave out unnecessary information

feat(dashboard page_user.py): working directory can be created

6de13a1

add functionality that user can create a new directory as working directory

feat(dashboard page_user.py) select user after add

798209e

automatically select a user when it has been added

style(dashboard page_components.py): put version add under select

49cd7ad

adding versiob is now part of options for components the workarea is integretd into the page

style(dashboard page_components.py): improve layout of page

c7679d1

chore(dashboard: page_components.py): move function

567af39

since the function is now only used here

style(dashboard page_digital_twins.py): prettify page

fb5eb4e

use a common layout for all dashboard pages

feat(dashboard page_digital_twins.py) select digital twin after add

dc8c396

automatically select a digital twin when it has been added

style(dashboard page_executions.py): rename unselect button

0cc7cb7

feat(dashboard page_executioms.py): mark when uploaded parameters

f8650d6

when parameters files are uploaded for components, then add a check mark, so that the users know that this has been done succcessfully

sabinem added 4 commits June 14, 2024 07:16

fix(.env.dist.local): add equal sign after variable

b21df83

chore(pyproject.toml): upgrade version to 0.3.0

5d77717

fix(helpers:evironment): case of no output dirs

9e3bad2

chore(.gitignore): ignore all .env files except dist

f947478

sabinem force-pushed the release0.3.0 branch from a1521f9 to 368e78f Compare June 17, 2024 10:03

feat(dashboard:homepage): add checks for database and git connect

97f0802

sabinem force-pushed the release0.3.0 branch from de8602f to 97f0802 Compare June 17, 2024 10:10

sabinem and others added 7 commits June 17, 2024 21:03

feat(run.py workflow.py): improve run logs

29ceb22

that way there can be more progress seen in the dashboard logging when the steps are run also seperate log level on runs from the general log level

feat(settings): add extra log level for run

e363c85

fix(docker-compose): correct variable naming in compose

4a63f6e

feat: Mount repository dir to /app

282f342

fix(Dockerfile): first change into WORKDIR

8b5efa3

fix(dashboard storage): build execution output

0aec99f

Merge pull request #141 from odtp-org/feature/make-sources-editable-i…

5742d5a

…n-dockerfile Feature/make sources editable in dockerfile

sabinem and others added 2 commits June 18, 2024 22:20

fix(run.py): case of empty parameter

ecac206

sabinem added 2 commits June 19, 2024 06:42

fix(.env.dist): PIP_INSTALL_ARGS belongs to the compose setupy

9744967

chore(.env.dist): update .env.dist files with new settings

81f378e

add log level settings and app path setting

sabinem added 2 commits June 19, 2024 11:47

feat(compose.yml): split up into a dev and prod version

ae40e44

refactor(compose): rename dev compose file

d857b8c

caviri approved these changes Jun 19, 2024

View reviewed changes

sabinem force-pushed the release0.3.0 branch from 4e8019b to d857b8c Compare June 19, 2024 13:47

sabinem merged commit b63ac2e into main Jun 19, 2024

sabinem deleted the release0.3.0 branch June 19, 2024 13:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Release0.3.0 #130

Release0.3.0 #130

Uh oh!

sabinem commented May 15, 2024 •

edited

Loading

Uh oh!

caviri commented May 16, 2024

Uh oh!

caviri commented May 16, 2024

Uh oh!

sabinem commented May 27, 2024

Uh oh!

sabinem commented Jun 18, 2024

Uh oh!

caviri commented Jun 18, 2024

Uh oh!

sabinem commented Jun 19, 2024

Uh oh!

caviri commented Jun 19, 2024 •

edited

Loading

Uh oh!

caviri commented Jun 19, 2024 via email

Uh oh!

sabinem commented Jun 19, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Release0.3.0 #130

Release0.3.0 #130

Uh oh!

Conversation

sabinem commented May 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

caviri commented May 16, 2024

Uh oh!

caviri commented May 16, 2024

Uh oh!

sabinem commented May 27, 2024

Uh oh!

sabinem commented Jun 18, 2024

Uh oh!

caviri commented Jun 18, 2024

Uh oh!

sabinem commented Jun 19, 2024

Uh oh!

caviri commented Jun 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

caviri commented Jun 19, 2024 via email

Uh oh!

sabinem commented Jun 19, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sabinem commented May 15, 2024 •

edited

Loading

caviri commented Jun 19, 2024 •

edited

Loading