Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit f754928

Browse files
authored
Add ClearMLHandler to track all MONAI Experiments (Project-MONAI#6013)
ClearML is a leading MLOps stack that can supercharge dialogues research with its state-of-the-art experiment tracking capability. ClearML: https://clear.ml/ I have added a `clearml_handler.py` which contains ClearMLHandler, ClearMLStatsHandler, and ClearMLImageHandler class. Basically, clearml can track everything which is tracked by tensorboard including scalars, and debug samples, and can also store models, artifacts, and the console in the ClearML server which can be easily accessed from ClearML WebUI shown as below: ![Screenshot from 2023-02-16 19-54-36](https://user-images.githubusercontent.com/46783803/219384828-77cb9c4c-4502-4529-95dd-8a95915385eb.png) ![Screenshot from 2023-02-16 19-54-29](https://user-images.githubusercontent.com/46783803/219384810-007ef976-b707-4dae-a31c-e9aba8200967.png) ![Screenshot from 2023-02-16 19-54-18](https://user-images.githubusercontent.com/46783803/219384795-2b5e60ac-fbff-49c7-8d6c-3eb2a34453dd.png) ![Screenshot from 2023-02-16 19-53-32](https://user-images.githubusercontent.com/46783803/219384770-b9b03667-8a12-4887-9049-84318e9a045b.png) ![Screenshot from 2023-02-16 19-53-49](https://user-images.githubusercontent.com/46783803/219384784-436f4ddb-d615-4720-ab62-1dd786a0989c.png) ![Screenshot from 2023-02-16 19-55-29](https://user-images.githubusercontent.com/46783803/219384449-fa41317c-21df-45a9-8605-7c7ed26fa637.png) ClearMLStatsHandler and ClearMLImageHandler can be used with Pytorch Trainer just like TensorboardStatsHandler and TensorboardImageHandler. Use `ClearMLStatsHandler()` & `ClearMLImageHandler(log_dir="./runs/", batch_transform=from_engine(["image", "label"]),output_transform=from_engine(["pred"]),)` with any MONAI example to test its functionality. Also, please let us where should we put further documentation and tutorials regarding this MLOps tool in the MONAI. --------- Signed-off-by: Muhammad Sakib Khan Inan <[email protected]> Signed-off-by: skinan <[email protected]> Signed-off-by: Victor Sonck <[email protected] Signed-off-by: Victor Sonck <[email protected]>
1 parent d118aa7 commit f754928

9 files changed

Lines changed: 355 additions & 27 deletions

File tree

docs/requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ pandas
2222
einops
2323
transformers<4.22 # https://github.com/Project-MONAI/MONAI/issues/5157
2424
mlflow>=1.28.0
25+
clearml >=1.10.0rc0
2526
tensorboardX
2627
imagecodecs; platform_system == "Linux" or platform_system == "Darwin"
2728
tifffile; platform_system == "Linux" or platform_system == "Darwin"

docs/source/handlers.rst

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -189,6 +189,17 @@ MLFlow handler
189189
.. autoclass:: MLFlowHandler
190190
:members:
191191

192+
ClearML handlers
193+
----------------
194+
.. autoclass:: ClearMLHandler
195+
:members:
196+
197+
.. autoclass:: ClearMLStatsHandler
198+
:members:
199+
200+
.. autoclass:: ClearMLImageHandler
201+
:members:
202+
192203
NVTX Handlers
193204
-------------
194205
.. automodule:: monai.handlers.nvtx_handlers

docs/source/installation.md

Lines changed: 66 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,21 @@
11
# Installation Guide
22

33
## Table of Contents
4-
1. [From PyPI](#from-pypi)
5-
1. [Milestone release](#milestone-release)
6-
2. [Weekly preview release](#weekly-preview-release)
7-
3. [Uninstall the packages](#uninstall-the-packages)
8-
1. [From conda-forge](#from-conda-forge)
9-
2. [From GitHub](#from-github)
10-
1. [System-wide](#milestone-release)
11-
2. [Editable](#weekly-preview-release)
12-
3. [Validating the install](#validating-the-install)
13-
4. [MONAI version string](#monai-version-string)
14-
5. [From DockerHub](#from-dockerhub)
15-
6. [Installing the recommended dependencies](#installing-the-recommended-dependencies)
4+
5+
- [Installation Guide](#installation-guide)
6+
- [Table of Contents](#table-of-contents)
7+
- [From PyPI](#from-pypi)
8+
- [Milestone release](#milestone-release)
9+
- [Weekly preview release](#weekly-preview-release)
10+
- [Uninstall the packages](#uninstall-the-packages)
11+
- [From conda-forge](#from-conda-forge)
12+
- [From GitHub](#from-github)
13+
- [Option 1 (as a part of your system-wide module)](#option-1-as-a-part-of-your-system-wide-module)
14+
- [Option 2 (editable installation)](#option-2-editable-installation)
15+
- [Validating the install](#validating-the-install)
16+
- [MONAI version string](#monai-version-string)
17+
- [From DockerHub](#from-dockerhub)
18+
- [Installing the recommended dependencies](#installing-the-recommended-dependencies)
1619

1720
---
1821

@@ -24,32 +27,39 @@ and the Python package index (PyPI). The pre-built Docker images are made availa
2427
To install optional features such as handling the NIfTI files using
2528
[Nibabel](https://nipy.org/nibabel/), or building workflows using [Pytorch
2629
Ignite](https://pytorch.org/ignite/), please follow the instructions:
30+
2731
- [Installing the recommended dependencies](#installing-the-recommended-dependencies)
2832

2933
The installation commands bellow usually end up installing CPU variant of PyTorch. To install GPU-enabled PyTorch:
34+
3035
1. Install the latest NVIDIA driver.
3136
1. Check [PyTorch Official Guide](https://pytorch.org/get-started/locally/) for the recommended CUDA versions. For Pip package, the user needs to download the CUDA manually, install it on the system, and ensure CUDA_PATH is set properly.
3237
1. Continue to follow the guide and install PyTorch.
3338
1. Install MONAI using one the ways described below.
34-
---
3539

40+
---
3641

3742
## From PyPI
3843

3944
### Milestone release
45+
4046
To install the [current milestone release](https://pypi.org/project/monai/):
47+
4148
```bash
4249
pip install monai
4350
```
4451

4552
### Weekly preview release
53+
4654
To install the [weekly preview release](https://pypi.org/project/monai-weekly/):
55+
4756
```bash
4857
pip install monai-weekly
4958
```
5059

5160
The weekly build is released to PyPI every Sunday with a pre-release build number `dev[%y%U]`.
5261
To report any issues on the weekly preview, please include the version and commit information:
62+
5363
```bash
5464
python -c "import monai; print(monai.__version__); print(monai.__commit_id__)"
5565
```
@@ -61,7 +71,9 @@ without uninstalling the existing one first.
6171
To address this issue, please uninstall both packages, and retry the installation.
6272

6373
### Uninstall the packages
74+
6475
The packages installed using `pip install` could be removed by:
76+
6577
```bash
6678
pip uninstall -y monai
6779
pip uninstall -y monai-weekly
@@ -70,51 +82,64 @@ pip uninstall -y monai-weekly
7082
## From conda-forge
7183

7284
To install the [current milestone release](https://anaconda.org/conda-forge/monai):
85+
7386
```bash
7487
conda install -c conda-forge monai
7588
```
7689

7790
## From GitHub
91+
7892
(_If you have installed the
79-
PyPI release version using ``pip install monai``, please run ``pip uninstall
80-
monai`` before using the commands from this section. Because ``pip`` by
93+
PyPI release version using `pip install monai`, please run `pip uninstall
94+
monai` before using the commands from this section. Because `pip` by
8195
default prefers the milestone release_.)
8296

83-
The milestone versions are currently planned and released every few months. As the
97+
The milestone versions are currently planned and released every few months. As the
8498
codebase is under active development, you may want to install MONAI from GitHub
8599
for the latest features:
86100

87101
### Option 1 (as a part of your system-wide module):
102+
88103
```bash
89104
pip install git+https://github.com/Project-MONAI/MONAI#egg=monai
90105
```
106+
91107
or, to build with MONAI C++/CUDA extensions:
108+
92109
```bash
93110
BUILD_MONAI=1 pip install git+https://github.com/Project-MONAI/MONAI#egg=monai
94111
```
95112

96113
To build the extensions, if the system environment already has a version of Pytorch installed,
97114
`--no-build-isolation` might be preferred:
115+
98116
```bash
99117
BUILD_MONAI=1 pip install --no-build-isolation git+https://github.com/Project-MONAI/MONAI#egg=monai
100118
```
119+
101120
this command will download and install the current `dev` branch of [MONAI from
102121
GitHub](https://github.com/Project-MONAI/MONAI).
103122

104123
This documentation website by default shows the information for the latest version.
105124

106125
### Option 2 (editable installation):
126+
107127
To install an editable version of MONAI, it is recommended to clone the codebase directly:
128+
108129
```bash
109130
git clone https://github.com/Project-MONAI/MONAI.git
110131
```
111-
This command will create a ``MONAI/`` folder in your current directory.
132+
133+
This command will create a `MONAI/` folder in your current directory.
112134
You can install it by running:
135+
113136
```bash
114137
cd MONAI/
115138
python setup.py develop
116139
```
140+
117141
or, to build with MONAI C++/CUDA extensions and install:
142+
118143
```bash
119144
cd MONAI/
120145
BUILD_MONAI=1 python setup.py develop
@@ -123,6 +148,7 @@ BUILD_MONAI=1 CC=clang CXX=clang++ python setup.py develop
123148
```
124149

125150
To uninstall the package please run:
151+
126152
```bash
127153
cd MONAI/
128154
python setup.py develop --uninstall
@@ -131,72 +157,82 @@ python setup.py develop --uninstall
131157
./runtests.sh --clean
132158
```
133159

134-
Alternatively, simply adding the root directory of the cloned source code (e.g., ``/workspace/Documents/MONAI``) to your ``$PYTHONPATH``
160+
Alternatively, simply adding the root directory of the cloned source code (e.g., `/workspace/Documents/MONAI`) to your `$PYTHONPATH`
135161
and the codebase is ready to use (without the additional features of MONAI C++/CUDA extensions).
136162

137163
> The C++/CUDA extension features are currently experimental, a pre-compiled version is made available via
138164
> [the recent docker image releases](https://hub.docker.com/r/projectmonai/monai).
139165
> Building the extensions from source may require [Ninja](https://ninja-build.org/) and [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit).
140166
> By default, CUDA extension is built if `torch.cuda.is_available()`. It's possible to force building by
141-
> setting ``FORCE_CUDA=1`` environment variable.
142-
167+
> setting `FORCE_CUDA=1` environment variable.
143168
144169
## Validating the install
170+
145171
You can verify the installation by:
172+
146173
```bash
147174
python -c "import monai; monai.config.print_config()"
148175
```
176+
149177
If the installation is successful, this command will print out the MONAI version information, and this confirms the core
150178
modules of MONAI are ready-to-use.
151179

152-
153180
## MONAI version string
181+
154182
The MONAI version string shows the current status of your local installation. For example:
183+
155184
```
156185
MONAI version: 0.1.0+144.g52c763d.dirty
157186
```
158-
- ``0.1.0`` indicates that your installation is based on the ``0.1.0`` milestone release.
159-
- ``+144`` indicates that your installation is 144 git commits ahead of the milestone release.
160-
- ``g52c763d`` indicates that your installation corresponds to the git commit hash ``52c763d``.
161-
- ``dirty`` indicates that you have modified the codebase locally, and the codebase is inconsistent with ``52c763d``.
162187

188+
- `0.1.0` indicates that your installation is based on the `0.1.0` milestone release.
189+
- `+144` indicates that your installation is 144 git commits ahead of the milestone release.
190+
- `g52c763d` indicates that your installation corresponds to the git commit hash `52c763d`.
191+
- `dirty` indicates that you have modified the codebase locally, and the codebase is inconsistent with `52c763d`.
163192

164193
## From DockerHub
194+
165195
Make sure you have installed the NVIDIA driver and Docker 19.03+ for your Linux distribution.
166196
Note that you do not need to install the CUDA toolkit on the host, but the driver needs to be installed.
167197
Please find out more information on [nvidia-docker](https://github.com/NVIDIA/nvidia-docker).
168198

169199
Assuming that you have the Nvidia driver and Docker 19.03+ installed, running the following command will
170200
download and start a container with the latest version of MONAI. The latest `dev` branch of MONAI from GitHub
171201
is included in the image.
202+
172203
```bash
173204
docker run --gpus all --rm -ti --ipc=host projectmonai/monai:latest
174205
```
175206

176207
You can also run a milestone release docker image by specifying the image tag, for example:
208+
177209
```
178210
docker run --gpus all --rm -ti --ipc=host projectmonai/monai:0.1.0
179211
```
180212

181213
## Installing the recommended dependencies
214+
182215
By default, the installation steps will only download and install the minimal requirements of MONAI.
183216
Optional dependencies can be installed using [the extras syntax](https://packaging.python.org/tutorials/installing-packages/#installing-setuptools-extras) to support additional features.
184217

185218
For example, to install MONAI with Nibabel and Scikit-image support:
219+
186220
```bash
187221
git clone https://github.com/Project-MONAI/MONAI.git
188222
cd MONAI/
189223
pip install -e '.[nibabel,skimage]'
190224
```
191225

192226
Alternatively, to install all optional dependencies:
227+
193228
```bash
194229
git clone https://github.com/Project-MONAI/MONAI.git
195230
cd MONAI/
196231
pip install -e '.[all]'
197232
```
198233

199234
To install all optional dependencies with `pip` based on MONAI development environment settings:
235+
200236
```bash
201237
git clone https://github.com/Project-MONAI/MONAI.git
202238
cd MONAI/
@@ -205,6 +241,7 @@ pip install -r requirements-dev.txt
205241

206242
To install all optional dependencies with `conda` based on MONAI development environment settings (`environment-dev.yml`;
207243
this will install PyTorch as well as `pytorch-cuda`, please follow https://pytorch.org/get-started/locally/#start-locally for more details about installing PyTorch):
244+
208245
```bash
209246
git clone https://github.com/Project-MONAI/MONAI.git
210247
cd MONAI/
@@ -215,10 +252,12 @@ conda env update -n <name> -f environment-dev.yml
215252
Since MONAI v0.2.0, the extras syntax such as `pip install 'monai[nibabel]'` is available via PyPI.
216253

217254
- The options are
255+
218256
```
219-
[nibabel, skimage, pillow, tensorboard, gdown, ignite, torchvision, itk, tqdm, lmdb, psutil, cucim, openslide, pandas, einops, transformers, mlflow, matplotlib, tensorboardX, tifffile, imagecodecs, pyyaml, fire, jsonschema, ninja, pynrrd, pydicom, h5py, nni, optuna]
257+
[nibabel, skimage, pillow, tensorboard, gdown, ignite, torchvision, itk, tqdm, lmdb, psutil, cucim, openslide, pandas, einops, transformers, mlflow, clearml, matplotlib, tensorboardX, tifffile, imagecodecs, pyyaml, fire, jsonschema, ninja, pynrrd, pydicom, h5py, nni, optuna]
220258
```
259+
221260
which correspond to `nibabel`, `scikit-image`, `pillow`, `tensorboard`,
222-
`gdown`, `pytorch-ignite`, `torchvision`, `itk`, `tqdm`, `lmdb`, `psutil`, `cucim`, `openslide-python`, `pandas`, `einops`, `transformers`, `mlflow`, `matplotlib`, `tensorboardX`, `tifffile`, `imagecodecs`, `pyyaml`, `fire`, `jsonschema`, `ninja`, `pynrrd`, `pydicom`, `h5py`, `nni`, `optuna`, respectively.
261+
`gdown`, `pytorch-ignite`, `torchvision`, `itk`, `tqdm`, `lmdb`, `psutil`, `cucim`, `openslide-python`, `pandas`, `einops`, `transformers`, `mlflow`, `clearml`, `matplotlib`, `tensorboardX`, `tifffile`, `imagecodecs`, `pyyaml`, `fire`, `jsonschema`, `ninja`, `pynrrd`, `pydicom`, `h5py`, `nni`, `optuna`, respectively.
223262

224263
- `pip install 'monai[all]'` installs all the optional dependencies.

monai/handlers/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
from .checkpoint_loader import CheckpointLoader
1515
from .checkpoint_saver import CheckpointSaver
1616
from .classification_saver import ClassificationSaver
17+
from .clearml_handlers import ClearMLHandler, ClearMLImageHandler, ClearMLStatsHandler
1718
from .confusion_matrix import ConfusionMatrix
1819
from .decollate_batch import DecollateBatch
1920
from .earlystop_handler import EarlyStopHandler

0 commit comments

Comments
 (0)