What's Changed

• Add reset_generator arg to samplers by @nkaenzig in #907
• Add JSON answer prompt template for multiple choice tasks by @nkaenzig in #905
• Add raw answer prompt template for multiple choice tasks by @elias-kaiko in #911
• Add QuiltVQA dataset & FreeFormQuestionPromptTemplate template by @nkaenzig in #912
• Add G-Eval LLM Judge by @nkaenzig in #914
• Add StratifiedRandomSampler to class samplers by @MaxFeucht in #916
• Add is_main_process util func for distributed inference by @MaxFeucht in #910
• Add CLI override recognition to ConfigurationLogger by @nkaenzig in #918
• Add .yaml config for QuiltVQA by @nkaenzig in #919
• Log session_id, run_id & log_dir by @nkaenzig in #925
• Add support for equals-separated cli args to ConfigurationLogger by @nkaenzig in #926
• ci: add zizmor for GH workflow SAST by @acidghost in #927
• ci: re-add id-token permission to CD workflow by @acidghost in #929
• ci: persist git credentials for pushing in CD by @acidghost in #930
• Fix wrong download instructions in tutorials by @nkaenzig in #933
• Expand init args in Resize transform by @nkaenzig in #931
• Update base64 image encoding logic to use jpeg by default by @nkaenzig in #935
• Unify prompting and answer extraction logic and introduce factories by @MaxFeucht in #932
• Free up space in release action by @nkaenzig in #936

New Contributors

• @elias-kaiko made their first contribution in #911
• @acidghost made their first contribution in #927

Full Changelog: 0.4.1...0.4.2

What's Changed

• Fix wandb rename run issue with wandb>=0.21.0 by @nkaenzig in #877
• Make processor source configurable through processor_kwargs in HuggingFaceModel by @nkaenzig in #878
• Change model output from List[str] to ModelOutput dict by @nkaenzig in #882
• Use unpack_batch function across multimodal wrappers by @MaxFeucht in #886
• Make image input kwarg in multimodal processor configurable by @MaxFeucht in #887
• Add is_global_zero check to ConfigurationLogger.setup() by @nkaenzig in #890
• Add temperature defaults for language and multimodal models by @MaxFeucht in #888
• Minor fixes for vision datasets by @nkaenzig in #894
• Enable litellm.drop_params in LiteLLMModel by @nkaenzig in #896
• Ensure compatibility of transforms with torchvision>=0.21.0 by @nkaenzig in #897
• Resolve multi-gpu issues in writer callbacks by @nkaenzig in #892
• Make BalancedSampler compatible with str inputs for language and multimodal by @MaxFeucht in #898
• Add radiology leaderboard by @nkaenzig in #856
• Use vit_tiny_patch16_224 with random weights in unit tests to avoid github rate limits when pulling models in CI by @nkaenzig in #901
• Add download_dir to PANDA by @nkaenzig in #899

Full Changelog: 0.4.0...0.4.1

What's Changed

• Create log_dir in ConfigurationLogger if it doesn't exist yet by @nkaenzig in #863
• Limit logger initalization to head node for multi-node inference by @MaxFeucht in #865
• Add seed argument to RandomSampler by @MaxFeucht in #866
• Add eva.multimodal by @nkaenzig in #874

New Contributors

• @MaxFeucht made their first contribution in #865

Full Changelog: 0.3.3...0.4.0

What's Changed

• Fix case where number of dataloader workers is 0 by @nkaenzig in #859
• Add log_table to eva.core.loggers by @nkaenzig in #860

Full Changelog: 0.3.2...0.3.3

What's Changed

• Make spacing configurable through env variables by @nkaenzig in #847
• Remove TotalSegmentator2D by @nkaenzig in #848
• Add SwinUNETRDecoderWithProjection by @nkaenzig in #850
• bump version to 0.3.1 [skip ci] by @github-actions[bot] in #855
• Add eva test command by @nkaenzig in #857

Full Changelog: 0.3.1...0.3.2

What's Changed

• Change datasets requirement to datasets<4.0.0,>=2.19.0 by @nkaenzig in #853

Full Changelog: 0.3.0...0.3.1

What's Changed

• bump version to 0.2.2 [skip ci] by @github-actions[bot] in #825
• Update LiTS17 dataset to make it compatible with 3D pipelines by @nkaenzig in #830
• Add MSDTask7Pancreas dataset by @nkaenzig in #832
• Replace input/output type hints in BaseModel by type vars by @nkaenzig in #839
• Add integration tests for segmentation tasks by @nkaenzig in #833
• Add function to seed dataloader workers & transforms by @nkaenzig in #844
• Add eva.language module by @kurbanrita in #831
• Add RandCropByLabelClasses and evaluate by step intervals by @nkaenzig in #842
• Add 2D radiology configs by @nkaenzig in #845
• Refactor Registry class and move to eva.core by @nkaenzig in #851

Full Changelog: 0.2.2...0.3.0

What's Changed

• chore(deps): pin peter-evans/create-pull-request action to c5a7806 by @renovate in #810
• chore(deps): update pdm-project/setup-pdm digest to 94a8231 by @renovate in #811
• chore(deps): update wntrblm/nox action to v2025 by @renovate in #815
• chore(deps): update peter-evans/create-pull-request action to v7 by @renovate in #814
• Fix missing image size bug in offline segmentation mode by @nkaenzig in #822
• Fix config urls in quick start section of README.md by @jklubienski in #824
• Add support for remote configs to SessionRecorder by @jklubienski in #823

New Contributors

• @jklubienski made their first contribution in #824

Full Changelog: 0.2.1...0.2.2

What's Changed

• Add missing coordinate checks to sampler unit tests by @nkaenzig in #783
• Add BTCV dataset / support for 3D radiology datasets by @nkaenzig in #785
• Add support for monai.inferers to SemanticSegmentationModule by @nkaenzig in #789
• Add unit tests for BTCV dataset by @nkaenzig in #793
• Add Midnight-12k to model registry by @nkaenzig in #795
• Add 3D data transforms by @nkaenzig in #787
• Add VoCo to model registry by @nkaenzig in #798
• Add Midnight-12k to leaderboard by @nkaenzig in #799
• Add CD action to build docs for main branch by @nkaenzig in #804
• Solve LFS issue with total segmentator test assets by @nkaenzig in #802
• Add config for BTCV eval by @nkaenzig in #800
• Add GPFM and H0-mini to model registry by @afilt in #770
• Remove BUILD file by @nkaenzig in #806
• Remove torchhub models from backbone registry by @nkaenzig in #808

New Contributors

• @github-actions made their first contribution in #782
• @afilt made their first contribution in #770

Full Changelog: 0.2.0...0.2.1

What's Changed

Main Updates

• Update to evaluation protocol - Increase patience to 20% and increase n_runs to 20 for slide level classification #772
• Added new pathology datasets: BRACS, BreaKHis, GleasonArvaniti, UniToPatho
• Added new pathology FMs such as UNI2-h & Virchow2 to model registry
• Setup for patch camelyon few-shot experiment (experimental) #773
• Fixes for using WandbLogger: #766

PRs

• Add on push to main release workflow by @ioangatop in #625
• Use GridSampler for CoNSeP by @nkaenzig in #645
• Make tqdm progress bar configurable through TQDM_REFRESH_RATE env variable by @nkaenzig in #650
• Add LiTSBalanced dataset by @nkaenzig in #648
• Bugfix in GeneralizedDiceScore move segmentation metrics to eva.vision by @nkaenzig in #653
• Use random split for LiTS by @nkaenzig in #651
• Add option to LiTS to adjust orientation by @nkaenzig in #659
• Update developer guide & pdm.toml by @nkaenzig in #666
• Add PANDASmall dataset by @nkaenzig in #664
• Create a small version of the Camelyon16 dataset by @roman807 in #667
• Corrected IN_FEATURES for H-optimus-0 in replicate_evaluations.md by @nkaenzig in #668
• Mention mean, std and dim explicitly in replicate_evaluations.md by @nkaenzig in #669
• Add N_PATCHES env variable by @nkaenzig in #671
• Fix: Skip register tokens in ExtractPatchFeatures by @nkaenzig in #672
• Increase N_PATCHES from 500 to 1000 for camelyon16_small.yaml by @nkaenzig in #676
• Update class indices in MoNuSAC to make compatible with generalized dice metric by @nkaenzig in #678
• Update Leaderboard by @nkaenzig in #670
• Add N_DATA_WORKERS env variable to make dataloader num_workers configurable by @nkaenzig in #680
• Make metrics device configurable through METRICS_DEVICE env variable by @nkaenzig in #683
• Add starplot leaderboard script to tools by @roman807 in #682
• Add CrossEntropyLoss and DiceCELoss wrappers with support for class weights by @nkaenzig in #686
• Set default metrics device to cpu by @nkaenzig in #688
• Add option to WSI datasets to save patch coordinates by @nkaenzig in #674
• Update metrics_device to use cpu when running on mps by @nkaenzig in #691
• Switch from SGD optimizer to AdamW for patch classification tasks by @nkaenzig in #690
• Remove leaderboard table from README.md by @nkaenzig in #692
• Fixed metrics device condition for mps by @nkaenzig in #693
• Add Virchow2 model to registry by @nkaenzig in #696
• Add phikon-v2 to registry by @nkaenzig in #697
• Add ConvDecoderWithImage decoder by @nkaenzig in #699
• Bump version to 0.1.2 by @nkaenzig in #701
• Add decompression preprocessing step to TotalSegmentator2D for more efficient slice loading by @nkaenzig in #705
• Update leaderboard with new segmentation results & adding new FMs by @nkaenzig in #711
• Relax eva's dependency requirements by @nkaenzig in #715
• Relax nibabel version dependency by @nkaenzig in #716
• Add TorchHubModel model wrapper by @nkaenzig in #721
• Update consep & monusac online configs to use ConvDecoderWithImage by @nkaenzig in #703
• Add CHECKPOINT_TYPE env variable to configs by @nkaenzig in #726
• Remove redundant line in DiceLoss by @nkaenzig in #710
• Change learning rate for segmentation configs to 0.002 by @nkaenzig in #722
• Add support for dataloader samplers by @nkaenzig in #713
• Reduce classes of TotalSegmentator2D dataset by @nkaenzig in #709
• Override checkpointing hooks to exclude backbone while saving checkpoints by @nkaenzig in #724
• Replace GeneralizedDiceScore by DiceScore & fix class-wise metrics by @nkaenzig in #719
• CI documentation fixing by @ioangatop in #730
• CD hotfix by @ioangatop in #732
• Update nibabel depedency by @ioangatop in #736
• Set shuffle option in train dataloader back to true in consep config by @nkaenzig in #738
• Add UNI2-h model to registry by @nkaenzig in #744
• Update head module documentation by @ioangatop in #753
• Update UNI download logic by @nkaenzig in #756
• Add GleasonArvaniti dataset by @nkaenzig in #754
• Add BRACS dataset by @nkaenzig in #751
• Add BreaKHis dataset by @nkaenzig in #749
• Add UNITOPatho dataset by @nkaenzig in #750
• Change default number of runs to 5 by @nkaenzig in #757
• Changes to BreaKHis & GleasonArvaniti by @nkaenzig in #759
• Set save_last default to False by @nkaenzig in #762
• Fix tracking of different runs when using WandbLogger by @nkaenzig in #766
• Add RESIZE_DIM parameter to yaml configs where it was missing by @nkaenzig in #769
• Increase patience to 20% and increase n_runs to 20 for slide level classification by @nkaenzig in #772
• Add PatchCamelyon 10-shot task config by @karasikov in #773
• Disable automatic releases & add PR creation to manual release pipeline by @nkaenzig in #777

New Contributors

• @karasikov made their first contribution in #773

Full Changelog: 0.1.0...0.2.0