Skip to content

Latest commit

 

History

History
1313 lines (650 loc) · 46.6 KB

CHANGELOG.md

File metadata and controls

1313 lines (650 loc) · 46.6 KB

CHANGELOG

v5.1.0 (2024-03-05)

Feature

  • feat: allow building pages on demand (10259ca)

Fix

  • fix: make toml_version tuple (56e0f7a)

  • fix: deactivate type check

This is to ignore new type errors. Issue is opened to fixe type errors (e9eb9a6)

Unknown

  • Deactivate isort and black in ci (8d8cad6)

  • Update README.md

Name change (b2f4eb3)

v5.0.1 (2023-07-02)

Fix

Unknown

v5.0.0 (2023-05-07)

Breaking

  • fix: remove scheduler arg

BREAKING CHANGE: scheduler is unset (03f60dc)

Fix

  • fix: do not track out, sbatch, ckpt (68d4c55)

  • fix: use train stage for setup (d0b63b1)

  • fix: update dataloader (b166512)

  • fix: update defaults (5964cc7)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (6385a6a)

v4.15.0 (2023-04-25)

Feature

  • feat: compute mean and confidence interval

Following TrivialAugment's recommendation to provide how the mean and confidence interval are calculated. (da2e834)

Fix

  • fix: add confidence typehint (c057652)

  • fix: add typehints (cb761f2)

  • fix: revert hidden layers to one hidden layer

with dropout. (3139c46)

  • fix: add typehint (f382d1f)

  • fix: ignore missing scipy imports (f562790)

v4.14.2 (2023-04-25)

Fix

  • fix: remove contrastive collate_fn

fixes #13 (1811bb5)

Unknown

v4.14.1 (2023-04-24)

Fix

  • fix: don't try to open augmented set if not exists (bf33e12)

Unknown

v4.14.0 (2023-04-24)

Feature

  • feat: do data augmentation in feature space

Following extrapolation from https://arxiv.org/pdf/1702.05538.pdf. (2d0fa05)

Unknown

v4.13.2 (2023-04-21)

Fix

  • fix: use clinical_context attr for export (446bb2b)

v4.13.1 (2023-04-21)

Fix

  • fix: alter seed config space (e8a652e)

  • fix: use variable splits_dirname (f3824e4)

  • fix: automatically set_clinical context (764b77c)

v4.13.0 (2023-04-21)

Feature

  • feat: add device to mean and std calc (4db5601)

Fix

  • fix: add batch size to compile features tool (7a13c09)

  • fix: add num_workers to compile features tool (018861f)

  • fix: add clinical context to compile features tool (8d89607)

  • fix: change location dtype to string in hdf5 (ef9e5cf)

  • fix: use normalization based on masked tiles (6e5e639)

  • fix: typo (cadd918)

Unknown

v4.12.0 (2023-04-21)

Feature

  • feat: calculate mean and stddev (4ddf6d1)

Fix

  • fix: raise error if no mask_root_dir is set

when loading from disk (2fd7157)

Unknown

v4.11.1 (2023-04-21)

Documentation

  • docs: add steps to adding-new-data (49f10d5)

  • docs: clarify tensorflow for gpu installation (2a8b90d)

Fix

  • fix: allow null as datamodule model (77dff7e)

  • fix: typo

wrong number of selected images was displayed (a25a251)

Unknown

v4.11.0 (2023-04-20)

Feature

  • feat: add denoising tool (ee43414)

Fix

  • fix: add mask creation to CLI and update docs (0dbdbc4)

Unknown

  • Merge pull request #43 from siemdejong/denoise

feat: add denoising tool (fb62c1b)

  • Merge pull request #42 from siemdejong/mask

fix: add mask creation to CLI and update docs (6121500)

v4.10.1 (2023-04-14)

Fix

v4.10.0 (2023-04-13)

Feature

  • feat: allow for CCMIL hparam tuning (9e587f8)

Fix

  • fix: typing of example array (cb98f39)

  • fix: make random tensor as example (a8d21f0)

  • fix: move tokenized inputs to device (2b75731)

  • fix: make new example input for lightning (a431ab2)

  • fix: only use one clinical context of the bag (82a71df)

v4.9.3 (2023-04-13)

Fix

  • fix: restructure config files

CCMIL is used by default. If another model is needed, specify with "--model path/to/varmil.yaml" when training the model. (35c24a9)

  • fix: revert switch to cpu (0bb86e7)

Unknown

v4.9.2 (2023-04-12)

Fix

  • fix: switch to clinical tiny BERT

Because it is much quicker. (ac199c1)

Unknown

v4.9.1 (2023-04-12)

Ci

  • ci: disable HDF5 version check

To bypass "UserWarning: h5py is running against HDF5 1.14.0 when it was built against 1.12.2". (148c8ab)

Documentation

  • docs: remove _static rom html_static_path (d1a1da8)

Fix

  • fix: set llm to eval mode and disable gradient cal

unless trainable_llm is set to True. (1303353)

Unknown

v4.9.0 (2023-04-12)

Feature

  • feat: add location to splits output (4440a53)

  • feat: add CCMIL (b0294ce)

  • feat: add clinical context to h5 dataset (3197f81)

  • feat: add clinical context to image dataset (e229d1e)

Fix

  • fix: add typehints (486b392)

  • fix: add LLM output last_hidden_state protocol (81e5e74)

  • fix: ignore transformers missing import (5c6806d)

  • fix: add transformers dependency (8145723)

Unknown

v4.8.0 (2023-04-06)

Feature

  • feat: add num_workers to feature compiling (0f4ef30)

v4.7.0 (2023-04-06)

Feature

  • feat: attention tiles viz (df9f90f)

  • feat: change hparam search vars (fbd5847)

Fix

  • fix: explicitly export attention model (7aa356d)

  • fix: remove argument linking (d0d28e6)

  • fix: change defaults (dcdfea6)

Unknown

  • Update issue templates

gh: add docs request issue template (7d0663c)

v4.6.0 (2023-04-04)

Feature

  • feat: make trials reproducible

by adding a seed

and reconfigure search space (6fb64c4)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (86bdd62)

v4.5.0 (2023-04-04)

Feature

Unknown

  • Merge pull request #35 from siemdejong/test

Test the model (cad0f37)

v4.4.0 (2023-03-30)

Ci

  • ci: only run sphinx build when release successful (e4d2c53)

Feature

  • feat: add hparam tuner (#31)

  • feat: add hparam tuner

  • docs: add clarification on wprkaround class

  • fix: don't assume self.D is a list

  • fix: move parameters to one place

  • fix: remove ray_lightning

  • fix: use minimum number of epochs

  • fix: use multivariate TPE with constant_liar

  • fix: report accuracies to ray

  • fix: add comments on tpesampler (b2e4094)

Fix

Unknown

  • Optuna (#33)

  • feat: add hparam tuner

  • docs: add clarification on wprkaround class

  • fix: don't assume self.D is a list

  • fix: move parameters to one place

  • fix: remove ray_lightning

  • fix: use minimum number of epochs

  • fix: use multivariate TPE with constant_liar

  • fix: report accuracies to ray

  • fix: add comments on tpesampler

  • fix: run precommit (53f16b4)

v4.3.2 (2023-03-24)

Ci

  • ci: only build docs if release workflow completed (9d49a34)

  • ci: remove sphinx action (b512b35)

  • ci: set bash defaults (18bbd18)

  • ci: activate dpat for docs compilation (f459363)

Documentation

  • docs: fix indent (2e7ebf1)

  • docs: ignore autosummary (e63a009)

  • docs: move docs from readme to gh pages (6012ebc)

  • docs: add docs to github pages (1f85ef8)

Fix

  • fix: type error (e2af84a)

  • fix: bug with reading from bytes

Seek first character of a string before reading it in the h5 dataset. (ba1c58e)

  • fix: give units to fraction logs (416749a)

v4.3.1 (2023-03-23)

Fix

  • fix: up default patience to 100 (289d7f9)

v4.3.0 (2023-03-23)

Feature

  • feat: add early stopping (7c4eb01)

  • feat: add dropout_p parameter to varmil

To possibly change the amount of dropout (6b59e50)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (73c584b)

v4.2.0 (2023-03-23)

Ci

  • ci: bypass conda bug

https://stackoverflow.com/a/72178361 (83c3991)

Feature

  • feat: add hidden features and dropout (452082c)

Fix

  • fix: typing (5dce7a4)

  • fix: add area under precision recall curve (1a2648f)

  • fix: remove optimizer/scheduler defaults

This was needed to set the optimizer and scheduler via the command line, like "--optimizer=Adam --optimizer.lr=0.003" etc. (d2b62e1)

  • fix: link max_epochs to scheduler T_max

Because otherwise the cosine annealing doesn't work and only uses the initial amount, regardless of what is set at the command line. (1a05932)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (6c3e27c)

v4.1.0 (2023-03-23)

Feature

  • feat: entropymasker experiment (ec70eef)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (62e2f10)

v4.0.1 (2023-03-20)

Fix

  • fix: compiling model doesn't work, rollback

ref: see snellius output slurm-2473416.out (e733a24)

v4.0.0 (2023-03-20)

Breaking

  • fix: filter h5 dataset by filenames

Using the paths_and_targets keyword, which is equivalent to the similar similar keyword in the tile dataset.

BREAKING CHANGE: train|val|test_path keywords now do not refer to the the target train|val|test paths to output h5 files to, but to the files providing paths_and_targets. file_path is the new target file, containing all images. (b3e1770)

Fix

  • fix: update mil config to reflect new signature (c4dc0ed)

  • fix: safely bypass sigint or sigterm

handy for using with slurm for example (388d368)

  • fix: remove trainer arguments for pl2 (99194e0)

v3.2.1 (2023-03-17)

Fix

  • fix: remove pl2 removed trainer arguments (817beb8)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (708e3fb)

v3.2.0 (2023-03-17)

Feature

  • feat: automate masking with dlup entropy_masker (1066ea1)

  • feat: compile the pytorch lightning models (c009004)

Fix

  • fix: lightning 2.0 ignore missing type imports (08b3e85)

  • fix: ddp default find_unused_parameters=False

Lightning 2+ sets find_unused_parameters for the ddp strategy to false. Therefore, it is unnecessary to set it to False in the config file. (3e2a15d)

  • fix: use recommended precision

bf16-mixed has less ambiguity than bf16. bf16-mixed makes it clear mixed precision is used. (6b654e2)

  • fix: rename epoch_end to on_epoch_end

*_epoch_end is deprecated in lightning 2 (37e0c52)

  • fix: add torch/lightning v2 deps (2afdf05)

  • fix: simclr-16-3 update (29fdf58)

Performance

  • perf: do not log to progress bar

Following the lightning 2.0 recommendation of not logging to the progress bar, AUC and F1 are not logged to the process bar anymore. They are still logged to the installed logger, e.g. Tensorboard. (da4e3b4)

Unknown

  • Mask load from disk (#25)

  • feat: mask tiles in pmchhgImageDataset

Using the new entropy_masker mask function from dlup.

  • fix: bumpy python version (340edde)

  • Merge pull request #24 from siemdejong/torchv2

Torchv2 (a0ef974)

  • Merge branch 'main' of github.com:siemdejong/dpat (51c0fad)

v3.1.1 (2023-03-15)

Fix

  • fix: pretrained config typo (e6a3778)

v3.1.0 (2023-03-15)

Feature

  • feat: add num_subfolds parameter to create_splits (dfe7ceb)

Fix

  • fix: splits tests with num_subfolds (3a9d44e)

Unknown

  • Merge pull request #23 from siemdejong/less-folds

feat: add num_subfolds parameter to create_splits

fix #22 (5d201d1)

v3.0.1 (2023-03-15)

Fix

  • fix: update viz of untrained model (e32d9f5)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (d63095a)

v3.0.0 (2023-03-15)

Breaking

  • feat: from torchvision.models to pytorchcv models

Torchvision has less pretrained and prebuilt models available than pytorchcv. Fort the application of a small dataset, it is nice to have smaller resnets available, which torchvision has less support for.

BREAKING CHANGE: in the config files, models must be specified by corresponding pytorchcv models, not torchvision. (bb5732d)

Feature

  • feat: show targets/img_id/case_id distribution (c757995)

Fix

  • fix: pytorchcv not typed ignore missing imports (2b33f70)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (79c5b72)

v2.7.3 (2023-03-14)

Performance

  • perf: speed up feature compilation

Fixes part of #16, namely the writing part. (772a0bd)

v2.7.2 (2023-03-14)

Fix

  • fix: create feature directory if not exists (674c8c4)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (0859ba1)

v2.7.1 (2023-03-13)

Fix

  • fix: bypass oom error (0177032)

  • fix: add ipywidgets dependency

for tqdm in notebooks (3c26dcd)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (65a9bd1)

  • Merge branch 'main' of github.com:siemdejong/dpat (07c8f20)

  • Merge branch 'main' of github.com:siemdejong/dpat (120fefb)

v2.7.0 (2023-03-10)

Documentation

  • docs: change installation order. (63d51c8)

Feature

  • feat: visualize embedding distribution using t-sne (91c4190)

v2.6.1 (2023-03-10)

Chore

  • chore: remove notebooks from linguist (98e5411)

Fix

  • fix: show case distribution (e2dcd66)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (4c2edf3)

  • Merge branch 'main' of github.com:siemdejong/dpat (28b5398)

  • Merge branch 'main' of github.com:siemdejong/dpat (90a8b81)

  • Merge branch 'main' of github.com:siemdejong/dpat (1b81024)

  • Merge branch 'main' of github.com:siemdejong/dpat (2639798)

  • Merge branch 'main' of github.com:siemdejong/dpat (bdbc689)

  • Merge branch 'main' of github.com:siemdejong/dpat (77ef0eb)

  • Merge branch 'main' of github.com:siemdejong/dpat (0ca5433)

  • Merge branch 'main' of github.com:siemdejong/dpat (f9152ee)

  • Merge branch 'main' of github.com:siemdejong/dpat (1cc30c4)

v2.6.0 (2023-03-10)

Feature

  • feat: nearest neighbours visualization features (6239607)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (b47b685)

v2.5.2 (2023-03-10)

Fix

  • fix: jsonargparse dependency (94242b2)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (8440021)

v2.5.1 (2023-03-10)

Fix

  • fix(log): log origin of image data (1bb50ae)

v2.5.0 (2023-03-09)

Feature

  • feat(pretrain): add SimCLR (502bba3)

Unknown

  • Merge pull request #21 from siemdejong/simclr

feat(pretrain): add SimCLR (c6dc62f)

v2.4.2 (2023-03-08)

Ci

  • ci: run mypy in the dpat directory at pre-commit (662276e)

Fix

  • fix(type): fix iter type error (d69fb24)

  • fix: use WeightedRandomSampler

Because the training dataset could be imbalanced. (55334c4)

Unknown

  • Merge pull request #20 from siemdejong/oversampling-minority

fix: use WeightedRandomSampler

Fixes #19 (f20f57e)

v2.4.1 (2023-03-08)

Performance

  • perf(varmil): set gradients to none instead of 0 (b8a850a)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (a344d8f)

v2.4.0 (2023-03-07)

Ci

  • ci: rm arch name from ci cache (488edfa)

Feature

Fix

  • fix: some typing issues (6a46853)

  • fix: bug where M was calculated wrong (d3a675b)

  • fix: export variables from packages (e0db9e3)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (fcaa977)

v2.3.3 (2023-03-04)

Ci

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (fca3841)

v2.3.2 (2023-03-04)

Ci

  • ci: flake8 for extract_features pkg (c84b6da)

Fix

  • fix: uncomment feature compilation (4e018fe)

  • fix: fix number of compiled feature vectors

Fix #17 (b659501)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (dbb61cf)

v2.3.1 (2023-03-03)

Fix

  • fix: concatenate features

Feature vectors of tiles belonging to the same image will be concatenated and stored in the belonging image group along with their metadata in datasets instead of h5-attrs.

This allows for MIL to use batch size=1bag and load one dataset at a time. (349b13d)

  • fix: h5 classmethod return type (bc4b24a)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (05dc1e9)

v2.3.0 (2023-03-02)

Feature

  • feat: add h5 dataset

Multi instance learning in stage 2 is easier on tensors obtained from an hdf5 file with datasets embedded. (0f6cfe2)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (87ce747)

v2.2.0 (2023-03-02)

Feature

  • feat: add feature extraction

Add feature extractor training. Add feature vector compilation to HDF5. Add config files to be used with pl.Trainer. Refactor. Add float32 matmul precision setting. Add cudnn auto tuner setting. Add Omegaconf "now" interpolation. Require lightning[extra] and h5py as deps. Run pre-commit also on dpat/extract_features. Link pl CLI to pl Trainers. (e71d5be)

v2.1.6 (2023-02-26)

Performance

  • perf: pin memory of dataloaders (4afcdf2)

  • perf: zero_grad to non (d6d62c7)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (df2974e)

v2.1.5 (2023-02-26)

Ci

  • ci: use conda shell (65d7e00)

  • ci: fix conda python version (9ab509d)

  • ci: deps and release needs test (fe5e2ed)

Fix

  • fix(tests): add tests for create_splits (780ed07)

Unknown

  • Merge branch 'main' of github.com:siemdejong/dpat (c015208)

v2.1.4 (2023-02-26)

Fix

  • fix(tests): add convert tests (819e23d)

v2.1.3 (2023-02-25)

Fix

  • fix(semantic-release): add build command

setup.py is not available, so python setup.py ... does not work. build_command installs 'build' en builds distributions. (c35a40f)

v2.1.2 (2023-02-25)

Fix

  • fix(typing): add typing (0842497)

v2.1.1 (2023-02-23)

Ci

  • ci: update psr repo name (4ccf2e8)

Fix

  • fix(swav): make swav return a number

Fix #7

SwAV by Lightly apparently needs the user to not use a datamodule. If using the datamodule, call setup() on it and use datamodule.x_dataloader() to pass to trainer.fit along with the model. (d21dcce)

  • fix(deps): add tensorboard to dependencies (5ee3528)

  • fix(convert): only log skip if skip_count>0 (0a58791)

v2.1.0 (2023-02-22)

Ci

  • ci: add workflow dispatch (10da34a)

  • ci: docformat, no-opt, print, ssort, black, flake8 (50e8ef4)

  • ci: add continuous integration

Move setup.py to pyproject.toml. Add semantic-release github ci. Add flake8 and yesqa to pre-commit. (81e50a0)

Feature

  • feat(data): add mean and std calculator (e6549f6)

  • feat(stage1): Implement swav with pl and lightly (3dcbdca)

Fix

  • fix(package): add dependencies (b3d340c)

  • fix(package): package dpat subfolder (2fb78ed)

Unknown

v2.0.0 (2023-02-16)

Breaking

  • feat(installation): remove the need for config.yml

To install dpat in a script to use as api on windows, it is able to achieve that with import dpat dpat.install_windows("path/to/vips/bin")

BREAKING CHANGE: installation via the config.yml is no longer possible. It is also no longer needed for splits/convert cli operations. For coming deep learning cli applications, it will be needed to fetch the path to vipsbin from a config with deep learning options. (bdbed64)

  • fix(logging): let the cli configure logging

This allows for a user using the library to configure logging, e.g. use logging.getLogger('dpat').propagate = False, if logging is not needed. The cli will always log to the terminal.

BREAKING CHANGE: logging with the config file is now unsupported. Configure logging in the application using the library. (85875fd)

Documentation

  • docs(cuda): add docs about cuda (58ca68f)

  • docs(logging): remove config.yml logging

Previous commit disabled logging config with config.yml. (60e6d90)

  • docs(readme): clarify log/vips config

Add examples for config.yml for logging. Show how to turn off logging propagation when using as a library. (0e06f66)

Fix

  • fix(dpat): only read config.yml if windows

Linux users should be able to install vips with conda. (f823ccc)

v1.4.0 (2023-02-15)

Documentation

  • docs(convert): change bulk to batch (2272355)

Feature

  • feat(logging): add logging and log config

Logging is done to the NullHandler. However, the user can turn logging on by setting the handler and level in config.yml, like LOGGING: handler: StreamHandler level: INFO (6c90359)

Fix

  • fix(cli): remove unnecessary name=main chk (c5f5223)

v1.3.0 (2023-02-15)

Feature

  • feat(dataset): add PMCHHGImageDataset

Add a dataset to loop through all tiles create from images in an image directory. (1c65f85)

v1.2.2 (2023-02-15)

Fix

  • fix(splits): change saved filename to relative

Fixes #1. (5ba6ed9)

v1.2.1 (2023-02-15)

Documentation

  • docs(cli): update docs of cli (b917e52)

Fix

  • fix(convert): decompressionbomberror help (9ff923d)

v1.2.0 (2023-02-15)

Feature

  • feat(cli): change from argeparse to click (470ae65)

v1.1.1 (2023-02-14)

Fix

  • fix(splits): a bug with default include/exclude

Argparse's action 'append' appends arguments to the default. Postprocessing the arguments fixes this. (3f46207)

v1.1.0 (2023-02-14)

Documentation

  • docs(readme): change installation heading (dd1416d)

Feature

  • feat(splits): change split logic

Create 5 stratified train-test folds. For every fold, randomly create 5 subfolds, dividing the training set into train-val (0.8/0.2). Save every filename of every fold to a file with a unique filename, denoting the set type, and subfold and fold id. (5f38156)

  • feat(splits): add inclusion and exclusion pattern

Some images might be needed to exclude by filename. E.g. images with 300fast inside it. Otherwise, only images with 200slow may be included. (caa5d2a)

  • feat(splits): add overwrite argument (f2a664f)

v1.0.0 (2023-02-14)

Breaking

  • feat(dpat): change project.ini to config.yml

YAML files are better structured.

BREAKING CHANGE: project.ini will not be read anymore. PATHS.vips has to be set in config.yml, somewhere near the root of the repository. (f13ed4b)

Documentation

  • docs(installation): clarify where to specify vipsbin (643009a)

  • docs(splits): add help of splits cli to README (e3ef258)

Unknown

v0.4.1 (2023-02-14)

Fix

  • fix(dpat): import pyvips before openslide (2d25ae0)

v0.4.0 (2023-02-14)

Feature

  • feat: add pre-commit

Make sure black and isort are ran. (620dccb)

  • feat(splits): create splits and run isort/black (fe8cf25)

v0.3.0 (2023-02-13)

Documentation

  • docs: fix typo (18971e2)

  • docs: fix 300fast typo (f0ed148)

  • docs: clarify image fn needs scanprogram for tif (f0f7384)

  • docs: clarify conda dependency (932569f)

Feature

  • feat: add num-workers and #chunks to cli (0f322c4)

v0.2.0 (2023-02-13)

Documentation

  • docs: remove logo (f932b71)

  • docs: make package and add readme

Add docs on how to build the package. Add README.md to document usage of bulk convert. (e5fd4a5)

Feature

  • feat: clarify dlup will be installed with dpat (c062884)

v0.1.0 (2023-02-13)

Feature

  • feat: initial commit

Add dlup and its dependencies, resolved for Windows. Add any (e.g. bmp) to TIFF converter. (5d2fc3a)