[DOC] Add example notebook for using aeon distances with sklearn clusterers #2511

SalmanDeveloperz · 2025-01-22T20:41:47Z

Reference Issues/PRs

What does this implement/fix? Explain your changes.

This pull request introduces a new Jupyter Notebook: sklearn_clustering_with_aeon_distances.ipynb. The notebook demonstrates how to integrate aeon's distance metrics with scikit-learn clustering algorithms. It includes:

Hierarchical Clustering: Using AgglomerativeClustering with metric="precomputed".
Density-Based Clustering: Using DBSCAN and OPTICS with metric="precomputed".
Spectral Clustering: Using SpectralClustering with affinity="precomputed" and the inverse of the distance matrix as the similarity matrix.
This addition enhances the clustering documentation, showing how to combine aeon’s distance metrics with widely-used scikit-learn clusterers.

Does your contribution introduce a new dependency? If yes, which one?

No new dependencies introduced.

Any other comments?

The notebook has been tested locally, and all cells execute without errors.
A reference to this notebook has been added to the clustering section of the documentation.

PR checklist

For all contributions

I've added myself to the list of contributors. Alternatively, you can use the @all-contributors bot to do this for you after the PR has been merged.
The PR title starts with either [ENH], [MNT], [DOC], [BUG], [REF], [DEP] or [GOV] indicating whether the PR topic is related to enhancement, maintenance, documentation, bugs, refactoring, deprecation or governance.

For new estimators and functions

I've added the estimator/function to the online API documentation.
(OPTIONAL) I've added myself as a __maintainer__ at the top of relevant files and want to be contacted regarding its maintenance. Unmaintained files may be removed. This is for the full file, and you should not add yourself if you are just making minor changes or do not want to help maintain its contents.

For developers with write access

(OPTIONAL) I've updated aeon's CODEOWNERS to receive notifications about future changes to these files.

review-notebook-app · 2025-01-22T20:41:53Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

aeon-actions-bot · 2025-01-22T20:42:12Z

Thank you for contributing to `aeon`

I did not find any labels to add based on the title. Please add the [ENH], [MNT], [BUG], [DOC], [REF], [DEP] and/or [GOV] tags to your pull requests titles. For now you can add the labels manually.
I have added the following labels to this PR based on the changes made: [ $\color{#45FD64}{\textsf{examples}}$ ]. Feel free to change these if they do not properly represent the PR.

The Checks tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our pre-commit code quality check fails, any trivial fixes will automatically be pushed to your PR unless it is a draft.

Don't hesitate to ask questions on the aeon Slack channel if you have any.

PR CI actions

These checkboxes will add labels to enable/disable CI functionality for this PR. This may not take effect immediately, and a new commit may be required to run the new configuration.

Run pre-commit checks for all files
Run mypy typecheck tests
Run all pytest tests and configurations
Run all notebook example tests
Run numba-disabled codecov tests
Stop automatic pre-commit fixes (always disabled for drafts)
Disable numba cache loading
Push an empty commit to re-run CI checks

SalmanDeveloperz · 2025-01-22T21:06:38Z

Dear maintainers,
I have created a new notebook and would like to know where to add its reference in the following documentation:

Clustering Overview
Clustering with sklearn.cluster
Could you please guide me on the most appropriate sections in these files to include the reference to the new notebook? I want to ensure it integrates well with the existing content before committing the changes.
Looking forward to your advice.
Best Regards

TonyBagnall · 2025-01-24T09:32:50Z

Dear maintainers, I have created a new notebook and would like to know where to add its reference in the following documentation:

Clustering Overview Clustering with sklearn.cluster Could you please guide me on the most appropriate sections in these files to include the reference to the new notebook? I want to ensure it integrates well with the existing content before committing the changes. Looking forward to your advice. Best Regards

hi, thanks for this, we will take a look

examples/clustering/sklearn_clustering_with_aeon_distances.ipynb

SebastianSchmidl · 2025-01-25T22:20:23Z

Dear maintainers, I have created a new notebook and would like to know where to add its reference in the following documentation:

Clustering Overview Clustering with sklearn.cluster Could you please guide me on the most appropriate sections in these files to include the reference to the new notebook? I want to ensure it integrates well with the existing content before committing the changes. Looking forward to your advice. Best Regards

Please refer to the links in my comment of the corresponding issue.

- Removed "metric=..." details from TOC and introduction. - Renamed "Loading Data" to "Example Dataset." - Deleted redundant Introduction section.

…uggested.

"For a comprehensive overview of all available distance metrics in aeon, see the aeon distances API reference."

"AgglomerativeClustering is, as the name suggests, an agglomerative approach that works by merging clusters bottom-up." Clarified Supported Linkage Methods: Included the supported linkage methods (single, complete, average, weighted) for precomputed distance matrices.

…d estimators.

…subtract from 1, ensuring proper preservation of distance distribution.

…er feedback.

…istances.ipynb) in the Clustering Overview under Clustering Notebooks.

SalmanDeveloperz · 2025-01-26T18:47:38Z

Please refer to the links in my comment of the corresponding issue.

Thank you for the guidance! I have added a reference in the clustering.ipynb notebook under the Clustering Notebooks section, as suggested.

Please let me know if there are additional updates or adjustments required!

MatthewMiddlehurst · 2025-02-01T17:34:57Z

this is failing the tests currently, please click on details to see why and resolve.

SebastianSchmidl · 2025-02-01T18:36:36Z

Could you add a link to your new notebook from the Clustering-with-sklearn.cluster section of the distances/sklearn_distances-notebook, too?

…version consistency.

… and avoid confusion.

…cluster section of sklearn_distances.ipynb.

SalmanDeveloperz · 2025-02-02T10:07:01Z

Could you add a link to your new notebook from the Clustering-with-sklearn.cluster section of the distances/sklearn_distances-notebook, too?

Inserted the required reference in the Clustering-with-sklearn.cluster section of sklearn_distances.ipynb. Please advise if any further refinements are necessary.

SebastianSchmidl · 2025-02-04T10:32:28Z

I think, we also miss a link to this notebook from https://aeon-toolkit--2511.org.readthedocs.build/en/2511/examples.html#clustering

…ering section of examples.md.

SalmanDeveloperz · 2025-02-04T16:48:54Z

Added the missing link to the new notebook in the Clustering section of examples.md. Please let me know if any further changes are needed!

SalmanDeveloperz · 2025-02-14T17:47:33Z

Hello,
I hope this message finds you well. A soft reminder, I think all the required changes are done and reviewed. Please take a look and let me know if any furture modifications required.
Thanks

SebastianSchmidl · 2025-02-18T15:59:37Z

Hi @SalmanDeveloperz,
the CI is still failing. We cannot merge any PRs with failing CI jobs. Please resolve the issues.

…s issue

…c' to fix CI Job issue.

…almanDeveloperz/aeon into add-sklearn-clustering-example

SalmanDeveloperz · 2025-02-18T19:37:58Z

Hi @SebastianSchmidl,
I've carefully reviewed the CI failures and made the necessary corrections based on the aeon distances API reference and a thorough review of the implementation.
After multiple iterations and meticulous debugging, all CI checks have now passed successfully.
Looking forward to your feedback!

SebastianSchmidl

Nice work 👍

The example plots (matplotlib) are not included in the notebook output, yet. I do not know if this should happen automatically or not.
@MatthewMiddlehurst Are the notebooks built and committed back by our nightly CI builds?

docs/examples.md

MatthewMiddlehurst · 2025-02-24T17:53:02Z

No this is not done automatically, but may be a good idea for those with no output. It would be good to run the notebook so the outputs are displayed here i agree.

https://aeon-toolkit--2511.org.readthedocs.build/en/2511/examples/clustering/sklearn_clustering_with_aeon_distances.html

SalmanDeveloperz · 2025-03-08T05:35:59Z

Hi,
The PR is updated as requested. Please review and merge if no further changes are needed. Thanks :)
Best

MatthewMiddlehurst · 2025-03-08T16:36:46Z

I see no changes since my comment. You should run the notebook so it has output visible on the website. Make sure you check the readthedocs output under checks.

@MatthewMiddlehurst

- Added outputs for Spectral and Hierarchical Clustering plots. - Fixed oversight per @MatthewMiddlehurst's feedback. - Refined code for more accurate clustering results.

…almanDeveloperz/aeon into add-sklearn-clustering-example

…istances.ipynb"

SalmanDeveloperz · 2025-03-09T03:58:27Z

Dear @MatthewMiddlehurst,

I sincerely apologize for the oversight in my previous updates. I have now resolved the issues by fixing the merge conflict and correcting the indentation in the notebook. All GitHub Actions checks have passed successfully, and the notebook outputs for both Spectral and Hierarchical Clustering are now visible as requested. I’ve also verified the readthedocs output under checks. Kindly review the changes, and please let me know if any further adjustments are needed. Thank you for your guidance and patience throughout this process :)
Best

Add example notebook for using aeon distances with sklearn clusterers

eccd899

aeon-actions-bot bot added the examples Example notebook related label Jan 22, 2025

SebastianSchmidl reviewed Jan 25, 2025

View reviewed changes

SebastianSchmidl assigned SalmanDeveloperz Jan 25, 2025

SebastianSchmidl added documentation Improvements or additions to documentation clustering Clustering package distances Distances package labels Jan 25, 2025

SalmanDeveloperz added 8 commits January 26, 2025 21:40

Resolved conflicts and updated notebook:

714be28

- Removed "metric=..." details from TOC and introduction. - Renamed "Loading Data" to "Example Dataset." - Deleted redundant Introduction section.

Simplified dataset loading using load_unit_test(split="train") as s…

fd26c31

…uggested.

Added this sentence after the introductory line:

450b0cc

"For a comprehensive overview of all available distance metrics in aeon, see the aeon distances API reference."

Changes made:-

49d8e66

"AgglomerativeClustering is, as the name suggests, an agglomerative approach that works by merging clusters bottom-up." Clarified Supported Linkage Methods: Included the supported linkage methods (single, complete, average, weighted) for precomputed distance matrices.

Added links to the scikit-learn documentation pages for all reference…

1ce463e

…d estimators.

Updated distance-to-similarity conversion to normalize distances and …

fb04db0

…subtract from 1, ensuring proper preservation of distance distribution.

Removed the references section as it was not cited in the notebook, p…

0c3435b

…er feedback.

Added a reference to the new notebook (sklearn_clustering_with_aeon_d…

b033484

…istances.ipynb) in the Clustering Overview under Clustering Notebooks.

MatthewMiddlehurst requested review from chrisholder and TonyBagnall February 1, 2025 17:34

SalmanDeveloperz added 3 commits February 2, 2025 14:42

Updated the aeon distances API reference link to a relative link for …

187e64f

…version consistency.

Removed duplicate "Hierarchical Clustering" header to improve clarity…

60a1950

… and avoid confusion.

Added a reference to the new notebook in the Clustering-with-sklearn.…

83862dc

…cluster section of sklearn_distances.ipynb.

SalmanDeveloperz requested a review from SebastianSchmidl February 3, 2025 12:14

SebastianSchmidl and others added 3 commits February 4, 2025 16:28

Merge branch 'main' into add-sklearn-clustering-example

f015b46

Automatic pre-commit fixes

4e5adde

Added a reference to the new sklearn clustering notebook in the Clust…

db35469

…ering section of examples.md.

SalmanDeveloperz and others added 6 commits February 18, 2025 23:04

Fix: Corrected DTW metric in aeon pairwise_distance to resolve CI job…

d4e57a4

…s issue

changing matric to method for dtw, trying to resolved the CI Jobs issue.

b86069b

Automatic pre-commit fixes

da602a4

Fix AgglomerativeClustering error by replacing 'affinity' with 'metri…

df7287d

…c' to fix CI Job issue.

Merge branch 'add-sklearn-clustering-example' of https://github.com/S…

3dc8397

…almanDeveloperz/aeon into add-sklearn-clustering-example

Automatic pre-commit fixes

566d15d

SebastianSchmidl reviewed Feb 18, 2025

View reviewed changes

docs/examples.md Show resolved Hide resolved

SalmanDeveloperz and others added 3 commits February 19, 2025 20:59

Added sklearn clustering example image

bf35035

Added sklearn clustering example image

ed052ee

Automatic pre-commit fixes

fb45d29

MatthewMiddlehurst changed the title ~~Add example notebook for using aeon distances with sklearn clusterers~~ [DOC] Add example notebook for using aeon distances with sklearn clusterers Feb 25, 2025

SalmanDeveloperz requested a review from SebastianSchmidl February 26, 2025 10:15

SalmanDeveloperz and others added 5 commits March 9, 2025 08:09

Display notebook outputs and improve clustering accuracy

145d64f

- Added outputs for Spectral and Hierarchical Clustering plots. - Fixed oversight per @MatthewMiddlehurst's feedback. - Refined code for more accurate clustering results.

Merge branch 'add-sklearn-clustering-example' of https://github.com/S…

6cb7d79

…almanDeveloperz/aeon into add-sklearn-clustering-example

resolve conflict

68860e6

git commit -m "Fix IndentationError in sklearn_clustering_with_aeon_d…

50ca0e8

…istances.ipynb"

Automatic pre-commit fixes

91562a7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DOC] Add example notebook for using aeon distances with sklearn clusterers #2511

[DOC] Add example notebook for using aeon distances with sklearn clusterers #2511

SalmanDeveloperz commented Jan 22, 2025

review-notebook-app bot commented Jan 22, 2025

aeon-actions-bot bot commented Jan 22, 2025

SalmanDeveloperz commented Jan 22, 2025

TonyBagnall commented Jan 24, 2025

SebastianSchmidl commented Jan 25, 2025

SalmanDeveloperz commented Jan 26, 2025

MatthewMiddlehurst commented Feb 1, 2025

SebastianSchmidl commented Feb 1, 2025

SalmanDeveloperz commented Feb 2, 2025

SebastianSchmidl commented Feb 4, 2025

SalmanDeveloperz commented Feb 4, 2025

SalmanDeveloperz commented Feb 14, 2025

SebastianSchmidl commented Feb 18, 2025

SalmanDeveloperz commented Feb 18, 2025

SebastianSchmidl left a comment

MatthewMiddlehurst commented Feb 24, 2025

SalmanDeveloperz commented Mar 8, 2025

MatthewMiddlehurst commented Mar 8, 2025

SalmanDeveloperz commented Mar 9, 2025

[DOC] Add example notebook for using aeon distances with sklearn clusterers #2511

Are you sure you want to change the base?

[DOC] Add example notebook for using aeon distances with sklearn clusterers #2511

Conversation

SalmanDeveloperz commented Jan 22, 2025

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Does your contribution introduce a new dependency? If yes, which one?

Any other comments?

PR checklist

For all contributions

For new estimators and functions

For developers with write access

review-notebook-app bot commented Jan 22, 2025

aeon-actions-bot bot commented Jan 22, 2025

Thank you for contributing to aeon

PR CI actions

SalmanDeveloperz commented Jan 22, 2025

TonyBagnall commented Jan 24, 2025

SebastianSchmidl commented Jan 25, 2025

SalmanDeveloperz commented Jan 26, 2025

MatthewMiddlehurst commented Feb 1, 2025

SebastianSchmidl commented Feb 1, 2025

SalmanDeveloperz commented Feb 2, 2025

SebastianSchmidl commented Feb 4, 2025

SalmanDeveloperz commented Feb 4, 2025

SalmanDeveloperz commented Feb 14, 2025

SebastianSchmidl commented Feb 18, 2025

SalmanDeveloperz commented Feb 18, 2025

SebastianSchmidl left a comment

Choose a reason for hiding this comment

MatthewMiddlehurst commented Feb 24, 2025

SalmanDeveloperz commented Mar 8, 2025

MatthewMiddlehurst commented Mar 8, 2025

SalmanDeveloperz commented Mar 9, 2025

Thank you for contributing to `aeon`