[ENH] Softmax activation function added

[KowshikB03]

Fixes Issue #2593

[aeon-actions-bot]

Thank you for contributing to aeon

I did not find any labels to add based on the title. Please add the [ENH], [MNT], [BUG], [DOC], [REF], [DEP] and/or [GOV] tags to your pull requests titles. For now you can add the labels manually.
I have added the following labels to this PR based on the changes made: [ $\color{#BCAE15}{\textsf{classification}}$ ]. Feel free to change these if they do not properly represent the PR.

The Checks tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our pre-commit code quality check fails, any trivial fixes will automatically be pushed to your PR unless it is a draft.

Don't hesitate to ask questions on the aeon Slack channel if you have any.

PR CI actions

These checkboxes will add labels to enable/disable CI functionality for this PR. This may not take effect immediately, and a new commit may be required to run the new configuration.

• Run pre-commit checks for all files
• Run mypy typecheck tests
• Run all pytest tests and configurations
• Run all notebook example tests
• Run numba-disabled codecov tests
• Stop automatic pre-commit fixes (always disabled for drafts)
• Disable numba cache loading
• Push an empty commit to re-run CI checks

[hadifawaz1999]

Thank you for this but i left a comment on the corresponding issue on how this is not a bug

[MatthewMiddlehurst]

Hi, I agree with Ali here if these are the published defaults then I don't really want to change them unless there is a publication showing it is better.

@hadifawaz1999 a warning or documentation change should be fine though if you think this would produce better results in certain scenarios? If not or it's unknown, keep it as it is. Up to you really.

[hadifawaz1999]

not sure how worth it it would be, its not really the best model at the end of the day, with or without softmax, so would just keep it as is and not add confusion to users

[MatthewMiddlehurst]

ok will leave it up for a bit for others to comment but likely declined