Model Tuner 0.0.27b

What's Changed

• Xgboost n estimators fix by @elemets in #186
• Google colab compatibility - pandas version by @elemets in #187

Full Changelog: 0.0.26b...0.0.27b

Model Tuner 0.0.26b

Model Tuner Version 0.0.26b Changelog

Release Notes Summary

• Optimal threshold: Users can now specify target precision or recall and an optimal threshold is computed for that
• Finalised testing: coverage is now at 86% total
• New get_feature_names() helper function for extracting features
• n_estimators calculation for boosting algorithms is now fixed

What's Changed

• Fix: AttributeError in get_feature_selection_pipeline with RFE and calibrate=True by @lshpaner in #144
• Fix: Incorrect Iteration Over score String in calibrateModel by @lshpaner in #146
• Add Printout of Rebalanced Classes in KFold Outputs by @lshpaner in #148
• Fix Inaccurate Confusion Matrix Counts Across Folds by @lshpaner in #150
• Pp tests ii by @panas89 in #153
• AF Tests by @elemets in #154
• GroupKFold by @elemets in #155
• added new logic for calc. fold metrics in report_model_metrics by @lshpaner in #156
• Pp tests iii by @panas89 in #159
• Account for Threshold Tuning in K-Fold Classification and Metrics Reporting by @lshpaner in #162
• Updated return_dict logic in return_metrics by @lshpaner in #165
• updated dict logic in return_metrics by @lshpaner in #168
• Pp tests iv by @panas89 in #167
• Modified aggregated_y_pred to aggregated_y_prob inside brier_score_loss by @lshpaner in #164
• updated bin class. test; (+) multilabel test by @lshpaner in #169
• AF tests IV by @elemets in #170
• updated threshold print logic by @lshpaner in #172
• added unittest for catboost_n_iterations, upd. (+1) inside model_tuner… by @lshpaner in #175
• Tests for feature selection pipeline by @elemets in #177
• Pp tests v by @panas89 in #176
• added shap script by @lshpaner in #178
• Calibrate report remove by @elemets in #179
• Adding tests for early stopping fit with and without a score by @elemets in #180
• Calculate Metrics by @elemets in #181
• Move logo to different script; streamlined by @lshpaner in #182
• added new method and script for threshold optimization by @lshpaner in #184
• get_feature_names by @elemets in #185

Full Changelog: 0.0.24a...0.0.26b

Model Tuner 0.0.25a

Model Tuner Version 0.0.25a Changelog

Release Notes for Model Tuner (Version 0.0.25a)

• Pushed fixes for the get_feature_selection_pipeline method.
• Updated scoring blocks for calibrated KFold models and folded confusion matrix metrics.
• Moved import statements to the top for unit tests.
• Added unittests for edge cases, including test_rfe_calibrate_model() and validating confusion matrix alignment.
• Fixed mismatches between confusion matrix and classification report.
• Provided fixes for all pipeline getter methods.
• Integrated verify_imb_sampler prints into KFold logic.
• Resolved issues in KFold confusion matrix output and updated corresponding unit tests.
• Added test coverage for selected best features and threshold-based F-beta tuning tests.
• Enhanced tests for KFold splits and removed unnecessary utilities.
• Introduced tests for conf_mat_class_kfold and initiated randomized search tests.
• Finished randomized grid tests and improved GroupKFold support.
• Resolved typos in group split configurations and refined nested KFold bug fixes.
• Adjusted fold metric calculations in report_model_metrics.
• Refined RFE logic with additional prints and threshold adjustment tests.
• Implemented bootstrap regression tests and probability predictions with KFold scenarios.
• Moved optimal threshold logic into prediction functionality.
• Updated tests to accommodate new return_metrics_kfold logic.
• Enhanced return metrics dictionary logic to handle all cases and added multilabel classification tests.
• Addressed Brier score calculation issues and optimized regression reports for KFold.
• Introduced threshold print updates for clearer reporting.
• Developed unittests for CatBoost best iteration logic and improved test coverage for feature selection post-calibration.
• Implemented SHAP scripts and tests for model explainability.
• Removed outdated calibration reports from documentation and codebase.
• Added tests for early stopping during fitting processes with and without scoring mechanisms.
• Fixed bugs in regression metric calculations and refined KFold metric aggregation.

Model Tuner 0.0.24a

Model Tuner Version 0.0.24a Changelog

• Updated .gitignore to incl. doctrees

• Added pickleObjects tests and updated reqs, tests passed

• Added boostrapper test and tests passed

• Adding multi class test script

• Updated Metrics Output

  • Added optl' threshold print inside return_metrics
  • KFold metric printing:
    • Added new input per_fold_print to allow user to return per fold metrics, otherwise average
    • Added tqdm output for KFold metric printing
    • Fixed KFold avg output in report_model_metrics

• Added a regression test and updating report_model_metrics to work with regression and multi class

• Augmented predict_proba test, and train_val_test_split

• Fixed pipeline_steps arg in model definition

• Refactored metrics_df in report_model_metrics for aesthetics

• Unit Tests

  • XGB early stopping multi class test
  • Added fit method tests
  • Added early stopping test
  • Added get_best_score_params() tests
  • Added return_bootstrap_metrics() tests
  • Added tests for get_preprocessing_and_feature_selection_pipeline and get_feature_selection_pipeline
  • Tested init, passed tests
  • Tested get_preprocessing_pipeline

• Imbalance Sampler

  • Addded process_imbalance_sampler() tests, passed
  • Renamed process_imbalance_sampler() to verify_imbalance_sampler

• Made return_dict optional in return_metrics

• Added openpyxl versions for all python versions in requirements.txt

• Refactor metrics, foldwise metrics and foldwise con_mat, class_labels

• Cleaned notebooks dir

• Renamed notebooks to py_example_scripts, linted files, cleaned code

• Added model_tuner version print to scripts

• Added fix for sort of pipelinesteps now optional:

  • Added required model_tuner import to xgb_multi.py
  • Added requisite model_tuner import to multi_class_test.py

• Added catboost_multi_class.py script

• Removed pip dependency from requirements

Model Tuner 0.0.23a

• Fixed a bug found when calibrating early stopping models
• Fixed early stopping in Column Transformer application

Model Tuner 0.0.22a

• Fixed an issue where the feature selection name was not referenced correctly, causing a bug when printing selected feature names with the updated pipeline.
• Removed resolved print statements from April, 2024.

Model Tuner 0.0.21a

• Specified the pipeline class otherwise the method just returned a list
• Removed need to specify self.estimator when its called
• Generalized (renamed) "K Best Features" to just "Best Features" inside returns of return_metrics()
• Generalized (renamed) k_best_features to best_features

Model Tuner 0.0.20a

• Added flexibility between boolean and None for stratification inputs
• Added custom exception for non pandas inputs in return_bootstrap_metrics
• Enforced required model_type input to be specified as "classification" or "regression"
• Removed extraneous "=" print below pipeline_steps
• Handled missing pipeline_steps when using imbalance_sampler
• Updated requirements for python==3.11
• Fixed SMOTE for early stopping
• Removed extra model_type input from xgb_early_test.py

Model Tuner 0.0.19a

• Requirements updated again to make compatible with google colab out of the box.
• Bug in fit() method where best_params wasn't defined if we didn't specify a score
• Threshold bug now actually fixed. Specificity and other metrics should reflect this. (Defaults to 0.5 if optimal_threshold is not specified).

Model Tuner 0.0.18a

• Updated requirements to include numpy versions <1.26 for Python 3.8-3.11.

This should stop a rerun occurring when using the library on a google colab.