Commit Graph

131 Commits (ml_pipeline)

Author SHA1 Message Date
Primoz 865225994b Added testing section after feature selection. 2023-04-20 13:29:14 +02:00
Primoz 0594993133 Add GroupKFold to feature selection CV. Start with generic metric calculation procedure. 2023-04-20 11:20:26 +02:00
Primoz 1cbc743cf7 Add kBest method to initially filter out the worst performing features. Update comments. 2023-04-20 10:12:16 +02:00
Primoz ce13a9e13b Implement feature selection method which is used in ML pipeline. 2023-04-19 15:56:34 +02:00
Primoz 8a532fa95a Add a ML pipeline script to develop a whole pipeline. 2023-02-23 10:41:36 +01:00
Primoz ef12f64fe5 Add feature selection Class skeleton. 2023-02-20 11:51:34 +01:00
Primoz 63741c3627 Add feature selection pipeline script with initail plan. 2023-02-15 12:27:39 +01:00
Primoz 806ca1b37d Added excel intermediate scores file 2023-02-06 14:02:24 +01:00
Primoz d06da18c26 Fix error with method name. 2023-02-06 11:54:54 +01:00
Primoz 93a34986d9 Comment code sections and change to pd.concatinate method. 2023-02-06 11:31:21 +01:00
Primoz 08e81610a9 Fix method params 2023-02-06 11:21:17 +01:00
Primoz ef78f179cd Add neccessary parameters to plot method. 2023-02-06 11:19:04 +01:00
Primoz afeb7b4872 Remove data_yield from features. 2023-02-06 11:16:53 +01:00
Primoz ea3f805ba7 Change ML model and ddd CV to sequential feat_select. Add std lines to plots. 2023-02-06 11:09:15 +01:00
Primoz e3aef2dae7 Add save to file code, and todo comment 2023-02-01 15:13:57 +01:00
Primoz b286753696 Add vizualization sections for sequential addition of sensors' features. 2023-02-01 13:51:56 +01:00
Primoz 07ef72dec5 Implement algorithm for sequential adding of the most important features. 2023-01-25 14:19:29 +01:00
Primoz 85e572fca0 Expand analysis of the features (individualy and by sensor groups). 2023-01-23 16:32:07 +01:00
Primoz 6a98c8cdcf Prepare scripts for feature importance analysis. 2023-01-19 16:20:43 +01:00
Primoz d263b32564 Temp: remove stratified logo from ml pipeline. 2023-01-19 09:26:55 +01:00
Primoz ad2fab133f Explore features with Entropy and IG. 2023-01-13 17:08:56 +01:00
junos 72fdd9c5ec Use stratified downsampling.
And run all models with a method from machine_learning.helper.
2023-01-04 21:48:37 +01:00
junos b0b9edccc4 Unhide jupyter code cells and outputs. 2023-01-04 21:25:12 +01:00
junos 8bbe0b2ba8 Merge branch 'ml_pipeline' of repo.ijs.si:junoslukan/straw2analysis into ml_pipeline
# Conflicts:
#	.gitignore
#	exploration/ml_pipeline_classification_with_clustering.py
2023-01-04 18:19:43 +01:00
junos 1516d1c000 Cluster by demand_control_ratio_quartile. 2023-01-04 18:16:14 +01:00
Primoz 339142ff31 Add expl stress event script and other changes. 2022-12-21 15:02:25 +01:00
Primoz adcb823d3f Add stress event duration exploration script. 2022-12-15 16:43:40 +01:00
Primoz a61ab9ee51 Add feature importance check. 2022-12-15 16:43:13 +01:00
Primoz 164d12ed2f Add undersampling method (with on/off parameter). 2022-12-13 17:01:46 +01:00
Primoz 0a45e35164 Remove unused imports prt. 2. 2022-12-09 13:56:42 +01:00
Primoz 78b6e7fa07 Remove unused imports from ML pipeline scripts. 2022-12-09 13:53:16 +01:00
Primoz 6507b053c5 Add StrtifiedKFold with shuffling as a default CV method. 2022-12-09 13:46:13 +01:00
junos 525496418f Configure jupytext. 2022-12-07 16:22:41 +01:00
junos 12f2c927fa Merge branch 'ml_pipeline' of repo.ijs.si:junoslukan/straw2analysis into ml_pipeline
# Conflicts:
#	exploration/ml_pipeline_daily.py - deleted
2022-12-07 15:36:52 +01:00
Primoz cf0e4f89be Test nonstandardized data with regular classification pipeline. 2022-11-29 14:06:06 +01:00
Primoz 7504aa34cf Add additional categorical features (uncomment). 2022-11-28 13:42:46 +01:00
Primoz 9a218c8e2a Add a script for two class train test split clustering classification. 2022-11-25 14:44:11 +01:00
Primoz 98f78d72fc Create a classification models class and use it in the ml pipeline script. 2022-11-25 12:35:45 +01:00
Primoz 218b684514 Automize clustering classification logic and add parameters at the begining of the scripts. General changes and improvements. 2022-11-24 16:12:20 +01:00
Primoz ddde80b421 Add classification with clustering ml pipeline script. 2022-11-24 09:24:13 +01:00
Primoz 7afef5582f Add TEMP lime_survey cols 2022-11-22 14:44:33 +01:00
Primoz 183758cd37 Improve general ml classification pipeline script. 2022-11-22 14:31:49 +01:00
Primoz 40029a8205 Add a script for ml classification pipeline. 2022-11-21 14:47:19 +01:00
Primoz ae0f54ecc2 Combine different segment scripts and set ml pipeline as a regression problem. 2022-11-21 11:41:11 +01:00
Primoz 8defb271c9 Extend ml pipeline scripts with two additional CV methods. 2022-11-21 11:23:47 +01:00
junos 848416bf6a Merge branch 'master' into ml_pipeline
# Conflicts:
#	.gitignore
#	rapids
2022-11-16 17:46:01 +01:00
Primoz b59798df26 Add a new file tailored for stressfulness event regression. 2022-11-16 14:49:40 +01:00
Primoz 1d8dcf8b21 Add 30 min features data and modify script. 2022-11-02 15:16:19 +01:00
Primoz 9f7fa0c8e0 Add 18 hour daily data and slightly modify jupyter script. 2022-10-18 10:29:59 +02:00
Primoz 466cd3dc23 Processing of a newly cleaned script. Addition of two ML models. And modifications with one hot encoding. 2022-10-10 16:47:00 +02:00