Commit Graph

52 Commits (c88cecc063c5443837111be980bf6e7e6ed2b05d)

Author SHA1 Message Date
Primoz 10ca47583c Implement feature selection methods (WIP). 2023-04-14 17:20:22 +02:00
Primoz bccc1cd1de Clean and fix Preprocessing module. 2023-02-23 10:40:58 +01:00
Primoz 9ed863b7a1 Add a CrossValidation module with all the required methods. 2023-02-23 10:40:17 +01:00
Primoz f69cb25266 Add planning comments. 2023-02-22 18:12:52 +01:00
Primoz 7f6ae9b323 Add imputation and One-Hot Encoding Methods. 2023-02-22 18:05:01 +01:00
Primoz 8f6cb3f444 Add preprocessing class. 2023-02-22 13:44:03 +01:00
Primoz ef12f64fe5 Add feature selection Class skeleton. 2023-02-20 11:51:34 +01:00
junos 852e17afbe Define classification models method. 2022-12-08 10:00:14 +01:00
junos 8131626c4a Include more metrics in regression helper methods. 2022-12-07 21:25:05 +01:00
junos 95ab66fd81 Move to presentation. 2022-12-07 21:24:20 +01:00
junos 12f2c927fa Merge branch 'ml_pipeline' of repo.ijs.si:junoslukan/straw2analysis into ml_pipeline
# Conflicts:
#	exploration/ml_pipeline_daily.py - deleted
2022-12-07 15:36:52 +01:00
junos 71e1fcf8ca Save results. 2022-12-07 15:33:18 +01:00
Primoz 9a218c8e2a Add a script for two class train test split clustering classification. 2022-11-25 14:44:11 +01:00
Primoz 98f78d72fc Create a classification models class and use it in the ml pipeline script. 2022-11-25 12:35:45 +01:00
junos ae2d7a038d Present results. 2022-11-16 21:36:43 +01:00
junos 389198b17f Prepare data in a separate step.
Change categorical features.
2022-11-16 19:34:18 +01:00
junos c462d55096 Update function with imputation already handled. 2022-11-16 18:18:12 +01:00
junos a5c09a292f Move function to helper.py 2022-11-16 18:13:30 +01:00
junos e33a49c9fc Add a demo of pipeline. 2021-11-17 10:44:49 +01:00
junos 005b09cfdf [WIP] Fix tests to use pyprojroot. 2021-10-29 12:07:12 +02:00
junos a63a7eac99 [WIP] Add a test for SensorFeatures.
Additional analysis for adherence.
Small corrections.
2021-10-13 13:39:58 +02:00
junos b8c7606664 Add an option to read cached labels from a file. 2021-09-15 15:45:49 +02:00
junos ed062d25ee Add export capabilities to labels.py. 2021-09-15 15:36:36 +02:00
junos 20748890a8 Further refactor by moving helper functions. 2021-09-15 15:14:54 +02:00
junos 28699a0fdf Enable reading features from csv files. 2021-09-14 17:42:34 +02:00
junos af9e81fe40 Document the SensorFeatures class and its __init__ method. 2021-09-13 17:43:47 +02:00
junos b19eebbb92 Refactor machine_learning/pipeline.py by defining one class by file. 2021-09-13 11:41:57 +02:00
junos c1bb4ddf0f Save calculated features to csv files. 2021-08-23 16:36:26 +02:00
junos 0152fbe4ac Delete the leftover class.
Add more prints.
2021-08-23 16:09:23 +02:00
junos 3611fc76f7 Fill NaNs after merging all features. 2021-08-21 19:48:57 +02:00
junos ee30c042ea Fill NaNs introduced in merge for proximity. 2021-08-21 19:40:42 +02:00
junos a71e132edf Prepare the first full pipeline. 2021-08-21 19:04:09 +02:00
junos 24c4bef7e2 Print some more messages. 2021-08-21 19:03:44 +02:00
junos 11381d6447 Add some print statements for monitoring progress. 2021-08-21 18:54:02 +02:00
junos d19995385d Account for the case when there is no data for days with labels. 2021-08-21 18:49:57 +02:00
junos f73f86486a Fill communication features with appropriate values. 2021-08-21 18:28:22 +02:00
junos 8507ff5761 Check for NaNs in the data, since sklearn.LinearRegression cannot handle them. 2021-08-21 17:46:00 +02:00
junos 065cd4347e [WIP] Add a class for model validation. 2021-08-20 19:44:50 +02:00
junos 0b98d59aad Aggregate labels using grouping_variable. 2021-08-20 19:17:22 +02:00
junos 08fdec34f1 Merge features into a common df.
But first, group communication by the grouping_variable.
2021-08-20 17:59:00 +02:00
junos 6592612db7 Add a similar class for labels. 2021-08-19 17:44:04 +02:00
junos 97c693d252 Add a getter for communication data. 2021-08-19 17:36:26 +02:00
junos 93f136b080 Add a method to get communication features. 2021-08-19 17:32:02 +02:00
junos 5be3e82797 Accept nested feature configuration.
To do this, pass a dict as parameters to SensorFeatures class, rather than actually reading the object from yaml file.
2021-08-19 17:23:23 +02:00
junos 429aa43bd1 Add communication features to pipeline. 2021-08-19 17:05:44 +02:00
junos 0ed34e97b3 Convert the class into a YAML object.
Add an example config file and demonstrate its usage in ex_ml_pipeline.ipynb.
2021-08-19 16:31:42 +02:00
junos 52664eb40b Implement getters. 2021-08-19 11:47:59 +02:00
junos 3821314dd9 [WIP] Separate the features part from the pipeline. 2021-08-16 18:11:25 +02:00
junos d6f36ec8f8 [WIP] Finish the class by assigning columns and validating model. 2021-08-13 17:41:04 +02:00
junos b06ec6e1ae [WIP] Methods to get the labels and data plus aggregate them. 2021-08-12 19:07:14 +02:00