20748890a8
Further refactor by moving helper functions.
2021-09-15 15:14:54 +02:00
28699a0fdf
Enable reading features from csv files.
2021-09-14 17:42:34 +02:00
af9e81fe40
Document the SensorFeatures class and its __init__ method.
2021-09-13 17:43:47 +02:00
b19eebbb92
Refactor machine_learning/pipeline.py by defining one class by file.
2021-09-13 11:41:57 +02:00
c1bb4ddf0f
Save calculated features to csv files.
2021-08-23 16:36:26 +02:00
0152fbe4ac
Delete the leftover class.
...
Add more prints.
2021-08-23 16:09:23 +02:00
3611fc76f7
Fill NaNs after merging all features.
2021-08-21 19:48:57 +02:00
ee30c042ea
Fill NaNs introduced in merge for proximity.
2021-08-21 19:40:42 +02:00
a71e132edf
Prepare the first full pipeline.
2021-08-21 19:04:09 +02:00
24c4bef7e2
Print some more messages.
2021-08-21 19:03:44 +02:00
11381d6447
Add some print statements for monitoring progress.
2021-08-21 18:54:02 +02:00
d19995385d
Account for the case when there is no data for days with labels.
2021-08-21 18:49:57 +02:00
f73f86486a
Fill communication features with appropriate values.
2021-08-21 18:28:22 +02:00
aed73bb7ed
Add fill values for communication for rows with no calls/smses.
2021-08-21 18:17:58 +02:00
8507ff5761
Check for NaNs in the data, since sklearn.LinearRegression cannot handle them.
2021-08-21 17:46:00 +02:00
0b85ee8fdc
Merge branch 'master' into ml_pipeline
2021-08-21 17:37:45 +02:00
e2e268148d
Fill in 0.5 for undefined ratio.
...
When there are no calls and no smses (of a particular type), the ratio is undefined. But since their number is the same, I argue that the ratio can represent that with a 0.5, similarly to the case where no_calls_all = no_sms_all != 0.
2021-08-21 17:33:31 +02:00
00015a3b8d
Fill in zeroes when joining or unstacking.
...
If there are no calls or smses for a particular day, there is no corresponding row in the features dataframe. When joining these, however, NaNs were introduced. Since a value of 0 is meaningful for all of these features, replace NaNs with 0's.
2021-08-21 17:31:15 +02:00
065cd4347e
[WIP] Add a class for model validation.
2021-08-20 19:44:50 +02:00
0b98d59aad
Aggregate labels using grouping_variable.
2021-08-20 19:17:22 +02:00
08fdec34f1
Merge features into a common df.
...
But first, group communication by the grouping_variable.
2021-08-20 17:59:00 +02:00
72b16af75c
Make group_by consistent with communication.
2021-08-20 17:52:31 +02:00
d6337e82ac
Merge branch 'master' into ml_pipeline
2021-08-20 17:43:53 +02:00
9a319ac6e5
Add an option to group on other than just participant_id.
2021-08-20 17:41:12 +02:00
6592612db7
Add a similar class for labels.
2021-08-19 17:44:04 +02:00
97c693d252
Add a getter for communication data.
2021-08-19 17:36:26 +02:00
93f136b080
Add a method to get communication features.
2021-08-19 17:32:02 +02:00
5be3e82797
Accept nested feature configuration.
...
To do this, pass a dict as parameters to SensorFeatures class, rather than actually reading the object from yaml file.
2021-08-19 17:23:23 +02:00
429aa43bd1
Add communication features to pipeline.
2021-08-19 17:05:44 +02:00
0ed34e97b3
Convert the class into a YAML object.
...
Add an example config file and demonstrate its usage in ex_ml_pipeline.ipynb.
2021-08-19 16:31:42 +02:00
52664eb40b
Implement getters.
2021-08-19 11:47:59 +02:00
de92e1309d
Merge branch 'master' into ml_pipeline
2021-08-18 17:30:36 +02:00
777e6f0a58
calls_sms_features() now returns all communication features.
2021-08-18 15:41:47 +02:00
2d78aacd18
Compile a list of contact features and add a test.
2021-08-18 15:35:42 +02:00
c88336481e
Add a test for SMS features.
2021-08-18 15:28:46 +02:00
1bc996413e
Clarify names for no_all calls/sms feature.
...
Add another test.
2021-08-18 15:23:30 +02:00
a2a44c202a
Calculate common features outside if...else.
2021-08-18 10:54:54 +02:00
4740e94d37
Fix a bug introduced in e7fe4e8398 .
2021-08-18 10:51:48 +02:00
b1ad8d1309
List calls features.
2021-08-17 16:27:34 +02:00
bb75abcb9b
Add tests for proximity.
2021-08-17 16:07:52 +02:00
e7fe4e8398
Simplify merge into join.
2021-08-17 13:53:19 +02:00
cf28aa547a
Merge branch 'communication' into 'master'
...
separated features
See merge request junoslukan/straw2analysis!2
2021-08-17 11:42:03 +00:00
4d73b9d5ff
Add tests for proximity.
2021-08-17 10:51:51 +02:00
3821314dd9
[WIP] Separate the features part from the pipeline.
2021-08-16 18:11:25 +02:00
d6f36ec8f8
[WIP] Finish the class by assigning columns and validating model.
2021-08-13 17:41:04 +02:00
b06ec6e1ae
[WIP] Methods to get the labels and data plus aggregate them.
2021-08-12 19:07:14 +02:00
622477f19f
[WIP] Start merging steps into a class for a pipeline.
2021-08-12 17:38:08 +02:00
577a874288
Add an example for linear regression.
2021-08-12 16:54:00 +02:00
c8bb481508
Add a parameter for grouping.
2021-08-12 15:07:20 +02:00
98f1df81c6
Use the same function for ESM and other data.
2021-08-11 17:26:44 +02:00
junos