578f5ed5f3
Start preparing the true usernames CSV file.
2021-12-01 11:29:22 +01:00
6f9b513ba0
Make the participants rule run.
2021-11-29 18:37:50 +01:00
c2b27aef45
Add yaml files to submodule.
2021-11-24 19:12:36 +01:00
4bb01ba871
Add functions to prepare participants files.
2021-11-24 19:09:06 +01:00
080dc1d1fe
Add instructions for RAPIDS.
2021-11-18 17:22:08 +01:00
4e87664fd6
Force UNIX eol characters.
2021-11-17 19:20:19 +01:00
c0ea17dbe3
Update rapids.
2021-11-17 18:58:12 +01:00
1d65a31d8c
Update rapids submodule.
2021-11-17 12:59:12 +01:00
d4ba701202
Add the rapids fork submodule.
...
Reproducible Analysis Pipeline for Data Streams
2021-11-17 12:47:42 +01:00
e33a49c9fc
Add a demo of pipeline.
2021-11-17 10:44:49 +01:00
d34c2ec5e9
Merge branch 'ambient' into ml_pipeline
2021-11-17 10:39:55 +01:00
005b09cfdf
[WIP] Fix tests to use pyprojroot.
2021-10-29 12:07:12 +02:00
6fc0d962ae
Remove low values of pressure.
2021-10-22 18:09:17 +02:00
92fbda242b
Explore barometer and temperature data.
...
Add docstrings to models.
2021-10-14 17:59:33 +02:00
6302a0f0d9
Merge ambient sensors into one file.
...
Explore barometer sensor data for one phone.
2021-10-13 16:57:38 +02:00
a63a7eac99
[WIP] Add a test for SensorFeatures.
...
Additional analysis for adherence.
Small corrections.
2021-10-13 13:39:58 +02:00
b8c7606664
Add an option to read cached labels from a file.
2021-09-15 15:45:49 +02:00
ed062d25ee
Add export capabilities to labels.py.
2021-09-15 15:36:36 +02:00
20748890a8
Further refactor by moving helper functions.
2021-09-15 15:14:54 +02:00
28699a0fdf
Enable reading features from csv files.
2021-09-14 17:42:34 +02:00
af9e81fe40
Document the SensorFeatures class and its __init__ method.
2021-09-13 17:43:47 +02:00
b19eebbb92
Refactor machine_learning/pipeline.py by defining one class by file.
2021-09-13 11:41:57 +02:00
c1bb4ddf0f
Save calculated features to csv files.
2021-08-23 16:36:26 +02:00
0152fbe4ac
Delete the leftover class.
...
Add more prints.
2021-08-23 16:09:23 +02:00
3611fc76f7
Fill NaNs after merging all features.
2021-08-21 19:48:57 +02:00
ee30c042ea
Fill NaNs introduced in merge for proximity.
2021-08-21 19:40:42 +02:00
a71e132edf
Prepare the first full pipeline.
2021-08-21 19:04:09 +02:00
24c4bef7e2
Print some more messages.
2021-08-21 19:03:44 +02:00
11381d6447
Add some print statements for monitoring progress.
2021-08-21 18:54:02 +02:00
d19995385d
Account for the case when there is no data for days with labels.
2021-08-21 18:49:57 +02:00
f73f86486a
Fill communication features with appropriate values.
2021-08-21 18:28:22 +02:00
aed73bb7ed
Add fill values for communication for rows with no calls/smses.
2021-08-21 18:17:58 +02:00
8507ff5761
Check for NaNs in the data, since sklearn.LinearRegression cannot handle them.
2021-08-21 17:46:00 +02:00
0b85ee8fdc
Merge branch 'master' into ml_pipeline
2021-08-21 17:37:45 +02:00
e2e268148d
Fill in 0.5 for undefined ratio.
...
When there are no calls and no smses (of a particular type), the ratio is undefined. But since their number is the same, I argue that the ratio can represent that with a 0.5, similarly to the case where no_calls_all = no_sms_all != 0.
2021-08-21 17:33:31 +02:00
00015a3b8d
Fill in zeroes when joining or unstacking.
...
If there are no calls or smses for a particular day, there is no corresponding row in the features dataframe. When joining these, however, NaNs were introduced. Since a value of 0 is meaningful for all of these features, replace NaNs with 0's.
2021-08-21 17:31:15 +02:00
065cd4347e
[WIP] Add a class for model validation.
2021-08-20 19:44:50 +02:00
0b98d59aad
Aggregate labels using grouping_variable.
2021-08-20 19:17:22 +02:00
08fdec34f1
Merge features into a common df.
...
But first, group communication by the grouping_variable.
2021-08-20 17:59:00 +02:00
72b16af75c
Make group_by consistent with communication.
2021-08-20 17:52:31 +02:00
d6337e82ac
Merge branch 'master' into ml_pipeline
2021-08-20 17:43:53 +02:00
9a319ac6e5
Add an option to group on other than just participant_id.
2021-08-20 17:41:12 +02:00
6592612db7
Add a similar class for labels.
2021-08-19 17:44:04 +02:00
97c693d252
Add a getter for communication data.
2021-08-19 17:36:26 +02:00
93f136b080
Add a method to get communication features.
2021-08-19 17:32:02 +02:00
5be3e82797
Accept nested feature configuration.
...
To do this, pass a dict as parameters to SensorFeatures class, rather than actually reading the object from yaml file.
2021-08-19 17:23:23 +02:00
429aa43bd1
Add communication features to pipeline.
2021-08-19 17:05:44 +02:00
0ed34e97b3
Convert the class into a YAML object.
...
Add an example config file and demonstrate its usage in ex_ml_pipeline.ipynb.
2021-08-19 16:31:42 +02:00
52664eb40b
Implement getters.
2021-08-19 11:47:59 +02:00
de92e1309d
Merge branch 'master' into ml_pipeline
2021-08-18 17:30:36 +02:00
junos