Commit Graph

266 Commits (baa94c4c4eedcd62c209b7b1b776b4343e89bbfa)

Author SHA1 Message Date
junos baa94c4c4e Correct additional error in feature file naming.
Add the final feature file to the list in Snakefile.
2022-04-06 17:29:17 +02:00
junos cbc8ae4e03 Add necessary checks for empty data frames. 2022-04-05 18:58:09 +02:00
junos e84c35a36a Remove unnecessary parameters from preprocess_esm.
And correct the newly named interim file.
2022-04-05 18:36:09 +02:00
junos e2ce68f591 Defer creation of feature files to esm_features rule. 2022-04-05 18:30:04 +02:00
junos 751b04f3f4 Pass scale names to Snakemake correctly. 2022-04-05 18:14:37 +02:00
junos 99245afca3 Try a different approach for preprocessing ESMs.
It is important that this follows generic RAPIDS pattern.
In the subsequent step of calculating features,
there is an expected file and folder structure of data/interim.
See rules/common.smk/find_features_files()
2022-04-05 18:02:31 +02:00
junos 798ec973b4 [WIP] Add a rule for ESM features. 2022-03-30 10:43:30 +02:00
junos 7173ca13e3 Rename a parameter. 2022-03-30 10:40:53 +02:00
junos 23f0aaba3a Get the name of the questionnaire from Snakefile. 2022-03-16 18:28:57 +01:00
junos 679f00dc19 Enable selecting any questionnaire as target. 2022-03-16 17:55:44 +01:00
junos cb116100dd Move preprocessing to features. 2022-03-16 17:06:42 +01:00
junos d470eef27e Add a rule to preprocess and clean ESM. 2022-03-09 18:38:46 +01:00
junos aedb8b6785 Write questionnaire data to data/interim. 2022-03-01 12:02:36 +01:00
junos 30ac8b1cd5 Start calculating demand control features. 2022-02-23 19:08:10 +01:00
junos 176367631b Prepare baseline feature rule. 2022-02-23 11:09:33 +01:00
junos bf9c764c97 Split baseline data to participants.
And some csv I/O settings.
2022-02-04 18:37:57 +01:00
junos 16e608db74 First merge baseline datasets. 2022-02-04 18:21:42 +01:00
junos 204f6f50b0 Read the relevant files. 2022-02-04 18:06:02 +01:00
junos 685ed6a546 Set up demographic data download. 2022-02-04 17:37:00 +01:00
junos ffa7a30575 Make place for STRAW models. 2022-02-04 17:25:24 +01:00
junos b29f902915 Look into ESM table for device_id. 2021-12-15 20:18:12 +01:00
junos 5a9252e46e Merge remote-tracking branch 'origin/master' 2021-12-15 18:32:36 +01:00
junos e5cc02501f Set the timezone.csv path in config.
Take into account that TZCODES_FILE can be created with a rule.
2021-12-15 18:09:30 +01:00
junos 1f54195437 Configure timezone file to be created automatically. 2021-12-08 18:21:29 +01:00
junos 0b8a493ff2 Incorporate mulitple timezones into RAPIDS. 2021-12-01 18:20:27 +01:00
junos f0d29d0d1a Incorporate DB query for usernames into snakemake workflow. 2021-12-01 18:14:27 +01:00
junos da5ff0f36e Correct small errors in settings. 2021-11-29 18:04:06 +01:00
junos 181e4f0118 Add parameters to yaml file.
And use these in the prepare_participants_file function.
2021-11-29 16:57:50 +01:00
junos 39bd244511 [WIP] Prepare yaml files.
These will be used to create participants files.
2021-11-24 19:11:19 +01:00
Meng Li 5bad3eb8b5
Data cleaning (#166)
* Refactor data cleaning module: move it from example workflow to main directory

* Replace NAs with 0 in selected event-based features

* Add one step to drop highly correlated features

Co-authored-by: Weiyu <weiyuhuang7@gmail.com>
2021-11-19 10:34:36 -05:00
Meng Li f340b89c58 Temporary revert PHONE_LOCATIONS BARNETT provider to use R script 2021-09-23 18:16:13 -04:00
Meng Li a3fb718aea Refactor PHONE_LOCATIONS DORYAB provider to compute features based on location episodes 2021-09-23 17:40:06 -04:00
Meng Li a8a178486b Refactor PHONE_CALLS RAPIDS provider to compute features based on call episodes or events 2021-09-15 10:28:37 -04:00
JulioV 834bd3b93d Refactor in Python of Barnett provider
Co-authored-by: Shirley Hayati <sahayati@ucdavis.edu>
Co-authored-by: JulioV <JulioV@users.noreply.github.com>
2021-08-04 15:33:02 -04:00
Meng Li cefcb0635b Update heatmap of recorded phone sensors 2021-06-29 09:49:24 -04:00
Meng Li bc06477d89 Update heatmap of sensor row count 2021-06-29 09:49:24 -04:00
Meng Li e98a8ff7ca Update histogram of phone data yield 2021-06-29 09:49:24 -04:00
Meng Li f436f1f530 Update heatmap of correlation matrix 2021-06-29 09:49:23 -04:00
Meng Li 4d37696158 Update heatmaps of overall data yield 2021-06-29 09:48:30 -04:00
kirtirajk 4b8698a4c6 adding app_episode with the changes as mentioned in the comments 2021-06-10 14:17:56 -04:00
Meng Li 9687081fbe Refactor the rule phone_locations_add_doryab_extra_columns 2021-05-28 09:48:36 -04:00
Meng Li 0d6f51be8b Refactor location features from Doryab provider & add a new strategy to infer home location & fix bugs 2021-05-26 17:36:52 -04:00
JulioV 32472461ec - Fix bug when no phone data yield is needed to process location data
- Remove location rows with the same timestamp based on their accuracy
2021-05-26 14:04:29 -04:00
Meng Li edf71e055d Add the EXCLUDE_SLEEP module for steps intraday features 2021-05-21 15:23:21 -04:00
JulioV 9c56422529 Add calories intraday features 2021-04-20 12:00:38 -04:00
JulioV c48c1c8f24 Optimize Barnett's computation multi-day segments 2021-03-28 15:29:07 -04:00
JulioV d0858f8833 Fix overlapping periodic time segments 2021-03-28 15:29:07 -04:00
Meng Li a5eb535126 Update visualizations docs & add time flag for heatmap of overall data yield 2021-03-23 21:40:55 -04:00
JulioV d6c22fdbc7 Fix an import bug and docs 2021-03-15 19:35:58 -04:00
JulioV 4528ab3641 Replace SRC LANGUAGE and FOLDER with SCRIPT 2021-03-14 22:14:13 -04:00