Commit Graph

286 Commits (a8cd16f88c3ad95058b87084c35073a1cb3bcdc0)

Author SHA1 Message Date
Meng Li aac87311e8 Update socialjetlag feature of sleep intraday: replace bedtime with midpoint sleep 2021-03-04 15:49:25 -05:00
Meng Li d74196cab4 Add sleep intraday features with PRICE provider 2021-02-26 17:47:01 -05:00
Meng Li 2d5e966530 Update fitbit sleep intraday docs & config file 2021-02-26 17:47:01 -05:00
Meng Li 8377c12efb Add sleep intraday features with RAPIDS provider 2021-02-26 17:47:01 -05:00
nikunjgoel95 3d6caea6c4 Added the timeathome feature using infer_home_location.py as interim file. 2021-02-24 16:57:25 -05:00
JulioV 5abca8bb0f Missing config validations 2021-02-21 19:57:21 -05:00
Weiyu 327b015206 Add validation for config keys 2021-02-21 19:55:55 -05:00
JulioV 9668dfac7a Update docs to support Empatica 2021-02-21 17:32:41 -05:00
JulioV 2e46f56111 Empatica zips must be placed in pid folder and small fixes 2021-02-21 17:32:41 -05:00
Joe Kim 4469cfd6bb add stats features for bvp, eda, ibi, temp 2021-02-21 17:32:41 -05:00
JulioV c6dc7e675a Add stats features for empatica heartrate
Turn off all empatica compute features
2021-02-21 17:32:41 -05:00
JulioV 3bb0230bac Add statistic features for empatica accelerometer 2021-02-21 17:32:41 -05:00
Juseong Kim 5f5f19866f implement extract_empatica_data script
add support for all data types

Fix name comparison of zipped files
2021-02-21 17:32:41 -05:00
JulioV 4b9857562b Add support for zip input files 2021-02-21 17:32:41 -05:00
JulioV 8c726f5d4f Start empatica support 2021-02-21 17:32:41 -05:00
JulioV 47bd695249 Deprecate Doryab circadian movement feature until it is fixed 2021-02-03 09:46:29 -05:00
Meng Li b67f990816 Add new `FITBIT_DATA_YIELD` `RAPIDS` provider 2021-02-02 18:30:21 -05:00
JulioV 1685b5cba6 Change default value for CLUSTER_ON 2021-02-02 12:04:48 -05:00
nikunjgoel95 9b248c449d Fixing and adding MAXIMUM_ROW_DURATION. 2021-02-02 11:38:13 -05:00
nikunjgoel95 e7fc8f44f2 Removing Sampling Frequency and fixing ROG, location entropy and normalized location entropy. 2021-02-02 11:38:08 -05:00
JulioV d0fe4d4c28 Add ALL_RESAMPLED flag and accuracy limit 2021-01-14 14:34:25 -05:00
JulioV 585bf7bc5d Add code so new feature providers can be added for the new four sensors 2021-01-14 14:31:40 -05:00
JulioV 8fd1d9dc29 Add four new sensors without providers 2021-01-14 14:31:40 -05:00
JulioV 38fadbf202
Feature/doryab location clustering (#111)
* Added OPTICS -  lightweight clustering algorithm.

* Changed the error message for inconsistent parameters in CONFIG

* Removing hardcoded values and changing default EPS value in the clustering algorithm.

* Added Observation in Doryab Feature docs.

Co-authored-by: nikunjgoel95 <nikunjgoel2009@gmail.com>
2021-01-14 14:22:51 -05:00
JulioV b7ba3c6407
Feature/location doryab fix (#109)
* Fixing the doryab location features for context of clustering.

* Fixed the wrong shifting while calculating the distance.

* Refractoring the haversine function

* Removed comments.

* Cleaning parts of the code.

* Updated the documentation for CLUSTER_ON parameter.

Co-authored-by: nikunjgoel95 <nikunjgoel2009@gmail.com>
2021-01-07 16:20:46 -05:00
JulioV 51c7739bfc Revert "Start support for phone_keyboard"
This reverts commit dd95b4f941.
2020-12-20 16:29:40 -05:00
JulioV d39b700b5f Update git flow docs, set keyboard flag to false 2020-12-20 13:55:41 -05:00
JulioV dd95b4f941 Start support for phone_keyboard 2020-12-14 13:42:22 -05:00
JulioV 5bd1bfe856 Add new bluetooth doryab features and deprecate rapids provider 2020-12-12 17:01:46 -05:00
JulioV 266dd28d02 Add bluetooth doryab features 2020-12-11 12:03:22 -05:00
JulioV 083e10dc38 Comment fixes in config.yaml 2020-12-04 16:49:29 -05:00
Meng Li 9a0e57301b Update 4 plots for time_segments 2020-12-03 21:00:32 -05:00
JulioV 614e759551 Refactor day segments to time segments 2020-12-02 18:41:03 -05:00
Meng Li 0bd43c139c Update plot#1: histogram_phone_data_yield 2020-12-01 14:21:33 -05:00
JulioV 5f51c94ac6 Separate device data configuration and update docs 2020-11-25 19:42:11 -05:00
JulioV ced3305ddb Update R feature scripts to add sensor and provider names automatically 2020-11-25 14:49:42 -05:00
JulioV f02ca2624d Add phone yield sensor 2020-11-24 19:12:16 -05:00
Meng Li deba6b9e4f Update sleep summary features for segments 2020-11-23 12:01:00 -05:00
Meng Li a71efd6b85 Split FITBIT_STEPS into FITBIT_STEPS_SUMMARY and FITBIT_STEPS_INTRADAY 2020-11-11 21:16:48 -05:00
Meng Li 9fc36f67e2 Split FITBIT_HEARTRATE into FITBIT_HEARTRATE_SUMMARY and FITBIT_HEARTRATE_INTRADAY 2020-11-11 17:27:46 -05:00
JulioV 432f11fbcb Add bluetooth, calls, sms and wifi docs 2020-11-04 19:37:24 -05:00
JulioV 2aa44fc504 Change SENSOR_DATA to DEVICE_DATA 2020-11-04 17:16:46 -05:00
JulioV 1ea2730afd Update minimal workflow example docs 2020-11-04 13:27:58 -05:00
Meng Li 39632c66f5 Update steps features for segments 2020-10-30 18:57:04 -04:00
JulioV 7f7eac0769 Merge branch 'ds2' into day_segments 2020-10-28 17:53:00 -04:00
JulioV c5197b6c5f Merge branch 'master' into ds2 2020-10-28 17:50:37 -04:00
Meng Li 93055dacee Update HR main.py 2020-10-28 17:02:21 -04:00
JulioV f36a7052b3 Add config update file to create participants file 2020-10-28 14:11:41 -04:00
JulioV 8f74d3f3b7 Minor updates to config and Snakefile 2020-10-28 14:03:02 -04:00
Meng Li ed7585c2bf Update heartrate features for segments 2020-10-27 17:55:57 -04:00
JulioV d5931c75d8 Refactor script to create participants files 2020-10-27 17:13:16 -04:00
JulioV b986599226 Add support to read fitbit data from csv files 2020-10-26 17:17:53 -04:00
JulioV f1ddfae04f Fix missing params battery and AR 2020-10-26 15:51:22 -04:00
JulioV 47adb8bc4b Add tz support to fitbit 2020-10-22 14:38:40 -04:00
JulioV cff83a7ceb Implement parse fitbit data 2020-10-22 13:08:52 -04:00
JulioV c266f6dd10 Update participant files structure and fitbit download rule 2020-10-20 19:12:01 -04:00
JulioV 24bf62a7ab Update file names 2020-10-19 15:07:12 -04:00
Meng Li ac9cf92732 Add accelerometer features of PANDA provider 2020-10-13 17:54:53 -04:00
Meng Li 29dcd1f284 Add accelerometer features of RAPIDS provider 2020-10-13 11:42:36 -04:00
Meng Li 236b1cd809 Update AR module for segments; Refactor input format 2020-10-07 18:11:06 -04:00
Meng Li f1717e59e7 Update screen&battery episodes features with different segment format 2020-09-29 17:13:34 -04:00
Meng Li f806cb44ac Fix the bug of screen duration features for different segments 2020-09-18 20:25:29 -04:00
JulioV 4dc8d38c66 Optimise assign to day segment 2020-09-16 14:54:20 -04:00
JulioV 3c45f6b152 Refactor assign to day segment 2020-09-14 14:21:36 -04:00
kaguillera 441cdf69c3 Fix Travis attempt 4 2020-09-03 14:36:42 -04:00
kaguillera 67d11e449f Fix Travis attempt 3 2020-09-03 14:18:01 -04:00
kaguillera 4be8accc45 Attempt to fix Travis 2 2020-09-03 13:44:07 -04:00
kaguillera 0613bcb4b8 Attempt to fix Travis 1 2020-09-03 13:19:44 -04:00
kaguillera 5d3e6efc41 New Bluetooth Testing 2020-09-03 12:04:29 -04:00
JulioV 98ebf9bd13 Migrate conversation to new file structure 2020-09-01 17:07:07 -04:00
JulioV 8d87f6e497 Migrate app foreground to new file structure 2020-09-01 15:25:35 -04:00
JulioV e643881058 Migrate light to new file structure 2020-09-01 12:01:24 -04:00
JulioV eda58d9d2c Migrate wifi to new file structure 2020-08-31 18:51:06 -04:00
JulioV 77d9d8f025 Migrate bluetooth to new file structure 2020-08-31 17:08:26 -04:00
JulioV 132311da77 Migrate messages to new file structure 2020-08-31 13:34:15 -04:00
JulioV e269062439 Migrate calls to new provider file structure 2020-08-28 17:50:49 -04:00
JulioV b0f1477d7e Migrate location providers to new file structure and segments 2020-08-28 13:53:00 -04:00
JulioV 31ec5b0da4 Finalise new day segment structure with calls as example 2020-08-26 12:09:53 -04:00
nikunjgoel95 6a5470e338 Splitting Energy Feature in Conversation to Voice and Noise. 2020-08-23 13:20:19 -04:00
JulioV 026d056378 Simplify window workflow 2020-08-20 14:51:22 -04:00
JulioV a0b5b5982b Setup rules and files to support multiple 2020-08-20 14:45:09 -04:00
Meng Li 7a4218a3ce Delete analysis section of config.yaml 2020-08-13 17:12:05 -04:00
JulioV 3f48a47a14 Fix optional input files for phone sensed bins 2020-08-11 18:02:52 -04:00
nikunjgoel95 df7a52cd6c Added Sampling Frequency to Doryab Location Features. 2020-08-05 13:02:34 -04:00
Meng Li c2b25a146d Replace sensor name with table name in heatmap_days_by_sensors section 2020-08-03 15:19:45 -04:00
Meng Li e13b89b125 Add restore_sql_file rule; notsummarised module; diff platforms for heatmap_days_by_sensors 2020-08-03 13:09:16 -04:00
Meng Li 93157db210 Data cleaning section: replace "day_type" with "day_idx" 2020-07-27 18:27:36 -04:00
Meng Li af6678c70c Update visualization section: add MIN_VALID_BINS_PER_HOUR 2020-07-26 20:11:25 -04:00
JulioV 388f09a003 Add missing wifi table 2020-07-24 16:35:41 -04:00
Meng Li b3aa4d82e1 Overall compliance heatmap: shows all dates for all participants (only supports last certain dates previously) 2020-07-24 12:58:48 -04:00
JulioV 5f771618ae Add screen episode filter parameters 2020-07-23 14:38:59 -04:00
nikunjgoel95 faad8f5a8f Added minutes of data in a day/epoch. 2020-07-23 14:23:32 -04:00
Meng Li b0354393ae Add "corr_method" param to heatmap_features_correlations.py 2020-07-22 20:00:14 -04:00
Meng Li c135b6ef78 Add visualization for data exploration 2020-07-22 16:47:32 -04:00
Meng Li 124843f96e Make phone_valid_sensed_days rule works for different min_valid_hours_per_day thresholds 2020-07-21 20:41:24 -04:00
nikunjgoel95 0ada1292ed Added maximum_gap_allowed parameter 2020-07-21 18:56:43 -04:00
JulioV e37b07098f
Fix defaults for Doryab's location features 2020-07-20 16:03:51 -04:00
kaguillera c8d8906359 Attempt to fix travis 1.2 2020-07-20 10:32:04 -04:00
kaguillera 9deb6eae5f Attempt to fix travis 2020-07-20 10:20:20 -04:00
nikunjgoel95 024ae1a549 Added Doryab's location features to rapids.
Co-authored-by: JulioV <juliovhz@gmail.com>
2020-07-16 14:26:43 -04:00
JulioV d2704296f3 Refactor valid sensed bins and days 2020-07-09 13:01:50 -04:00
JulioV b381672cae
Update config.yaml 2020-06-29 13:50:33 -04:00
Meng Li 62f862a738 Fix default for ACCELEROMETER VALID_SENSED_MINUTES 2020-06-26 17:35:29 -04:00
Meng Li 29b04b0601 Fix step bouts features 2020-06-26 17:33:30 -04:00
kaguillera 533d0adab3 Updated sms sensor to messages 2020-06-26 12:20:47 -04:00
nikunjgoel95 0c53fd275e Required changes in config 2020-06-24 17:03:14 -04:00
JulioV 8c647b3b4c
Fix default for STEP EXCLUDE_SLEEP 2020-06-24 13:40:15 -04:00
JulioV 1bdb52dd65 Add functionality to exlcude step data based on sleep 2020-06-24 13:33:58 -04:00
Meng Li 8aa25144f0 Refactor analysis part of snakefile 2020-06-23 20:46:42 -04:00
kaguillera 7bcf674793 Refactored testing after pipeline refactoring 2020-06-23 18:29:01 -04:00
JulioV 36017d5dca Refactor Snakefile and docs. Rename SMS scripts 2020-06-23 11:33:34 -04:00
Meng Li 4d1ff10701 Rename heartrate zone features 2020-06-22 14:15:56 -04:00
Meng Li 99fe86ac2a Rename accelerometer features names 2020-06-22 13:47:25 -04:00
Meng Li a121c0fe7b Split accelerometer features in config file; drop count; add more stats features 2020-06-22 12:01:17 -04:00
Meng Li f07ba22f98 Minor fix for applications_foreground sensor 2020-06-19 16:22:57 -04:00
nikunjgoel95 1409de2832 Added Converstion Features.
Updated config.yaml, Snakefile, Features.Snakefile and documentation.
2020-06-19 01:27:28 -04:00
Meng Li e133b8b530 Refactor fitbit sleep features 2020-06-12 18:44:05 -04:00
Meng Li dcc7ca14e3 Refactor fitbit heartrate features 2020-06-12 16:04:03 -04:00
JulioV 8fbafcbe0c Fix location bug and improve fused resampling 2020-06-11 12:25:49 -04:00
Meng Li aa1baaf948 Refactor activity recognition features 2020-06-04 19:32:28 -04:00
JulioV 5656c673d3 Remove varqn and huberm features from calls 2020-06-02 19:48:22 -04:00
Meng Li 8c8378f74a Split modeling module into two rules; Add RandomOverSampler for resampling; Add log; Fix bug of AUC 2020-05-15 18:42:03 -04:00
Meng Li 915bdd04b1 Parse Fitbit summary and intraday data; Extract Fitbit daily features from summary data 2020-05-15 17:51:00 -04:00
Meng Li 7cbb227214 Add modeling module 2020-04-29 18:53:54 -04:00
Meng Li 9ddb50ed59 Split days threshold of data cleaning into days_before_surgery and days_after_discharge
Co-authored-by: JulioV <juliovhz@gmail.com>
2020-04-29 14:37:40 -04:00
Meng Li 490599c742 Implement AR features for iOS
Co-authored-by: JulioV <juliovhz@gmail.com>
2020-04-20 18:05:54 -04:00
Meng Li 5696b4f6d4 Add merge module for demographic features and target 2020-04-16 14:20:16 -04:00
Meng Li eac721de84 Add demographic_features and targets module; refactor analysis code
Co-authored-by: JulioV <juliovhz@gmail.com>
2020-04-16 12:38:28 -04:00
Meng Li 695984586f Add calories module for fitbit_readable_datetime.py 2020-04-15 16:01:02 -04:00
JulioV 010114c1aa Add wifi features 2020-04-13 13:24:52 -04:00
Mingze Cao 8cc93c8791 Refactor battery feature: replace "metrics" with "features"
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
2020-04-09 15:06:25 -05:00
Mingze Cao 2d7d3bfccf Refactor location_barnett features: replace "metrics" with "features"
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
2020-04-09 12:20:39 -05:00
Mingze Cao c2fbc3ad7a Refactor sms feature: replace "metrics" with "features"
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
2020-04-08 15:02:58 -05:00
Mingze Cao 97c83caebc Refactor fitbit_heartrate feature: replace "metrics" with "features"
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
2020-04-08 14:31:43 -05:00
Mingze Cao a53ba242f8 Refactor google_activity_recognition feature: replace "metrics" with "features"
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
2020-04-08 13:36:36 -05:00
Mingze Cao 0809001dfa Refactor light features: replace "metrics" with "features"
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
2020-04-08 13:02:53 -05:00
Mingze Cao f46e8c0666 Refactor applications_foreground features: replace "metrics" with "features"
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
2020-04-08 12:31:43 -05:00
Mingze Cao d1c38016de Refactor call features: replace "metrics" with "features"
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
2020-04-08 10:51:18 -05:00
Mingze Cao e0d19cbc1b Refactor screen features: replace "metrics" with "features"
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
2020-04-08 10:05:16 -05:00
Mingze Cao 3986f4e185 Refactor fitbit_step features: replace "metrics" with "features"
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
Co-authored-by: JulioV <juliovhz@gmail.com>
2020-04-03 16:03:45 -05:00
Mingze Cao 6045df494d Modify the Accelerometer ‘Metrics’ to Accelerometer ‘Features’
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
Co-authored-by: JulioV <juliovhz@gmail.com>
2020-04-02 16:36:28 -05:00
Mingze Cao 7d770c91f3 Add sumdurationsedentarybout of the fitbit_step sensor
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
2020-04-02 13:36:54 -05:00
JulioV 0e173872df Refactor select_days_to_analyse, fix merge bugs, add clean metrics for model 2020-03-17 21:15:53 -04:00
Meng Li 7c240a9613 Select days to analyse
Co-authored-by: JulioV <juliovhz@gmail.com>
2020-03-17 17:26:30 -04:00
JulioV 3d4c26754e Rename merge metrics for models and add filter valid sensed days
Co-authored-by: Meng Li <AnnieLM1996@gmail.com>
2020-03-12 17:31:46 -04:00
Meng Li b23bb0e402 Add INCLUDE_ZERO_STEP_ROWS flag for step sensor 2020-03-09 15:59:51 -04:00
Meng Li aba9f13332 Add merge metrics module for analysis rules 2020-03-09 13:32:14 -04:00
JulioV 301bd0bf1d
Change default database group name 2020-03-09 12:55:43 -04:00
JulioV 12ceb87090 Merge branch 'master' of https://github.com/carissalow/rapids 2020-03-05 10:32:46 -05:00
JulioV ccc283db16 Add day segment parameter to location barnett rule 2020-03-05 10:32:38 -05:00