Commit Graph

749 Commits (imputation_and_cleaning)

Author SHA1 Message Date
junos 32025cbd8c Start with a tibble from CSV. 2021-11-29 17:51:07 +01:00
junos 181e4f0118 Add parameters to yaml file.
And use these in the prepare_participants_file function.
2021-11-29 16:57:50 +01:00
junos ab84109d55 Prepare a function to compile participants data.
It combines functions from container.R
2021-11-24 19:07:56 +01:00
junos f9863ec622 Fix small mistakes. 2021-11-24 19:01:30 +01:00
junos c1f56c61e8 Add a function to pull start and end datetimes. 2021-11-24 18:33:06 +01:00
junos 3acf6ece14 Add a function to pull device IDs. 2021-11-24 18:23:53 +01:00
junos 8b2717122d Add a function to get participants' IDs. 2021-11-24 18:05:17 +01:00
Meng Li 5bad3eb8b5
Data cleaning (#166)
* Refactor data cleaning module: move it from example workflow to main directory

* Replace NAs with 0 in selected event-based features

* Add one step to drop highly correlated features

Co-authored-by: Weiyu <weiyuhuang7@gmail.com>
2021-11-19 10:34:36 -05:00
Meng Li 296960f425 Fix the bug of location doryab features when a participant is moving during the whole time segment 2021-11-18 18:42:19 -05:00
Meng Li 3d34036eae
Add firststeptime and laststeptime features to FITBIT_STEPS_INTRADAY RAPIDS provider (#168)
* Add firststeptime and laststeptime features to FITBIT_STEPS_INTRADAY RAPIDS provider

* Update test config files
2021-11-18 18:35:27 -05:00
junos ed193d2290 Revert "Correct the name of a field."
This reverts commit b335561a55.

It was actually correct.
2021-11-17 19:16:35 +01:00
junos b335561a55 Correct the name of a field. 2021-11-17 18:50:06 +01:00
junos fcec3e2f93 Implement the necessary functions for PSQL. 2021-11-17 18:49:25 +01:00
junos 7a1e4f7139 Add the format file copied from MySQL. 2021-11-17 18:46:16 +01:00
Meng Li f340b89c58 Temporary revert PHONE_LOCATIONS BARNETT provider to use R script 2021-09-23 18:16:13 -04:00
Meng Li a3fb718aea Refactor PHONE_LOCATIONS DORYAB provider to compute features based on location episodes 2021-09-23 17:40:06 -04:00
Meng Li a8a178486b Refactor PHONE_CALLS RAPIDS provider to compute features based on call episodes or events 2021-09-15 10:28:37 -04:00
JulioV 2e553dc9e7 Add tqdm package to environment.yaml 2021-08-16 11:04:03 -04:00
Meng Li 3ac12e7dad Fix the bug of step intraday features when INCLUDE_ZERO_STEP_ROWS is False 2021-08-11 12:40:40 -04:00
Weiyu 35eebe8a51 Bug fixed: set ratiovalidyielded mins/hours value to the range 0 to 1 2021-08-09 17:56:29 -04:00
JulioV 3e69966c91 Update error message 2021-08-04 15:33:02 -04:00
Shirley 4ddb2845a6 Update initialize_params 2021-08-04 15:33:02 -04:00
JulioV 834bd3b93d Refactor in Python of Barnett provider
Co-authored-by: Shirley Hayati <sahayati@ucdavis.edu>
Co-authored-by: JulioV <JulioV@users.noreply.github.com>
2021-08-04 15:33:02 -04:00
Weiyu 7f1c502ea0 Fixed bug: Added local_segment column if no data left after filtered 2021-08-04 11:06:31 -04:00
Hannah Roberts b52059b027 Ensure date/time format is maintained
Within the 'determine which is home' for loop, 'xx' is the midpoint of two datetime objects. When the midpoint is calculated to be midnight, only the date is returned. This can be replicated with:

mydates <- as.POSIXct("2018-01-01 00:00:00", tz = "UTC")
mydates
[1] "2018-01-01 UTC"

This results in 'hourofday' being NA as an hour cannot be found. By adding the suggested format wrapper, the time is maintained and 'hourofday' can be determined. It can then successfully be applied to the embedded if-statement within the loop.

mydates <- format(as.POSIXct("2018-01-01 00:00:00", tz = "UTC"), "%Y-%m-%d %H:%M:%S")
mydates
[1] "01-01-2018 00:00:00"
2021-07-23 10:12:11 -04:00
Weiyu 5a465873c4 Tested fitbit heartrate intraday feature 2021-07-21 10:24:02 -04:00
JulioV 6fa1875bf3 Add app foreground episode count 2021-07-01 16:20:16 -04:00
JulioV bc5c0c9a4f Fix app episode length bug 2021-07-01 16:20:16 -04:00
JulioV 065a926a87 Change own to custom categories name 2021-07-01 16:20:16 -04:00
JulioV e74c745f86 Add own categories to app foreground features 2021-07-01 16:20:16 -04:00
JulioV 5892b6d838 Fix create_participants_files.R to handle numeric PIDs 2021-07-01 16:20:16 -04:00
Meng Li 97ef8a8368 Set color range and avoid SettingWithCopyWarning 2021-06-29 09:50:19 -04:00
Meng Li 1c57320ab3 Update segment labels and fix the bug when we do not have any labels for event segments 2021-06-29 09:49:24 -04:00
Meng Li cefcb0635b Update heatmap of recorded phone sensors 2021-06-29 09:49:24 -04:00
Meng Li bc06477d89 Update heatmap of sensor row count 2021-06-29 09:49:24 -04:00
Meng Li e98a8ff7ca Update histogram of phone data yield 2021-06-29 09:49:24 -04:00
Meng Li f436f1f530 Update heatmap of correlation matrix 2021-06-29 09:49:23 -04:00
Meng Li 4d37696158 Update heatmaps of overall data yield 2021-06-29 09:48:30 -04:00
Weiyu f374c67bd5 Bug fixed: Added unknown activity case 2021-06-23 19:04:55 -04:00
Weiyu 3e4d167adc Bug fixed: sort bt_address alphabetically before picking the most frequent bt_address 2021-06-22 17:40:00 -04:00
Meng Li f248b6c97d Fix bugs of Fitbit mutation scripts 2021-06-11 18:18:33 -04:00
kirtirajk 4b8698a4c6 adding app_episode with the changes as mentioned in the comments 2021-06-10 14:17:56 -04:00
Weiyu 65d5cb7bd4 Bug fixed: countscansmostuniquedevice stays the same for all time segments 2021-06-10 10:49:22 -04:00
JulioV e123a14082 Improve aware_csv msg when CSV files don't exist 2021-06-01 10:57:17 -04:00
Meng Li 9687081fbe Refactor the rule phone_locations_add_doryab_extra_columns 2021-05-28 09:48:36 -04:00
Meng Li 0d6f51be8b Refactor location features from Doryab provider & add a new strategy to infer home location & fix bugs 2021-05-26 17:36:52 -04:00
JulioV 32472461ec - Fix bug when no phone data yield is needed to process location data
- Remove location rows with the same timestamp based on their accuracy
2021-05-26 14:04:29 -04:00
Nikunj Goel 9b21196f35
Fixed `expected_minutes` to account for different time segments. (#136) 2021-05-26 11:44:48 -04:00
Meng Li edf71e055d Add the EXCLUDE_SLEEP module for steps intraday features 2021-05-21 15:23:21 -04:00
Nikunj Goel 5e451f99b0
Added phone keyboard features including docs/tests (#134) 2021-05-21 11:45:27 -04:00
JulioV e9cd9c94d7 Fix PID matching when joining data from participants 2021-05-11 16:49:04 -04:00
JulioV 32818a4802 Fix parse of pids with more than 1 devices 2021-05-11 16:42:20 -04:00
Meng Li 809845143f Test & fix bugs of sleep intraday features 2021-04-27 14:40:14 -04:00
Meng Li 7c7f34ec45 Test & fix bugs of sleep summary features 2021-04-27 14:40:14 -04:00
Meng Li 50fe09cfac Update data streams mutation of fitbit data 2021-04-27 14:40:14 -04:00
Meng Li 66d9a9d640 Update params & docs of sleep features 2021-04-27 14:33:19 -04:00
JulioV 4beafd233d Fix crash when scraping data for an app that does not exist 2021-04-22 14:28:52 -04:00
JulioV ea8094e028 Fix length of periodic segments on days with DLS 2021-04-22 11:32:10 -04:00
JulioV 9c56422529 Add calories intraday features 2021-04-20 12:00:38 -04:00
Meng Li 00a3335623 Add device_id column for sleep intraday episodes 2021-04-08 11:21:28 -04:00
JulioV 286d317af4 Fix crash when there are no periodic segments to assign
This includes a simplification of how periodic segments are computed based on all local dates in the data independently of their time zones
2021-04-07 12:03:25 -04:00
JulioV 9551669d47 Fix periodic segments bug when there are no segments to assign 2021-04-06 20:29:30 -04:00
Meng Li 78173c54ab Convert date time object to string in assign_tz_code() function 2021-04-06 23:28:53 +00:00
JulioV 1025e6d9d8 Fix datetime labels of event segments across multiple tzs 2021-04-06 13:58:58 -04:00
Meng Li 8909876cff Add local_segment column for phone data yield features 2021-04-05 21:13:36 +00:00
Meng Li 68125dc1bf Fix the bug of phone data yield features when the input is empty 2021-04-05 20:57:05 +00:00
JulioV 46f5e24814 Fix Fitbit tz inference from phone data 2021-04-05 11:51:57 -04:00
JulioV 636b64c61a Revert "Added more keyboard features."
This reverts commit 94c72e3172.
2021-04-05 11:25:00 -04:00
Meng Li 68e12a2563 Fix bugs of bluetooth feature extraction when number of unique bt_address is 2 2021-04-05 14:09:50 +00:00
Meng Li 8414977331 Fix the bug of utils.py when one participant have multi timezones 2021-04-03 21:20:10 -04:00
nikunjgoel95 94c72e3172 Added more keyboard features. 2021-04-01 20:54:13 -04:00
Meng Li 1ea5b74eff Fix the bug of utils.py when one participant have multi timezones 2021-03-31 19:34:14 -04:00
Meng Li 136dfef56b Fix the bug of Analysis Workflows while parsing targets with updated segments 2021-03-30 16:41:50 -04:00
JulioV 99dae079d5 Add iOS BT and Wifi visible to formats for old devices 2021-03-30 15:32:50 -04:00
JulioV 30ad3cd586 Validate participant files without device ids 2021-03-28 15:29:08 -04:00
JulioV 87fbbbe402 Refactor and simplify time segments 2021-03-28 15:29:07 -04:00
JulioV c48c1c8f24 Optimize Barnett's computation multi-day segments 2021-03-28 15:29:07 -04:00
JulioV d0858f8833 Fix overlapping periodic time segments 2021-03-28 15:29:07 -04:00
Meng Li e177aa6386 Update create participant files section 2021-03-25 11:39:31 -04:00
Meng Li a5eb535126 Update visualizations docs & add time flag for heatmap of overall data yield 2021-03-23 21:40:55 -04:00
nikunjgoel95 cfc5039918 Fixed the empty dataframe case in infer_home_locations.py and added array condition in doryab location 2021-03-19 11:15:57 -04:00
Meng Li 294d84277d Fix bug of sleep intraday PRICE provider when the dataframe of a segment is empty 2021-03-17 15:21:31 -04:00
JulioV 771c14a928 Improve mysql containers error messages 2021-03-16 20:02:44 -04:00
JulioV 6e234f7951 Fix warn instead of stop when there are not device ids 2021-03-16 20:02:16 -04:00
JulioV 4c2f60fffd Fix bugs in readable datetime and screen episodes 2021-03-16 20:01:43 -04:00
JulioV bb737237d0 Fixes for aware_influxdb 2021-03-16 11:26:46 -04:00
JulioV 4528ab3641 Replace SRC LANGUAGE and FOLDER with SCRIPT 2021-03-14 22:14:13 -04:00
JulioV f4b2bd1fb2 Cleanup data/ 2021-03-14 13:36:22 -04:00
JulioV 42cee67664 Add aware_influxdb in beta 2021-03-14 13:33:43 -04:00
JulioV 3d4a04effe Refactor testing 2021-03-14 00:09:08 -05:00
Meng Li 2b6447105a Migrate analysis example to new data stream 2021-03-12 19:52:34 -05:00
JulioV fae0c2ac05 Swap TABLE for CONTAINER 2021-03-12 18:14:49 -05:00
Meng Li d529490999 Migrate fitbit features to new data stream 2021-03-12 12:38:36 -05:00
JulioV 6e898beca5 Add aware_csv 2021-03-11 19:32:11 -05:00
JulioV 13174b0c2a Fix a bug when fitbit data is empty 2021-03-11 14:51:16 -05:00
JulioV 2ee45995f2 Update config docs and create participant files script 2021-03-11 14:40:33 -05:00
JulioV 1e66dad838 Fix bug in empatica_zip container script 2021-03-11 14:39:26 -05:00
JulioV a79997e0ac Add empatica_zip docs 2021-03-11 14:39:26 -05:00
JulioV d48194fc07 Add fitbitparsed_csv 2021-03-11 14:39:26 -05:00
JulioV b97b70e3a1 Add fitbitjson_csv 2021-03-11 14:39:26 -05:00
JulioV 470f4276af Add fitbitparsed_mysql 2021-03-11 14:39:26 -05:00
JulioV 1b0ee4bbf0 Add sleep intraday to fitbitjson_mysql 2021-03-11 14:39:20 -05:00
JulioV a420f5ef92 Add sleep summary to fitbitjson_mysql 2021-03-11 14:37:22 -05:00
Meng Li 93baff9f83 Migrate phone keyboard sensor to new data stream 2021-03-11 14:37:22 -05:00
Meng Li 35968e2fd0 Migrate phone log sensor to new data stream 2021-03-11 14:37:22 -05:00
Meng Li 091f9c048a Migrate phone apps notifications sensor to new data stream 2021-03-11 14:37:22 -05:00
Meng Li b49dab0949 Migrate phone apps crashes sensor to new data stream 2021-03-11 14:37:22 -05:00
JulioV 47e1b33816 Add hr intraday to fitbitjson_mysql 2021-03-11 14:37:22 -05:00
JulioV 47f449555a Add hr summary to fitbitjson_mysql 2021-03-11 14:37:22 -05:00
JulioV 72f6b2d621 Add steps intraday to fitbitjson_mysql 2021-03-11 14:37:22 -05:00
JulioV 9a276c1c66 Add steps summary to jsonfitbit_mysql 2021-03-11 14:37:22 -05:00
Meng Li 13290cd444 Migrate phone wifi visible sensor to new data stream 2021-03-11 14:36:52 -05:00
Meng Li d42c6e9c91 Migrate phone wifi connected sensor to new data stream 2021-03-11 14:35:34 -05:00
Meng Li 4825962361 Migrate phone screen sensor to new data stream 2021-03-11 14:35:34 -05:00
Meng Li bd4f647d37 Migrate phone messages sensor to new data stream 2021-03-11 14:35:34 -05:00
Meng Li 3a65b3864d Migrate phone locations sensor to new data stream 2021-03-11 14:35:34 -05:00
Meng Li 7a50a52a9d Migrate phone light sensor to new data stream 2021-03-11 14:35:34 -05:00
Meng Li c1682d8cd3 Migrate phone calls sensor to new data stream 2021-03-11 14:35:34 -05:00
Meng Li 0e96f39599 Migrate phone bluetooth sensor to new data stream 2021-03-11 14:35:34 -05:00
Meng Li 6d06d2b1eb Update MUTATION structure code & docs for AR, Apps foreground, and battery sensors 2021-03-11 14:35:34 -05:00
JulioV 6970954358 Change MUTATION structure 2021-03-11 14:35:34 -05:00
JulioV 58ef276179 Add stream_parameters arg to phone and empatica mutation scripts 2021-03-11 14:35:34 -05:00
JulioV 1063b4ca65 Add steps summary to fitbitjson_mysql 2021-03-11 14:35:34 -05:00
Meng Li f7cf316133 Migrate phone battery sensor to new data stream 2021-03-11 14:35:34 -05:00
Meng Li 824523e32c Migrate phone apps foreground sensor to new data stream 2021-03-11 14:35:34 -05:00
Meng Li 6b13c80e40 Fix bug while checking OS in stream_schema 2021-03-11 14:35:34 -05:00
JulioV 8c79cfc56f Fix OS specific dependcies 2021-03-11 14:35:34 -05:00
JulioV 7372fca0dd Fix bug when sensor is not available for an OS 2021-03-11 14:35:34 -05:00
JulioV 41711fcdb7 Rename download_data add support for py containers 2021-03-11 14:35:34 -05:00
JulioV 4b33ee43ba Replace .env with credentials.yaml 2021-03-11 14:35:34 -05:00
JulioV f65e3c8b1a Migrate empatica sensors to new data stream 2021-03-11 14:35:34 -05:00
Meng Li 2eae84ff05 Add docs of AR & conversation sensors 2021-03-11 14:35:34 -05:00
JulioV fb054b539f Add support for multiple time zones 2021-03-11 14:35:34 -05:00
JulioV 6c51c6c239 Move phone_conversation to aware_mysql stream 2021-03-11 14:35:34 -05:00
JulioV ab1a3dbf79 Move phone_activity_recognition to aware_mysql stream 2021-03-11 14:35:34 -05:00
JulioV dc11cb593d Add support for smartphone sources and schemas.
Initial support for accelerometer
Update docs for automatically create participants
Update docs for initial multiple time zones
2021-03-11 14:35:33 -05:00
Meng Li aac87311e8 Update socialjetlag feature of sleep intraday: replace bedtime with midpoint sleep 2021-03-04 15:49:25 -05:00
Meng Li 7b4598357d Update the PRICE provider's example in sleep intraday docs 2021-02-26 17:47:01 -05:00
Meng Li 716ff3c592 Fix PRICE provider's bug when input is an empty dataframe 2021-02-26 17:47:01 -05:00
Meng Li d74196cab4 Add sleep intraday features with PRICE provider 2021-02-26 17:47:01 -05:00
Meng Li bedf7106e5 Fix sleep episodes bug 2021-02-26 17:47:01 -05:00
Meng Li 8377c12efb Add sleep intraday features with RAPIDS provider 2021-02-26 17:47:01 -05:00
JulioV 724027e383 Small fixes to timeathome docs, add config validation 2021-02-24 17:49:22 -05:00
nikunjgoel95 3d6caea6c4 Added the timeathome feature using infer_home_location.py as interim file. 2021-02-24 16:57:25 -05:00
JulioV 9668dfac7a Update docs to support Empatica 2021-02-21 17:32:41 -05:00
JulioV faefca8b9a Fix extra index column when dataset is empty 2021-02-21 17:32:41 -05:00
JulioV 2e46f56111 Empatica zips must be placed in pid folder and small fixes 2021-02-21 17:32:41 -05:00
Joe Kim a26a44819a Add stats features for empatica bvp, eda, ibi, temp
Fix Snakefile file indentention
2021-02-21 17:32:41 -05:00
Joe Kim 4469cfd6bb add stats features for bvp, eda, ibi, temp 2021-02-21 17:32:41 -05:00
JulioV c6dc7e675a Add stats features for empatica heartrate
Turn off all empatica compute features
2021-02-21 17:32:41 -05:00
JulioV 3bb0230bac Add statistic features for empatica accelerometer 2021-02-21 17:32:41 -05:00
Juseong Kim 5f5f19866f implement extract_empatica_data script
add support for all data types

Fix name comparison of zipped files
2021-02-21 17:32:41 -05:00
JulioV 4b9857562b Add support for zip input files 2021-02-21 17:32:41 -05:00
JulioV 8c726f5d4f Start empatica support 2021-02-21 17:32:41 -05:00
JulioV d8813e2d04 Fix bug when any of the rows from any sensor do not belong a time segment 2021-02-09 14:51:54 -05:00
Meng Li b67f990816 Add new `FITBIT_DATA_YIELD` `RAPIDS` provider 2021-02-02 18:30:21 -05:00
nikunjgoel95 9b248c449d Fixing and adding MAXIMUM_ROW_DURATION. 2021-02-02 11:38:13 -05:00
nikunjgoel95 e7fc8f44f2 Removing Sampling Frequency and fixing ROG, location entropy and normalized location entropy. 2021-02-02 11:38:08 -05:00
nikunjgoel95 0bbf15f52e Fixed the features dependent on time duration. 2021-02-02 11:36:14 -05:00
Meng Li 3d0d062491 Fix HR summary bug: do not consider rows with restinghr=0 2021-02-01 17:29:30 -05:00
Meng Li f83c5a585e Fix HR intraday bug: minutesonZONE features are 0 2021-02-01 13:57:12 -05:00
JulioV aefc794274 Fix location processing when certain columns don't exist 2021-02-01 11:49:22 -05:00
abhineethreddyk dbf57f43f3 Updated battery feature and its testing for periodic 2021-01-31 20:24:43 -05:00
Meng Li 25a3492eba Drop rows without "assigned_segments" column before feature extraction 2021-01-21 19:41:17 -05:00
Meng Li 797de54b34 Fix merge bug of fetch_provider_features() function 2021-01-21 14:58:31 -05:00
Meng Li 5f60aac5c8 Fix KeyError bug of parsing steps data 2021-01-20 11:26:28 -05:00
JulioV d0fe4d4c28 Add ALL_RESAMPLED flag and accuracy limit 2021-01-14 14:34:25 -05:00
JulioV 38fadbf202
Feature/doryab location clustering (#111)
* Added OPTICS -  lightweight clustering algorithm.

* Changed the error message for inconsistent parameters in CONFIG

* Removing hardcoded values and changing default EPS value in the clustering algorithm.

* Added Observation in Doryab Feature docs.

Co-authored-by: nikunjgoel95 <nikunjgoel2009@gmail.com>
2021-01-14 14:22:51 -05:00
JulioV b7ba3c6407
Feature/location doryab fix (#109)
* Fixing the doryab location features for context of clustering.

* Fixed the wrong shifting while calculating the distance.

* Refractoring the haversine function

* Removed comments.

* Cleaning parts of the code.

* Updated the documentation for CLUSTER_ON parameter.

Co-authored-by: nikunjgoel95 <nikunjgoel2009@gmail.com>
2021-01-07 16:20:46 -05:00
JulioV 4926497ae2 Fix bugs in Fitbit data parsing
- Fix the script that was breaking with an empty file
- Fix the script that was breaking when start/end dates were empty
- Ambigous and nonexistent DST times are handled now
- Remove unnecessary else clause
2021-01-06 11:43:01 -05:00
JulioV 5203aa60d1 Fix bugs in create participants files script
- The PHONE and FITBIT flags were mixed up
- The start/end dates from the CSV file weren't being parsed correctly
2021-01-06 11:14:15 -05:00
JulioV 3a80f93771 Fix segment error when device ids is empty 2021-01-06 11:12:10 -05:00
JulioV 51c7739bfc Revert "Start support for phone_keyboard"
This reverts commit dd95b4f941.
2020-12-20 16:29:40 -05:00
JulioV dd95b4f941 Start support for phone_keyboard 2020-12-14 13:42:22 -05:00
JulioV 63202c62cc Fix BT doryab bug and update change log 2020-12-12 17:11:48 -05:00
JulioV 05627296f4 Fix filter_data_by_segment bug 2020-12-12 17:10:59 -05:00
JulioV 5bd1bfe856 Add new bluetooth doryab features and deprecate rapids provider 2020-12-12 17:01:46 -05:00
JulioV 266dd28d02 Add bluetooth doryab features 2020-12-11 12:03:22 -05:00
JulioV ae21c076cd Fix conversation data bug 2020-12-04 15:23:08 -05:00
JulioV 9a02e93b25 Swap RMySQL for RMariaDB 2020-12-04 14:57:13 -05:00
Meng Li 9a0e57301b Update 4 plots for time_segments 2020-12-03 21:00:32 -05:00
JulioV 614e759551 Refactor day segments to time segments 2020-12-02 18:41:03 -05:00
JulioV c24cba8679 Event segments' device id can be one of any assigned to a participant 2020-12-02 17:17:55 -05:00
Meng Li 0bd43c139c Update plot#1: histogram_phone_data_yield 2020-12-01 14:21:33 -05:00
Meng Li 016bdbfe8c Update Python feature scripts to add sensor and provider names automatically 2020-11-30 14:42:19 -05:00
Meng Li 70991d6667 Add dates filter while parsing fitbit data 2020-11-30 12:34:14 -05:00
Meng Li 5178be585d Rename modeling.py to modelling.py & Update example_config.yaml 2020-11-25 22:35:38 -05:00
JulioV 5f51c94ac6 Separate device data configuration and update docs 2020-11-25 19:42:11 -05:00
Meng Li b4a512faf3 Add analysis example workflow 2020-11-25 16:34:05 -05:00
JulioV ced3305ddb Update R feature scripts to add sensor and provider names automatically 2020-11-25 14:49:42 -05:00
JulioV f02ca2624d Add phone yield sensor 2020-11-24 19:12:16 -05:00
JulioV 555811211d Fix column check in day segment compute 2020-11-24 14:51:06 -05:00
Meng Li 767e838c87 Fix HR&Sleep bugs 2020-11-24 13:43:24 -05:00
Meng Li deba6b9e4f Update sleep summary features for segments 2020-11-23 12:01:00 -05:00
Meng Li 10384204a1 Update Fitbit HR feature name 2020-11-20 11:31:16 -05:00
Meng Li d3241c79f1 Update filter_data_by_segment() function: call chunk_episodes() inside the filter function 2020-11-19 17:27:53 -05:00
JulioV 895b70719e Update minimal example and fix create participants bug 2020-11-17 17:27:48 -05:00
Meng Li a71efd6b85 Split FITBIT_STEPS into FITBIT_STEPS_SUMMARY and FITBIT_STEPS_INTRADAY 2020-11-11 21:16:48 -05:00
Meng Li 9fc36f67e2 Split FITBIT_HEARTRATE into FITBIT_HEARTRATE_SUMMARY and FITBIT_HEARTRATE_INTRADAY 2020-11-11 17:27:46 -05:00
Meng Li 39632c66f5 Update steps features for segments 2020-10-30 18:57:04 -04:00
JulioV 7f7eac0769 Merge branch 'ds2' into day_segments 2020-10-28 17:53:00 -04:00
JulioV c5197b6c5f Merge branch 'master' into ds2 2020-10-28 17:50:37 -04:00
Meng Li 93055dacee Update HR main.py 2020-10-28 17:02:21 -04:00
JulioV f36a7052b3 Add config update file to create participants file 2020-10-28 14:11:41 -04:00
Meng Li 84068ab33c Update fitbit_parse_steps.py: add summary data 2020-10-28 13:07:35 -04:00
Meng Li ed7585c2bf Update heartrate features for segments 2020-10-27 17:55:57 -04:00
JulioV d5931c75d8 Refactor script to create participants files 2020-10-27 17:13:16 -04:00
Meng Li 25e1f1fbb5 Update Python chunk_episodes 2020-10-26 18:47:57 -04:00
JulioV b986599226 Add support to read fitbit data from csv files 2020-10-26 17:17:53 -04:00
JulioV f0a551164a Fix AR names 2020-10-26 15:44:21 -04:00
JulioV e10e030baf Unify AR iOS walking and running conversion 2020-10-26 15:44:09 -04:00
JulioV 234588c94d Expose episode threshold between rows parameter 2020-10-26 15:28:19 -04:00
JulioV 2fe5a0d822 Update R chunk_episodes 2020-10-26 15:10:32 -04:00
JulioV c78ccfced7 Remove microseconds from chuncked date times 2020-10-26 15:03:31 -04:00
Meng Li 2b7bd0ae6e Modify output format of chunk_episodes() function 2020-10-26 13:00:53 -04:00
JulioV 1d04aa6807 Add check for non-overlapping event day segments 2020-10-23 12:15:26 -04:00
JulioV 86509207ac Turn off warnings for tidyverse and dplyr 2020-10-23 10:41:00 -04:00
JulioV c41d24df45 Fix bug when there are no event segments for participants 2020-10-22 16:41:26 -04:00
JulioV 47adb8bc4b Add tz support to fitbit 2020-10-22 14:38:40 -04:00
JulioV cff83a7ceb Implement parse fitbit data 2020-10-22 13:08:52 -04:00
JulioV c266f6dd10 Update participant files structure and fitbit download rule 2020-10-20 19:12:01 -04:00
Meng Li 8c0f6a000d Fix chunk_episodes() bugs: set segment_start_timestamp as int 2020-10-19 19:36:26 -04:00
JulioV 24bf62a7ab Update file names 2020-10-19 15:07:12 -04:00
Meng Li ac9cf92732 Add accelerometer features of PANDA provider 2020-10-13 17:54:53 -04:00
Meng Li 29dcd1f284 Add accelerometer features of RAPIDS provider 2020-10-13 11:42:36 -04:00
nikunjgoel95 dcf1255b4b Fix runtime bug in Radius of Gyration of Doryab Location. 2020-10-10 13:26:06 -04:00
JulioV 3996e0ab17 Fix bug when episodes are empty 2020-10-09 17:00:20 -04:00
JulioV 2c7beec1da Fix NAs are now 0 in some calls and msgs features 2020-10-08 12:23:54 -04:00
Meng Li 236b1cd809 Update AR module for segments; Refactor input format 2020-10-07 18:11:06 -04:00
JulioV 6b1e006b82 Refactor fused location resampling 2020-10-07 11:51:31 -04:00
nikunjgoel95 80ff3f4229 Cleaning the Doryab location features code to fix the bug of Emoty dataframe in ClusterAndLabel 2020-10-01 11:34:41 -04:00
JulioV c0c32d9f9e Fix bug in assign date segment when file is empty 2020-09-30 14:53:59 -04:00
Meng Li bccc9a0697 Move deduplicate_episodes() function into chunk_episodes() function; rename "time_diff" with duration 2020-09-29 18:05:25 -04:00
JulioV 39f6e3841c Update AR episode computation 2020-09-29 17:26:43 -04:00
JulioV 10b0386c8f Merge branch 'day_segments' of https://github.com/carissalow/rapids into day_segments 2020-09-29 17:24:41 -04:00
Meng Li f1717e59e7 Update screen&battery episodes features with different segment format 2020-09-29 17:13:34 -04:00
JulioV 499b92e240 Fix missing R library 2020-09-29 15:35:44 -04:00
JulioV 0dafdd1340 Redo resample episodes in R, new battery episodes 2020-09-29 15:18:48 -04:00
JulioV a6b99259f7 Fix bug when filtering by day segment and there are no rows belonging to that segment 2020-09-28 15:53:38 -04:00
nikunjgoel95 4f64f7eeef Fix the bug on ClusterAndLabel when recieving empty data. 2020-09-28 15:02:03 -04:00
JulioV 9e15f46fc3 Update day segment format 2020-09-28 11:38:47 -04:00
JulioV 303d1472ed Change how we compute AR episodes 2020-09-22 17:32:51 -04:00
JulioV a851a460e4 Change the way we compute battery episodes 2020-09-22 16:53:51 -04:00
JulioV 47ab0a2ce5 Fix phone_sensed_bins when there is no data 2020-09-21 10:28:42 -04:00
Meng Li f806cb44ac Fix the bug of screen duration features for different segments 2020-09-18 20:25:29 -04:00
kaguillera ca14eba8e1 Initial testing restructure for 30mins segments 2020-09-18 19:14:07 -04:00
JulioV 3022a60fe9 Fix day segment error with no rows 2020-09-18 17:19:11 -04:00
JulioV 3c27bb2d18 Refactor frequency segments and fix periodic labels 2020-09-18 16:29:48 -04:00
JulioV 06bb2b9c73 Fix accuracy bug in barnett location features 2020-09-18 11:26:10 -04:00
JulioV 4dc8d38c66 Optimise assign to day segment 2020-09-16 14:54:20 -04:00