13290cd444
Migrate phone wifi visible sensor to new data stream
2021-03-11 14:36:52 -05:00
d42c6e9c91
Migrate phone wifi connected sensor to new data stream
2021-03-11 14:35:34 -05:00
4825962361
Migrate phone screen sensor to new data stream
2021-03-11 14:35:34 -05:00
bd4f647d37
Migrate phone messages sensor to new data stream
2021-03-11 14:35:34 -05:00
3a65b3864d
Migrate phone locations sensor to new data stream
2021-03-11 14:35:34 -05:00
7a50a52a9d
Migrate phone light sensor to new data stream
2021-03-11 14:35:34 -05:00
c1682d8cd3
Migrate phone calls sensor to new data stream
2021-03-11 14:35:34 -05:00
0e96f39599
Migrate phone bluetooth sensor to new data stream
2021-03-11 14:35:34 -05:00
6d06d2b1eb
Update MUTATION structure code & docs for AR, Apps foreground, and battery sensors
2021-03-11 14:35:34 -05:00
6970954358
Change MUTATION structure
2021-03-11 14:35:34 -05:00
58ef276179
Add stream_parameters arg to phone and empatica mutation scripts
2021-03-11 14:35:34 -05:00
1063b4ca65
Add steps summary to fitbitjson_mysql
2021-03-11 14:35:34 -05:00
f7cf316133
Migrate phone battery sensor to new data stream
2021-03-11 14:35:34 -05:00
824523e32c
Migrate phone apps foreground sensor to new data stream
2021-03-11 14:35:34 -05:00
6b13c80e40
Fix bug while checking OS in stream_schema
2021-03-11 14:35:34 -05:00
8c79cfc56f
Fix OS specific dependcies
2021-03-11 14:35:34 -05:00
7372fca0dd
Fix bug when sensor is not available for an OS
2021-03-11 14:35:34 -05:00
41711fcdb7
Rename download_data add support for py containers
2021-03-11 14:35:34 -05:00
4b33ee43ba
Replace .env with credentials.yaml
2021-03-11 14:35:34 -05:00
f65e3c8b1a
Migrate empatica sensors to new data stream
2021-03-11 14:35:34 -05:00
2eae84ff05
Add docs of AR & conversation sensors
2021-03-11 14:35:34 -05:00
fb054b539f
Add support for multiple time zones
2021-03-11 14:35:34 -05:00
6c51c6c239
Move phone_conversation to aware_mysql stream
2021-03-11 14:35:34 -05:00
ab1a3dbf79
Move phone_activity_recognition to aware_mysql stream
2021-03-11 14:35:34 -05:00
dc11cb593d
Add support for smartphone sources and schemas.
...
Initial support for accelerometer
Update docs for automatically create participants
Update docs for initial multiple time zones
2021-03-11 14:35:33 -05:00
aac87311e8
Update socialjetlag feature of sleep intraday: replace bedtime with midpoint sleep
2021-03-04 15:49:25 -05:00
7b4598357d
Update the PRICE provider's example in sleep intraday docs
2021-02-26 17:47:01 -05:00
716ff3c592
Fix PRICE provider's bug when input is an empty dataframe
2021-02-26 17:47:01 -05:00
d74196cab4
Add sleep intraday features with PRICE provider
2021-02-26 17:47:01 -05:00
bedf7106e5
Fix sleep episodes bug
2021-02-26 17:47:01 -05:00
8377c12efb
Add sleep intraday features with RAPIDS provider
2021-02-26 17:47:01 -05:00
724027e383
Small fixes to timeathome docs, add config validation
2021-02-24 17:49:22 -05:00
3d6caea6c4
Added the timeathome feature using infer_home_location.py as interim file.
2021-02-24 16:57:25 -05:00
9668dfac7a
Update docs to support Empatica
2021-02-21 17:32:41 -05:00
faefca8b9a
Fix extra index column when dataset is empty
2021-02-21 17:32:41 -05:00
2e46f56111
Empatica zips must be placed in pid folder and small fixes
2021-02-21 17:32:41 -05:00
a26a44819a
Add stats features for empatica bvp, eda, ibi, temp
...
Fix Snakefile file indentention
2021-02-21 17:32:41 -05:00
4469cfd6bb
add stats features for bvp, eda, ibi, temp
2021-02-21 17:32:41 -05:00
c6dc7e675a
Add stats features for empatica heartrate
...
Turn off all empatica compute features
2021-02-21 17:32:41 -05:00
3bb0230bac
Add statistic features for empatica accelerometer
2021-02-21 17:32:41 -05:00
5f5f19866f
implement extract_empatica_data script
...
add support for all data types
Fix name comparison of zipped files
2021-02-21 17:32:41 -05:00
4b9857562b
Add support for zip input files
2021-02-21 17:32:41 -05:00
8c726f5d4f
Start empatica support
2021-02-21 17:32:41 -05:00
d8813e2d04
Fix bug when any of the rows from any sensor do not belong a time segment
2021-02-09 14:51:54 -05:00
b67f990816
Add new `FITBIT_DATA_YIELD` `RAPIDS` provider
2021-02-02 18:30:21 -05:00
9b248c449d
Fixing and adding MAXIMUM_ROW_DURATION.
2021-02-02 11:38:13 -05:00
e7fc8f44f2
Removing Sampling Frequency and fixing ROG, location entropy and normalized location entropy.
2021-02-02 11:38:08 -05:00
0bbf15f52e
Fixed the features dependent on time duration.
2021-02-02 11:36:14 -05:00
3d0d062491
Fix HR summary bug: do not consider rows with restinghr=0
2021-02-01 17:29:30 -05:00
f83c5a585e
Fix HR intraday bug: minutesonZONE features are 0
2021-02-01 13:57:12 -05:00
aefc794274
Fix location processing when certain columns don't exist
2021-02-01 11:49:22 -05:00
dbf57f43f3
Updated battery feature and its testing for periodic
2021-01-31 20:24:43 -05:00
25a3492eba
Drop rows without "assigned_segments" column before feature extraction
2021-01-21 19:41:17 -05:00
797de54b34
Fix merge bug of fetch_provider_features() function
2021-01-21 14:58:31 -05:00
5f60aac5c8
Fix KeyError bug of parsing steps data
2021-01-20 11:26:28 -05:00
d0fe4d4c28
Add ALL_RESAMPLED flag and accuracy limit
2021-01-14 14:34:25 -05:00
38fadbf202
Feature/doryab location clustering ( #111 )
...
* Added OPTICS - lightweight clustering algorithm.
* Changed the error message for inconsistent parameters in CONFIG
* Removing hardcoded values and changing default EPS value in the clustering algorithm.
* Added Observation in Doryab Feature docs.
Co-authored-by: nikunjgoel95 <nikunjgoel2009@gmail.com>
2021-01-14 14:22:51 -05:00
b7ba3c6407
Feature/location doryab fix ( #109 )
...
* Fixing the doryab location features for context of clustering.
* Fixed the wrong shifting while calculating the distance.
* Refractoring the haversine function
* Removed comments.
* Cleaning parts of the code.
* Updated the documentation for CLUSTER_ON parameter.
Co-authored-by: nikunjgoel95 <nikunjgoel2009@gmail.com>
2021-01-07 16:20:46 -05:00
4926497ae2
Fix bugs in Fitbit data parsing
...
- Fix the script that was breaking with an empty file
- Fix the script that was breaking when start/end dates were empty
- Ambigous and nonexistent DST times are handled now
- Remove unnecessary else clause
2021-01-06 11:43:01 -05:00
5203aa60d1
Fix bugs in create participants files script
...
- The PHONE and FITBIT flags were mixed up
- The start/end dates from the CSV file weren't being parsed correctly
2021-01-06 11:14:15 -05:00
3a80f93771
Fix segment error when device ids is empty
2021-01-06 11:12:10 -05:00
51c7739bfc
Revert "Start support for phone_keyboard"
...
This reverts commit dd95b4f941
2020-12-20 16:29:40 -05:00
dd95b4f941
Start support for phone_keyboard
2020-12-14 13:42:22 -05:00
63202c62cc
Fix BT doryab bug and update change log
2020-12-12 17:11:48 -05:00
05627296f4
Fix filter_data_by_segment bug
2020-12-12 17:10:59 -05:00
5bd1bfe856
Add new bluetooth doryab features and deprecate rapids provider
2020-12-12 17:01:46 -05:00
266dd28d02
Add bluetooth doryab features
2020-12-11 12:03:22 -05:00
ae21c076cd
Fix conversation data bug
2020-12-04 15:23:08 -05:00
9a02e93b25
Swap RMySQL for RMariaDB
2020-12-04 14:57:13 -05:00
9a0e57301b
Update 4 plots for time_segments
2020-12-03 21:00:32 -05:00
614e759551
Refactor day segments to time segments
2020-12-02 18:41:03 -05:00
c24cba8679
Event segments' device id can be one of any assigned to a participant
2020-12-02 17:17:55 -05:00
0bd43c139c
Update plot#1: histogram_phone_data_yield
2020-12-01 14:21:33 -05:00
016bdbfe8c
Update Python feature scripts to add sensor and provider names automatically
2020-11-30 14:42:19 -05:00
70991d6667
Add dates filter while parsing fitbit data
2020-11-30 12:34:14 -05:00
5178be585d
Rename modeling.py to modelling.py & Update example_config.yaml
2020-11-25 22:35:38 -05:00
5f51c94ac6
Separate device data configuration and update docs
2020-11-25 19:42:11 -05:00
b4a512faf3
Add analysis example workflow
2020-11-25 16:34:05 -05:00
ced3305ddb
Update R feature scripts to add sensor and provider names automatically
2020-11-25 14:49:42 -05:00
f02ca2624d
Add phone yield sensor
2020-11-24 19:12:16 -05:00
555811211d
Fix column check in day segment compute
2020-11-24 14:51:06 -05:00
767e838c87
Fix HR&Sleep bugs
2020-11-24 13:43:24 -05:00
deba6b9e4f
Update sleep summary features for segments
2020-11-23 12:01:00 -05:00
10384204a1
Update Fitbit HR feature name
2020-11-20 11:31:16 -05:00
d3241c79f1
Update filter_data_by_segment() function: call chunk_episodes() inside the filter function
2020-11-19 17:27:53 -05:00
895b70719e
Update minimal example and fix create participants bug
2020-11-17 17:27:48 -05:00
a71efd6b85
Split FITBIT_STEPS into FITBIT_STEPS_SUMMARY and FITBIT_STEPS_INTRADAY
2020-11-11 21:16:48 -05:00
9fc36f67e2
Split FITBIT_HEARTRATE into FITBIT_HEARTRATE_SUMMARY and FITBIT_HEARTRATE_INTRADAY
2020-11-11 17:27:46 -05:00
39632c66f5
Update steps features for segments
2020-10-30 18:57:04 -04:00
7f7eac0769
Merge branch 'ds2' into day_segments
2020-10-28 17:53:00 -04:00
c5197b6c5f
Merge branch 'master' into ds2
2020-10-28 17:50:37 -04:00
93055dacee
Update HR main.py
2020-10-28 17:02:21 -04:00
f36a7052b3
Add config update file to create participants file
2020-10-28 14:11:41 -04:00
84068ab33c
Update fitbit_parse_steps.py: add summary data
2020-10-28 13:07:35 -04:00
ed7585c2bf
Update heartrate features for segments
2020-10-27 17:55:57 -04:00
d5931c75d8
Refactor script to create participants files
2020-10-27 17:13:16 -04:00
25e1f1fbb5
Update Python chunk_episodes
2020-10-26 18:47:57 -04:00
b986599226
Add support to read fitbit data from csv files
2020-10-26 17:17:53 -04:00
f0a551164a
Fix AR names
2020-10-26 15:44:21 -04:00
e10e030baf
Unify AR iOS walking and running conversion
2020-10-26 15:44:09 -04:00
234588c94d
Expose episode threshold between rows parameter
2020-10-26 15:28:19 -04:00
2fe5a0d822
Update R chunk_episodes
2020-10-26 15:10:32 -04:00
c78ccfced7
Remove microseconds from chuncked date times
2020-10-26 15:03:31 -04:00
2b7bd0ae6e
Modify output format of chunk_episodes() function
2020-10-26 13:00:53 -04:00
1d04aa6807
Add check for non-overlapping event day segments
2020-10-23 12:15:26 -04:00
86509207ac
Turn off warnings for tidyverse and dplyr
2020-10-23 10:41:00 -04:00
c41d24df45
Fix bug when there are no event segments for participants
2020-10-22 16:41:26 -04:00
47adb8bc4b
Add tz support to fitbit
2020-10-22 14:38:40 -04:00
cff83a7ceb
Implement parse fitbit data
2020-10-22 13:08:52 -04:00
c266f6dd10
Update participant files structure and fitbit download rule
2020-10-20 19:12:01 -04:00
8c0f6a000d
Fix chunk_episodes() bugs: set segment_start_timestamp as int
2020-10-19 19:36:26 -04:00
24bf62a7ab
Update file names
2020-10-19 15:07:12 -04:00
ac9cf92732
Add accelerometer features of PANDA provider
2020-10-13 17:54:53 -04:00
29dcd1f284
Add accelerometer features of RAPIDS provider
2020-10-13 11:42:36 -04:00
dcf1255b4b
Fix runtime bug in Radius of Gyration of Doryab Location.
2020-10-10 13:26:06 -04:00
3996e0ab17
Fix bug when episodes are empty
2020-10-09 17:00:20 -04:00
2c7beec1da
Fix NAs are now 0 in some calls and msgs features
2020-10-08 12:23:54 -04:00
236b1cd809
Update AR module for segments; Refactor input format
2020-10-07 18:11:06 -04:00
6b1e006b82
Refactor fused location resampling
2020-10-07 11:51:31 -04:00
80ff3f4229
Cleaning the Doryab location features code to fix the bug of Emoty dataframe in ClusterAndLabel
2020-10-01 11:34:41 -04:00
c0c32d9f9e
Fix bug in assign date segment when file is empty
2020-09-30 14:53:59 -04:00
bccc9a0697
Move deduplicate_episodes() function into chunk_episodes() function; rename "time_diff" with duration
2020-09-29 18:05:25 -04:00
39f6e3841c
Update AR episode computation
2020-09-29 17:26:43 -04:00
10b0386c8f
Merge branch 'day_segments' of https://github.com/carissalow/rapids into day_segments
2020-09-29 17:24:41 -04:00
f1717e59e7
Update screen&battery episodes features with different segment format
2020-09-29 17:13:34 -04:00
499b92e240
Fix missing R library
2020-09-29 15:35:44 -04:00
0dafdd1340
Redo resample episodes in R, new battery episodes
2020-09-29 15:18:48 -04:00
a6b99259f7
Fix bug when filtering by day segment and there are no rows belonging to that segment
2020-09-28 15:53:38 -04:00
4f64f7eeef
Fix the bug on ClusterAndLabel when recieving empty data.
2020-09-28 15:02:03 -04:00
9e15f46fc3
Update day segment format
2020-09-28 11:38:47 -04:00
303d1472ed
Change how we compute AR episodes
2020-09-22 17:32:51 -04:00
a851a460e4
Change the way we compute battery episodes
2020-09-22 16:53:51 -04:00
47ab0a2ce5
Fix phone_sensed_bins when there is no data
2020-09-21 10:28:42 -04:00
f806cb44ac
Fix the bug of screen duration features for different segments
2020-09-18 20:25:29 -04:00
ca14eba8e1
Initial testing restructure for 30mins segments
2020-09-18 19:14:07 -04:00
3022a60fe9
Fix day segment error with no rows
2020-09-18 17:19:11 -04:00
3c27bb2d18
Refactor frequency segments and fix periodic labels
2020-09-18 16:29:48 -04:00
06bb2b9c73
Fix accuracy bug in barnett location features
2020-09-18 11:26:10 -04:00
4dc8d38c66
Optimise assign to day segment
2020-09-16 14:54:20 -04:00
c577229f18
Optimise assign to day segment
2020-09-14 17:56:04 -04:00
a665aedbe8
Optimise assign day segment
2020-09-14 16:19:42 -04:00
132e52aeeb
Fix day segment
2020-09-14 15:14:41 -04:00
3c45f6b152
Refactor assign to day segment
2020-09-14 14:21:36 -04:00
e7db230426
New Application_foreground testing & bugfixes
2020-09-11 17:55:09 -04:00
d4c56642c7
New testing and bug fix for conversation features
2020-09-10 14:01:32 -04:00
a25849f7a4
Fix timeattop location name bug
2020-09-09 11:17:49 -04:00
6436bf79db
Fix overall heatmap bug
2020-09-09 11:17:05 -04:00
b1baf6d9df
Update overall compliance heatmap: x-axis label; plot title
2020-09-08 16:21:46 -04:00
cf2a8f13a3
New testing and minor bug fix for light
2020-09-04 12:33:13 -04:00
1ed85f1180
Delete unused files
2020-09-02 14:12:53 -04:00
98ebf9bd13
Migrate conversation to new file structure
2020-09-01 17:07:07 -04:00
8d87f6e497
Migrate app foreground to new file structure
2020-09-01 15:25:35 -04:00
681a77f23c
Clean utils.py
2020-09-01 12:02:31 -04:00
d674010e08
Rename param to provider
2020-09-01 12:01:45 -04:00
e643881058
Migrate light to new file structure
2020-09-01 12:01:24 -04:00
eda58d9d2c
Migrate wifi to new file structure
2020-08-31 18:51:06 -04:00
77d9d8f025
Migrate bluetooth to new file structure
2020-08-31 17:08:26 -04:00
132311da77
Migrate messages to new file structure
2020-08-31 13:34:15 -04:00
148301d468
Testing, test docs and bugfixes for conversation
2020-08-28 18:28:00 -04:00
e269062439
Migrate calls to new provider file structure
2020-08-28 17:50:49 -04:00
011b9736d5
Refactor the function to fetch provider features
2020-08-28 17:40:23 -04:00
b0f1477d7e
Migrate location providers to new file structure and segments
2020-08-28 13:53:00 -04:00
3052693573
Add missing entropy lib to call features
2020-08-26 13:04:02 -04:00
c3a6b1f9a2
Fix to calls change segment label column name
2020-08-26 13:02:27 -04:00
14d2d694ce
Migrate messages to new segments
2020-08-26 13:01:58 -04:00
31ec5b0da4
Finalise new day segment structure with calls as example
2020-08-26 12:09:53 -04:00
6a5470e338
Splitting Energy Feature in Conversation to Voice and Noise.
2020-08-23 13:20:19 -04:00
4b6ba12e62
raise exception for invalid frequency segmentation info; fixed inconsistency in frequency segmentation output columns
2020-08-20 14:51:22 -04:00
5f672b1f39
verification and creation of frequency segments
2020-08-20 14:51:22 -04:00
c2d011fb6a
Add base functions to compute_day_segments.py
2020-08-20 14:51:22 -04:00
026d056378
Simplify window workflow
2020-08-20 14:51:22 -04:00
a0b5b5982b
Setup rules and files to support multiple
2020-08-20 14:45:09 -04:00
b116accb6d
Disable error message and add prints for restore_sql_file
2020-08-18 17:39:25 -04:00
75deaf7c26
Fix sql file name for restore_sql_file.py
2020-08-18 14:32:26 -04:00
0b2abc692c
Fix restore_sql_file bug: table already exists
2020-08-18 14:18:30 -04:00
b3fbc79195
Minor fix of metrics section
2020-08-17 17:27:58 -04:00
00385dc54d
Discard useless parameters and related code of example
2020-08-17 10:48:15 -04:00
123b78d438
Fix minor bugs of modeling.py: f1-macro and proba
2020-08-16 16:08:51 -04:00
f5ea5c3116
Conversation testing, testing_docs & minor bugfix
2020-08-13 18:36:38 -04:00
1ad8bb0365
Add testing, test docs and minor fixes for AR
2020-08-13 13:41:45 -04:00
622d2997c3
Fixed the time first and last conversation features bug.
2020-08-13 12:45:08 -04:00
b3b16bdb73
Minor fix of heatmap_days_by_sensors.py
2020-08-12 16:29:33 -04:00
5c25002507
Return an empty html file of heatmap_days_by_sensor when there are no records for any sensors.
2020-08-12 16:26:11 -04:00
a3cd341bf2
Fixed the zero conversation time issues.
2020-08-11 20:24:21 -04:00
08ce250839
Unify conversation timestamp for ios
2020-08-11 16:18:06 -04:00
b04088f11f
Fix valid sensed days with 0 valid bins
2020-08-07 11:07:37 -04:00
18d220d6c0
Fix data cleaning bug: days threshold
2020-08-06 17:26:37 -04:00
2859a53202
Small necessary changes in Location Doryab Features.
2020-08-05 13:39:37 -04:00
38b3f55ab8
Added 'sampling_frequency' in Documentation.
2020-08-05 13:14:31 -04:00
df7a52cd6c
Added Sampling Frequency to Doryab Location Features.
2020-08-05 13:02:34 -04:00
8cfe2dabb7
Improved code for radius of gyration in location features.
2020-08-05 12:06:39 -04:00
973e1669fa
Move baselines into a folder; rename column of "num_of_participants" with "num_of_rows" in modeling.py
2020-08-05 07:51:35 -04:00
2f4c137bc1
fixed the bug "Radius of Gyration"
2020-08-03 18:28:02 -04:00
e13b89b125
Add restore_sql_file rule; notsummarised module; diff platforms for heatmap_days_by_sensors
2020-08-03 13:09:16 -04:00
dbc51de4e3
Updated WIFI testing, bugfix and docs
2020-07-29 14:14:47 -04:00
93157db210
Data cleaning section: replace "day_type" with "day_idx"
2020-07-27 18:27:36 -04:00
84d4ea9258
HEATMAP_DAYS_BY_SENSORS: shows all dates when EXPECTED_NUM_OF_DAYS = -1
2020-07-26 20:37:24 -04:00
388f09a003
Add missing wifi table
2020-07-24 16:35:41 -04:00
b3aa4d82e1
Overall compliance heatmap: shows all dates for all participants (only supports last certain dates previously)
2020-07-24 12:58:48 -04:00
5f771618ae
Add screen episode filter parameters
2020-07-23 14:38:59 -04:00
Meng Li
JulioV
nikunjgoel95
Joe Kim
Juseong Kim
abhineethreddyk
kaguillera
Yasaman Sefidgar