e123a14082
Improve aware_csv msg when CSV files don't exist
2021-06-01 10:57:17 -04:00
9687081fbe
Refactor the rule phone_locations_add_doryab_extra_columns
2021-05-28 09:48:36 -04:00
0d6f51be8b
Refactor location features from Doryab provider & add a new strategy to infer home location & fix bugs
2021-05-26 17:36:52 -04:00
32472461ec
- Fix bug when no phone data yield is needed to process location data
...
- Remove location rows with the same timestamp based on their accuracy
2021-05-26 14:04:29 -04:00
9b21196f35
Fixed `expected_minutes` to account for different time segments. ( #136 )
2021-05-26 11:44:48 -04:00
edf71e055d
Add the EXCLUDE_SLEEP module for steps intraday features
2021-05-21 15:23:21 -04:00
5e451f99b0
Added phone keyboard features including docs/tests ( #134 )
2021-05-21 11:45:27 -04:00
e9cd9c94d7
Fix PID matching when joining data from participants
2021-05-11 16:49:04 -04:00
32818a4802
Fix parse of pids with more than 1 devices
2021-05-11 16:42:20 -04:00
809845143f
Test & fix bugs of sleep intraday features
2021-04-27 14:40:14 -04:00
7c7f34ec45
Test & fix bugs of sleep summary features
2021-04-27 14:40:14 -04:00
50fe09cfac
Update data streams mutation of fitbit data
2021-04-27 14:40:14 -04:00
66d9a9d640
Update params & docs of sleep features
2021-04-27 14:33:19 -04:00
4beafd233d
Fix crash when scraping data for an app that does not exist
2021-04-22 14:28:52 -04:00
ea8094e028
Fix length of periodic segments on days with DLS
2021-04-22 11:32:10 -04:00
9c56422529
Add calories intraday features
2021-04-20 12:00:38 -04:00
00a3335623
Add device_id column for sleep intraday episodes
2021-04-08 11:21:28 -04:00
286d317af4
Fix crash when there are no periodic segments to assign
...
This includes a simplification of how periodic segments are computed based on all local dates in the data independently of their time zones
2021-04-07 12:03:25 -04:00
9551669d47
Fix periodic segments bug when there are no segments to assign
2021-04-06 20:29:30 -04:00
78173c54ab
Convert date time object to string in assign_tz_code() function
2021-04-06 23:28:53 +00:00
1025e6d9d8
Fix datetime labels of event segments across multiple tzs
2021-04-06 13:58:58 -04:00
8909876cff
Add local_segment column for phone data yield features
2021-04-05 21:13:36 +00:00
68125dc1bf
Fix the bug of phone data yield features when the input is empty
2021-04-05 20:57:05 +00:00
46f5e24814
Fix Fitbit tz inference from phone data
2021-04-05 11:51:57 -04:00
636b64c61a
Revert "Added more keyboard features."
...
This reverts commit 94c72e3172
2021-04-05 11:25:00 -04:00
68e12a2563
Fix bugs of bluetooth feature extraction when number of unique bt_address is 2
2021-04-05 14:09:50 +00:00
8414977331
Fix the bug of utils.py when one participant have multi timezones
2021-04-03 21:20:10 -04:00
94c72e3172
Added more keyboard features.
2021-04-01 20:54:13 -04:00
1ea5b74eff
Fix the bug of utils.py when one participant have multi timezones
2021-03-31 19:34:14 -04:00
136dfef56b
Fix the bug of Analysis Workflows while parsing targets with updated segments
2021-03-30 16:41:50 -04:00
99dae079d5
Add iOS BT and Wifi visible to formats for old devices
2021-03-30 15:32:50 -04:00
30ad3cd586
Validate participant files without device ids
2021-03-28 15:29:08 -04:00
87fbbbe402
Refactor and simplify time segments
2021-03-28 15:29:07 -04:00
c48c1c8f24
Optimize Barnett's computation multi-day segments
2021-03-28 15:29:07 -04:00
d0858f8833
Fix overlapping periodic time segments
2021-03-28 15:29:07 -04:00
e177aa6386
Update create participant files section
2021-03-25 11:39:31 -04:00
a5eb535126
Update visualizations docs & add time flag for heatmap of overall data yield
2021-03-23 21:40:55 -04:00
cfc5039918
Fixed the empty dataframe case in infer_home_locations.py and added array condition in doryab location
2021-03-19 11:15:57 -04:00
294d84277d
Fix bug of sleep intraday PRICE provider when the dataframe of a segment is empty
2021-03-17 15:21:31 -04:00
771c14a928
Improve mysql containers error messages
2021-03-16 20:02:44 -04:00
6e234f7951
Fix warn instead of stop when there are not device ids
2021-03-16 20:02:16 -04:00
4c2f60fffd
Fix bugs in readable datetime and screen episodes
2021-03-16 20:01:43 -04:00
bb737237d0
Fixes for aware_influxdb
2021-03-16 11:26:46 -04:00
4528ab3641
Replace SRC LANGUAGE and FOLDER with SCRIPT
2021-03-14 22:14:13 -04:00
f4b2bd1fb2
Cleanup data/
2021-03-14 13:36:22 -04:00
42cee67664
Add aware_influxdb in beta
2021-03-14 13:33:43 -04:00
3d4a04effe
Refactor testing
2021-03-14 00:09:08 -05:00
2b6447105a
Migrate analysis example to new data stream
2021-03-12 19:52:34 -05:00
fae0c2ac05
Swap TABLE for CONTAINER
2021-03-12 18:14:49 -05:00
d529490999
Migrate fitbit features to new data stream
2021-03-12 12:38:36 -05:00
6e898beca5
Add aware_csv
2021-03-11 19:32:11 -05:00
13174b0c2a
Fix a bug when fitbit data is empty
2021-03-11 14:51:16 -05:00
2ee45995f2
Update config docs and create participant files script
2021-03-11 14:40:33 -05:00
1e66dad838
Fix bug in empatica_zip container script
2021-03-11 14:39:26 -05:00
a79997e0ac
Add empatica_zip docs
2021-03-11 14:39:26 -05:00
d48194fc07
Add fitbitparsed_csv
2021-03-11 14:39:26 -05:00
b97b70e3a1
Add fitbitjson_csv
2021-03-11 14:39:26 -05:00
470f4276af
Add fitbitparsed_mysql
2021-03-11 14:39:26 -05:00
1b0ee4bbf0
Add sleep intraday to fitbitjson_mysql
2021-03-11 14:39:20 -05:00
a420f5ef92
Add sleep summary to fitbitjson_mysql
2021-03-11 14:37:22 -05:00
93baff9f83
Migrate phone keyboard sensor to new data stream
2021-03-11 14:37:22 -05:00
35968e2fd0
Migrate phone log sensor to new data stream
2021-03-11 14:37:22 -05:00
091f9c048a
Migrate phone apps notifications sensor to new data stream
2021-03-11 14:37:22 -05:00
b49dab0949
Migrate phone apps crashes sensor to new data stream
2021-03-11 14:37:22 -05:00
47e1b33816
Add hr intraday to fitbitjson_mysql
2021-03-11 14:37:22 -05:00
47f449555a
Add hr summary to fitbitjson_mysql
2021-03-11 14:37:22 -05:00
72f6b2d621
Add steps intraday to fitbitjson_mysql
2021-03-11 14:37:22 -05:00
9a276c1c66
Add steps summary to jsonfitbit_mysql
2021-03-11 14:37:22 -05:00
13290cd444
Migrate phone wifi visible sensor to new data stream
2021-03-11 14:36:52 -05:00
d42c6e9c91
Migrate phone wifi connected sensor to new data stream
2021-03-11 14:35:34 -05:00
4825962361
Migrate phone screen sensor to new data stream
2021-03-11 14:35:34 -05:00
bd4f647d37
Migrate phone messages sensor to new data stream
2021-03-11 14:35:34 -05:00
3a65b3864d
Migrate phone locations sensor to new data stream
2021-03-11 14:35:34 -05:00
7a50a52a9d
Migrate phone light sensor to new data stream
2021-03-11 14:35:34 -05:00
c1682d8cd3
Migrate phone calls sensor to new data stream
2021-03-11 14:35:34 -05:00
0e96f39599
Migrate phone bluetooth sensor to new data stream
2021-03-11 14:35:34 -05:00
6d06d2b1eb
Update MUTATION structure code & docs for AR, Apps foreground, and battery sensors
2021-03-11 14:35:34 -05:00
6970954358
Change MUTATION structure
2021-03-11 14:35:34 -05:00
58ef276179
Add stream_parameters arg to phone and empatica mutation scripts
2021-03-11 14:35:34 -05:00
1063b4ca65
Add steps summary to fitbitjson_mysql
2021-03-11 14:35:34 -05:00
f7cf316133
Migrate phone battery sensor to new data stream
2021-03-11 14:35:34 -05:00
824523e32c
Migrate phone apps foreground sensor to new data stream
2021-03-11 14:35:34 -05:00
6b13c80e40
Fix bug while checking OS in stream_schema
2021-03-11 14:35:34 -05:00
8c79cfc56f
Fix OS specific dependcies
2021-03-11 14:35:34 -05:00
7372fca0dd
Fix bug when sensor is not available for an OS
2021-03-11 14:35:34 -05:00
41711fcdb7
Rename download_data add support for py containers
2021-03-11 14:35:34 -05:00
4b33ee43ba
Replace .env with credentials.yaml
2021-03-11 14:35:34 -05:00
f65e3c8b1a
Migrate empatica sensors to new data stream
2021-03-11 14:35:34 -05:00
2eae84ff05
Add docs of AR & conversation sensors
2021-03-11 14:35:34 -05:00
fb054b539f
Add support for multiple time zones
2021-03-11 14:35:34 -05:00
6c51c6c239
Move phone_conversation to aware_mysql stream
2021-03-11 14:35:34 -05:00
ab1a3dbf79
Move phone_activity_recognition to aware_mysql stream
2021-03-11 14:35:34 -05:00
dc11cb593d
Add support for smartphone sources and schemas.
...
Initial support for accelerometer
Update docs for automatically create participants
Update docs for initial multiple time zones
2021-03-11 14:35:33 -05:00
aac87311e8
Update socialjetlag feature of sleep intraday: replace bedtime with midpoint sleep
2021-03-04 15:49:25 -05:00
7b4598357d
Update the PRICE provider's example in sleep intraday docs
2021-02-26 17:47:01 -05:00
716ff3c592
Fix PRICE provider's bug when input is an empty dataframe
2021-02-26 17:47:01 -05:00
d74196cab4
Add sleep intraday features with PRICE provider
2021-02-26 17:47:01 -05:00
bedf7106e5
Fix sleep episodes bug
2021-02-26 17:47:01 -05:00
8377c12efb
Add sleep intraday features with RAPIDS provider
2021-02-26 17:47:01 -05:00
724027e383
Small fixes to timeathome docs, add config validation
2021-02-24 17:49:22 -05:00
3d6caea6c4
Added the timeathome feature using infer_home_location.py as interim file.
2021-02-24 16:57:25 -05:00
9668dfac7a
Update docs to support Empatica
2021-02-21 17:32:41 -05:00
faefca8b9a
Fix extra index column when dataset is empty
2021-02-21 17:32:41 -05:00
2e46f56111
Empatica zips must be placed in pid folder and small fixes
2021-02-21 17:32:41 -05:00
a26a44819a
Add stats features for empatica bvp, eda, ibi, temp
...
Fix Snakefile file indentention
2021-02-21 17:32:41 -05:00
4469cfd6bb
add stats features for bvp, eda, ibi, temp
2021-02-21 17:32:41 -05:00
c6dc7e675a
Add stats features for empatica heartrate
...
Turn off all empatica compute features
2021-02-21 17:32:41 -05:00
3bb0230bac
Add statistic features for empatica accelerometer
2021-02-21 17:32:41 -05:00
5f5f19866f
implement extract_empatica_data script
...
add support for all data types
Fix name comparison of zipped files
2021-02-21 17:32:41 -05:00
4b9857562b
Add support for zip input files
2021-02-21 17:32:41 -05:00
8c726f5d4f
Start empatica support
2021-02-21 17:32:41 -05:00
d8813e2d04
Fix bug when any of the rows from any sensor do not belong a time segment
2021-02-09 14:51:54 -05:00
b67f990816
Add new `FITBIT_DATA_YIELD` `RAPIDS` provider
2021-02-02 18:30:21 -05:00
9b248c449d
Fixing and adding MAXIMUM_ROW_DURATION.
2021-02-02 11:38:13 -05:00
e7fc8f44f2
Removing Sampling Frequency and fixing ROG, location entropy and normalized location entropy.
2021-02-02 11:38:08 -05:00
0bbf15f52e
Fixed the features dependent on time duration.
2021-02-02 11:36:14 -05:00
3d0d062491
Fix HR summary bug: do not consider rows with restinghr=0
2021-02-01 17:29:30 -05:00
f83c5a585e
Fix HR intraday bug: minutesonZONE features are 0
2021-02-01 13:57:12 -05:00
aefc794274
Fix location processing when certain columns don't exist
2021-02-01 11:49:22 -05:00
dbf57f43f3
Updated battery feature and its testing for periodic
2021-01-31 20:24:43 -05:00
25a3492eba
Drop rows without "assigned_segments" column before feature extraction
2021-01-21 19:41:17 -05:00
797de54b34
Fix merge bug of fetch_provider_features() function
2021-01-21 14:58:31 -05:00
5f60aac5c8
Fix KeyError bug of parsing steps data
2021-01-20 11:26:28 -05:00
d0fe4d4c28
Add ALL_RESAMPLED flag and accuracy limit
2021-01-14 14:34:25 -05:00
38fadbf202
Feature/doryab location clustering ( #111 )
...
* Added OPTICS - lightweight clustering algorithm.
* Changed the error message for inconsistent parameters in CONFIG
* Removing hardcoded values and changing default EPS value in the clustering algorithm.
* Added Observation in Doryab Feature docs.
Co-authored-by: nikunjgoel95 <nikunjgoel2009@gmail.com>
2021-01-14 14:22:51 -05:00
b7ba3c6407
Feature/location doryab fix ( #109 )
...
* Fixing the doryab location features for context of clustering.
* Fixed the wrong shifting while calculating the distance.
* Refractoring the haversine function
* Removed comments.
* Cleaning parts of the code.
* Updated the documentation for CLUSTER_ON parameter.
Co-authored-by: nikunjgoel95 <nikunjgoel2009@gmail.com>
2021-01-07 16:20:46 -05:00
4926497ae2
Fix bugs in Fitbit data parsing
...
- Fix the script that was breaking with an empty file
- Fix the script that was breaking when start/end dates were empty
- Ambigous and nonexistent DST times are handled now
- Remove unnecessary else clause
2021-01-06 11:43:01 -05:00
5203aa60d1
Fix bugs in create participants files script
...
- The PHONE and FITBIT flags were mixed up
- The start/end dates from the CSV file weren't being parsed correctly
2021-01-06 11:14:15 -05:00
3a80f93771
Fix segment error when device ids is empty
2021-01-06 11:12:10 -05:00
51c7739bfc
Revert "Start support for phone_keyboard"
...
This reverts commit dd95b4f941
2020-12-20 16:29:40 -05:00
dd95b4f941
Start support for phone_keyboard
2020-12-14 13:42:22 -05:00
63202c62cc
Fix BT doryab bug and update change log
2020-12-12 17:11:48 -05:00
05627296f4
Fix filter_data_by_segment bug
2020-12-12 17:10:59 -05:00
5bd1bfe856
Add new bluetooth doryab features and deprecate rapids provider
2020-12-12 17:01:46 -05:00
266dd28d02
Add bluetooth doryab features
2020-12-11 12:03:22 -05:00
ae21c076cd
Fix conversation data bug
2020-12-04 15:23:08 -05:00
9a02e93b25
Swap RMySQL for RMariaDB
2020-12-04 14:57:13 -05:00
9a0e57301b
Update 4 plots for time_segments
2020-12-03 21:00:32 -05:00
614e759551
Refactor day segments to time segments
2020-12-02 18:41:03 -05:00
c24cba8679
Event segments' device id can be one of any assigned to a participant
2020-12-02 17:17:55 -05:00
0bd43c139c
Update plot#1: histogram_phone_data_yield
2020-12-01 14:21:33 -05:00
016bdbfe8c
Update Python feature scripts to add sensor and provider names automatically
2020-11-30 14:42:19 -05:00
70991d6667
Add dates filter while parsing fitbit data
2020-11-30 12:34:14 -05:00
5178be585d
Rename modeling.py to modelling.py & Update example_config.yaml
2020-11-25 22:35:38 -05:00
5f51c94ac6
Separate device data configuration and update docs
2020-11-25 19:42:11 -05:00
b4a512faf3
Add analysis example workflow
2020-11-25 16:34:05 -05:00
ced3305ddb
Update R feature scripts to add sensor and provider names automatically
2020-11-25 14:49:42 -05:00
f02ca2624d
Add phone yield sensor
2020-11-24 19:12:16 -05:00
555811211d
Fix column check in day segment compute
2020-11-24 14:51:06 -05:00
767e838c87
Fix HR&Sleep bugs
2020-11-24 13:43:24 -05:00
JulioV
Meng Li
Nikunj Goel
Joe Kim
Juseong Kim
abhineethreddyk