Primoz
87e5209a9f
Squashed commit of the following:
...
commit 8a6b52a97c
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Nov 29 11:35:49 2022 +0000
Switch to 30_before ERS with corresponding targets.
commit 244a053730
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Nov 29 11:19:43 2022 +0000
Change output files settings to nonstandardized.
commit be0324fd01
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Nov 28 12:44:25 2022 +0000
Fix some bugs and set categorical columns as categories dtypes.
commit 99c2fab8f9
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Nov 16 09:50:18 2022 +0000
Fix a bug in the making of the individual model (when there is no target in the participants columns).
commit 286de93bfd
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Nov 15 11:21:51 2022 +0000
Fix some bugs and extend ERS and cleaning scripts with multiple stress event targets logic.
commit ab803ee49c
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Nov 15 10:14:07 2022 +0000
Add additional appraisal targets.
commit 621f11b2d9
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Nov 15 09:53:31 2022 +0000
Fix a bug related to wrong user input (duplicated events).
commit bd41f42a5d
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Nov 14 15:07:36 2022 +0000
Rename target_ to segmenting_ method.
commit a543ce372f
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Nov 14 15:04:16 2022 +0000
Add comments for event_related_script understanding.
commit 74b454b07b
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Nov 11 09:15:12 2022 +0000
Apply changes to string answers to make them language-generic.
commit 6ebe83e47e
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Nov 10 12:42:52 2022 +0000
Improve the ERS extract method with a couple of validations.
commit 00350ef8ca
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Nov 10 10:32:58 2022 +0000
Change config for stressfulness event target method.
commit e4985c9121
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Nov 10 10:29:11 2022 +0000
Override stressfulness event target with extracted values from csv.
commit a668b6e8da
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Nov 10 09:37:27 2022 +0000
Extract ERS and stress event targets to csv files (completed).
commit 9199b53ded
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Nov 9 15:11:51 2022 +0000
Get, join and start processing required ERS stress event data.
commit f3c6a66da9
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Nov 8 15:53:43 2022 +0000
Begin with stress events in the ERS script.
commit 0b3e9226b3
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Nov 8 14:44:24 2022 +0000
Make small corrections in ERS file.
commit 2d83f7ddec
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Nov 8 11:32:05 2022 +0000
Begin the ERS logic for 90-minutes events.
commit 1da72a7cbe
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Nov 8 09:45:37 2022 +0000
Rename targets method in config.
commit 9f441afc16
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Nov 4 15:09:04 2022 +0000
Begin ERS logic for 90-minutes events.
commit c1c9f4d05a
Merge: 62f46ea3
7ab0280d
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Nov 4 09:11:58 2022 +0000
Merge branch 'imputation_and_cleaning' of https://repo.ijs.si/junoslukan/rapids into imputation_and_cleaning
commit 62f46ea376
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Nov 4 09:11:53 2022 +0000
Prepare method-based logic for ERS generating.
commit 7ab0280d7e
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Nov 4 08:58:08 2022 +0000
Correctly rename stressful event target variable.
commit eefa9f3f4d
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Nov 3 14:49:54 2022 +0000
Add new target: stressfulness_event.
commit 5e8174dd41
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Nov 3 13:52:45 2022 +0000
Add new target: stressfulness_period.
commit 35c1a762e7
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Nov 3 13:51:18 2022 +0000
Improve filtering by esm_session and device_id.
commit 02264b21fd
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Nov 3 09:30:12 2022 +0000
Add logic for target selection in ERS processing.
commit 0ce8723bdb
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Nov 2 14:01:21 2022 +0000
Extend imputation logic within the cleaning script.
commit 30b38bfc02
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Oct 28 09:00:13 2022 +0000
Fix the generating procedure of ERS file for participants with multiple devices.
commit cd137af15a
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Oct 27 14:20:15 2022 +0000
Config for 30 minute EMA segments.
commit 3c0585a566
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Oct 27 14:12:56 2022 +0000
Remove obsolete comments.
commit 6b487fcf7b
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Oct 27 14:11:42 2022 +0000
Set E4 data yield to 1 if it is over 1. Optimize E4 data_yield script.
commit 5d17c92e54
Merge: a31fdd14
0d143e6a
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 26 14:18:20 2022 +0000
Merge branch 'imputation_and_cleaning' of https://repo.ijs.si/junoslukan/rapids into imputation_and_cleaning
commit a31fdd1479
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 26 14:18:08 2022 +0000
Start to test empatica_data_yield precieved error.
commit 936324d234
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 26 14:17:27 2022 +0000
Switch config for 30 minutes event related segments.
commit da0a4596f8
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 26 14:16:25 2022 +0000
Add additional ESM processing logic for ERS csv extraction.
commit d4d74818e6
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 26 14:14:32 2022 +0000
Fix a bug - missing time_segment column when df is empty
commit 14ff59914b
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 26 09:59:46 2022 +0000
Fix to correct dtypes.
commit 6ab0ac5329
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 26 09:57:26 2022 +0000
Optimize memory consumption with dtype definition while reading csv file.
commit 0d143e6aad
Merge: 8acac501
b92a3aa3
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Oct 25 15:28:27 2022 +0000
Merge branch 'imputation_and_cleaning' of https://repo.ijs.si/junoslukan/rapids into imputation_and_cleaning
commit 8acac50125
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Oct 25 15:26:43 2022 +0000
Add safenet when features dataframe is empty.
commit b92a3aa37a
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Oct 25 15:25:22 2022 +0000
Remove unwanted output or other error producing code.
commit bfd637eb9c
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Oct 25 08:53:44 2022 +0000
Improve strings formatting in straw_events file.
commit 0d81ad5756
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 19 13:35:04 2022 +0000
Debug assignment of segments to rows
commit cea451d344
Merge: e88bbd54
cf38d9f1
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Oct 18 09:15:06 2022 +0000
Merge branch 'imputation_and_cleaning' of https://repo.ijs.si/junoslukan/rapids into imputation_and_cleaning
commit e88bbd548f
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Oct 18 09:15:00 2022 +0000
Add new daily segment and filter by segment in the cleaning script.
commit cf38d9f175
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Oct 17 15:07:33 2022 +0000
Implement ERS generating logic.
commit f3ca56cdbf
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Oct 14 14:46:28 2022 +0000
Start with ERS logic integration within Snakemake.
commit 797aa98f4f
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 12 15:51:50 2022 +0000
Config for ERS testing.
commit 9baff159cd
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 12 15:51:23 2022 +0000
Changes needed for testing and starting of the Event-Related Segments.
commit 0f21273508
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 12 12:32:51 2022 +0000
Bugs fix
commit 55517eb737
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 12 12:23:11 2022 +0000
Necessary commit before proceeding.
commit de15a52dba
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Oct 11 08:36:23 2022 +0000
Bug fix
commit 1ad25bb572
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Oct 11 08:26:17 2022 +0000
Few modifications of some imputation values in cleaning script and feature extraction.
commit 9884b383cf
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Oct 10 16:45:38 2022 +0000
Testing new data with AutoML.
commit 2dc89c083c
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Oct 7 08:52:12 2022 +0000
Small changes in cleaning overall
commit 001d400729
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Oct 6 14:28:12 2022 +0000
Clean features and create input files based on all possible targets.
commit 1e38d9bf1e
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Oct 6 13:27:38 2022 +0000
Standardization and correlation visualization in overall cleaning script.
commit a34412a18d
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 5 14:16:55 2022 +0000
E4 data yield corrections. Changes in overal cs - standardization.
commit 437459648f
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Oct 5 13:35:05 2022 +0000
Errors fix: individual script - treat participants missing data.
commit 53f6cc60d5
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Oct 3 13:06:39 2022 +0000
Config and cleaning script necessary changes ...
commit bbeabeee6f
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Oct 3 12:53:31 2022 +0000
Last changes before processing on the server.
commit 44531c6d94
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Sep 30 10:04:07 2022 +0000
Code cleaning, reworking cleaning individual based on changes in overall script. Changes in thresholds.
commit 7ac7cd5a37
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Sep 29 14:33:21 2022 +0000
Preparation of the overall cleaning script.
commit 68fd69dada
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Sep 29 11:55:25 2022 +0000
Cleaning script for individuals: corrections and comments.
commit a4f0d056a0
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Sep 29 11:44:27 2022 +0000
Fillna for app foreground and activity recognition
commit 6286e7a44c
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Sep 28 12:47:08 2022 +0000
firstuseafter column removed from contextual imputation
commit 9b3447febd
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Sep 28 12:40:05 2022 +0000
Contextual imputation correction
commit d6adda30cf
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Sep 28 12:37:51 2022 +0000
Contextual imputation on time(first/last) features.
commit 8af4ef11dc
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Sep 28 10:02:47 2022 +0000
Contextual imputation by feature type.
commit 536b9494cd
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Sep 27 14:12:08 2022 +0000
Cleaning script corrections
commit f0b87c9dd0
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Sep 27 09:54:15 2022 +0000
Debugging of the empatica data yield integration.
commit 7fcdb873fe
Merge: 5c7bb0f4
bd53dc16
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Sep 27 07:50:29 2022 +0000
Merge branch 'imputation_and_cleaning' of https://repo.ijs.si/junoslukan/rapids into imputation_and_cleaning
commit 5c7bb0f4c1
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Sep 27 07:48:32 2022 +0000
Config changes
commit bd53dc1684
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Sep 26 15:54:00 2022 +0000
Empatica data yield usage in the cleaning script.
commit d9a574c550
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Sep 23 13:24:50 2022 +0000
Changes in the cleaning script and preparation of empatica data yield method.
commit 19aa8707c0
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Sep 22 13:45:51 2022 +0000
Redefined cleaning steps after revision
commit 247d758cb7
Merge: 90ee99e4
7493aaa6
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Sep 21 07:18:01 2022 +0000
Merge branch 'imputation_and_cleaning' of https://repo.ijs.si/junoslukan/rapids into imputation_and_cleaning
commit 90ee99e4b9
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Sep 21 07:16:00 2022 +0000
Remove TODO comments
commit 7493aaa643
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Sep 20 12:57:55 2022 +0000
Small changes in cleaning scrtipt and missing vals testing.
commit eaf4340afd
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Sep 20 08:03:48 2022 +0000
Small imputation and cleaning corrections.
commit a96ea508c6
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Sep 19 07:34:02 2022 +0000
Fill NaN of Empatica's SD second order feature (must be tested).
commit 52e11cdcab
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Sep 19 07:25:54 2022 +0000
Configurations for new standardization path.
commit 92aff93e65
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Sep 19 07:25:16 2022 +0000
Remove standardization script.
commit 18b63127de
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Sep 19 06:16:26 2022 +0000
Removed all standardizaton rules and configurations.
commit 62982866cd
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Sep 16 13:24:21 2022 +0000
Phone wifi visible inspection (WIP)
commit 0ce6da5444
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Sep 16 11:30:08 2022 +0000
kNN imputation relocation and execution only on specific columns.
commit e3b78c8a85
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Sep 16 10:58:57 2022 +0000
Impute selected phone features with 0.
Wifi visible, screen, and light.
commit 7d85f75d21
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Sep 16 09:03:30 2022 +0000
Changes in phone features NaN values script.
commit 385e21409d
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Sep 15 14:16:58 2022 +0000
Changes in NaN values testing script.
commit 18002f59e1
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Sep 15 10:48:59 2022 +0000
Doryab bluetooth and locations features fill in NaN values.
commit 3cf7ca41aa
Merge: d27a4a71
d5ab5a03
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Sep 14 15:38:32 2022 +0000
Merge branch 'imputation_and_cleaning' of https://repo.ijs.si/junoslukan/rapids into imputation_and_cleaning
commit d5ab5a0394
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Sep 14 14:13:03 2022 +0000
Writing testing scripts to determine the point of manual imputation.
commit dfbb758902
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Sep 13 13:54:06 2022 +0000
Changes in AutoML params and environment.yml
commit 4ec371ed96
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Sep 13 09:51:03 2022 +0000
Testing auto-sklearn
commit d27a4a71c8
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Sep 12 13:44:17 2022 +0000
Reorganisation and reordering of the cleaning script.
commit 15d792089d
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Sep 1 10:33:36 2022 +0000
Changes in cleaning script:
- target extracted from config to remove rows where target is nan
- prepared sns.heatmap for further missing values analysis
- necessary changes in config and participant p01
- picture of heatmap which shows the values state after cleaning
commit cb351e0ff6
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Sep 1 10:06:57 2022 +0000
Unnecessary line (rows with no target value will be removed in cleaning script).
commit 86299d346b
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Sep 1 09:57:21 2022 +0000
Impute phone and sms NAs with 0
commit 3f7ec80c18
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Aug 31 10:18:50 2022 +0000
Preparation a) phone_calls 0 imputation b) remove rows with NaN target
2022-12-08 16:04:39 +00:00
Sam
e5dbbfce44
Avoid NA problem in barnett location evaluation ( #172 )
...
* Avoid occasional issue where does_not_span evaluates to NA, which breaks the if()
* Restored original warning
2022-01-18 10:16:37 -05:00
Meng Li
296960f425
Fix the bug of location doryab features when a participant is moving during the whole time segment
2021-11-18 18:42:19 -05:00
Meng Li
f340b89c58
Temporary revert PHONE_LOCATIONS BARNETT provider to use R script
2021-09-23 18:16:13 -04:00
Meng Li
a3fb718aea
Refactor PHONE_LOCATIONS DORYAB provider to compute features based on location episodes
2021-09-23 17:40:06 -04:00
JulioV
2e553dc9e7
Add tqdm package to environment.yaml
2021-08-16 11:04:03 -04:00
JulioV
3e69966c91
Update error message
2021-08-04 15:33:02 -04:00
Shirley
4ddb2845a6
Update initialize_params
2021-08-04 15:33:02 -04:00
JulioV
834bd3b93d
Refactor in Python of Barnett provider
...
Co-authored-by: Shirley Hayati <sahayati@ucdavis.edu>
Co-authored-by: JulioV <JulioV@users.noreply.github.com>
2021-08-04 15:33:02 -04:00
Hannah Roberts
b52059b027
Ensure date/time format is maintained
...
Within the 'determine which is home' for loop, 'xx' is the midpoint of two datetime objects. When the midpoint is calculated to be midnight, only the date is returned. This can be replicated with:
mydates <- as.POSIXct("2018-01-01 00:00:00", tz = "UTC")
mydates
[1] "2018-01-01 UTC"
This results in 'hourofday' being NA as an hour cannot be found. By adding the suggested format wrapper, the time is maintained and 'hourofday' can be determined. It can then successfully be applied to the embedded if-statement within the loop.
mydates <- format(as.POSIXct("2018-01-01 00:00:00", tz = "UTC"), "%Y-%m-%d %H:%M:%S")
mydates
[1] "01-01-2018 00:00:00"
2021-07-23 10:12:11 -04:00
Meng Li
1c57320ab3
Update segment labels and fix the bug when we do not have any labels for event segments
2021-06-29 09:49:24 -04:00
Meng Li
9687081fbe
Refactor the rule phone_locations_add_doryab_extra_columns
2021-05-28 09:48:36 -04:00
Meng Li
0d6f51be8b
Refactor location features from Doryab provider & add a new strategy to infer home location & fix bugs
2021-05-26 17:36:52 -04:00
JulioV
c48c1c8f24
Optimize Barnett's computation multi-day segments
2021-03-28 15:29:07 -04:00
nikunjgoel95
cfc5039918
Fixed the empty dataframe case in infer_home_locations.py and added array condition in doryab location
2021-03-19 11:15:57 -04:00
JulioV
fb054b539f
Add support for multiple time zones
2021-03-11 14:35:34 -05:00
nikunjgoel95
3d6caea6c4
Added the timeathome feature using infer_home_location.py as interim file.
2021-02-24 16:57:25 -05:00
nikunjgoel95
9b248c449d
Fixing and adding MAXIMUM_ROW_DURATION.
2021-02-02 11:38:13 -05:00
nikunjgoel95
e7fc8f44f2
Removing Sampling Frequency and fixing ROG, location entropy and normalized location entropy.
2021-02-02 11:38:08 -05:00
nikunjgoel95
0bbf15f52e
Fixed the features dependent on time duration.
2021-02-02 11:36:14 -05:00
JulioV
d0fe4d4c28
Add ALL_RESAMPLED flag and accuracy limit
2021-01-14 14:34:25 -05:00
JulioV
38fadbf202
Feature/doryab location clustering ( #111 )
...
* Added OPTICS - lightweight clustering algorithm.
* Changed the error message for inconsistent parameters in CONFIG
* Removing hardcoded values and changing default EPS value in the clustering algorithm.
* Added Observation in Doryab Feature docs.
Co-authored-by: nikunjgoel95 <nikunjgoel2009@gmail.com>
2021-01-14 14:22:51 -05:00
JulioV
b7ba3c6407
Feature/location doryab fix ( #109 )
...
* Fixing the doryab location features for context of clustering.
* Fixed the wrong shifting while calculating the distance.
* Refractoring the haversine function
* Removed comments.
* Cleaning parts of the code.
* Updated the documentation for CLUSTER_ON parameter.
Co-authored-by: nikunjgoel95 <nikunjgoel2009@gmail.com>
2021-01-07 16:20:46 -05:00
JulioV
614e759551
Refactor day segments to time segments
2020-12-02 18:41:03 -05:00
Meng Li
016bdbfe8c
Update Python feature scripts to add sensor and provider names automatically
2020-11-30 14:42:19 -05:00
JulioV
ced3305ddb
Update R feature scripts to add sensor and provider names automatically
2020-11-25 14:49:42 -05:00
JulioV
c5197b6c5f
Merge branch 'master' into ds2
2020-10-28 17:50:37 -04:00
JulioV
86509207ac
Turn off warnings for tidyverse and dplyr
2020-10-23 10:41:00 -04:00
JulioV
24bf62a7ab
Update file names
2020-10-19 15:07:12 -04:00