385e21409d
Changes in NaN values testing script.
2022-09-15 14:16:58 +00:00
18002f59e1
Doryab bluetooth and locations features fill in NaN values.
2022-09-15 10:48:59 +00:00
3cf7ca41aa
Merge branch 'imputation_and_cleaning' of https://repo.ijs.si/junoslukan/rapids into imputation_and_cleaning
2022-09-14 15:38:32 +00:00
d5ab5a0394
Writing testing scripts to determine the point of manual imputation.
2022-09-14 14:13:03 +00:00
dfbb758902
Changes in AutoML params and environment.yml
2022-09-13 13:54:06 +00:00
4ec371ed96
Testing auto-sklearn
2022-09-13 09:51:03 +00:00
d27a4a71c8
Reorganisation and reordering of the cleaning script.
2022-09-12 13:44:17 +00:00
15d792089d
Changes in cleaning script:
...
- target extracted from config to remove rows where target is nan
- prepared sns.heatmap for further missing values analysis
- necessary changes in config and participant p01
- picture of heatmap which shows the values state after cleaning
2022-09-01 10:33:36 +00:00
cb351e0ff6
Unnecessary line (rows with no target value will be removed in cleaning script).
2022-09-01 10:06:57 +00:00
86299d346b
Impute phone and sms NAs with 0
2022-09-01 09:57:21 +00:00
3f7ec80c18
Preparation a) phone_calls 0 imputation b) remove rows with NaN target
2022-08-31 10:18:50 +00:00
f78aa3e7b3
Preparation for cleaning & imputation
2022-08-26 10:56:14 +00:00
a620def209
Generate standardized model input files (NOTE: commented unstandardized sections!)
2022-08-24 13:42:39 +00:00
c498ecb742
Include baseline models (+corrections), disable columns drop in cleaning function.
2022-08-23 14:12:14 +00:00
f088e9586f
Handle empty ACC.csv
2022-08-22 14:20:47 +00:00
0aa0e82673
Handle empty Empatica csv files.
2022-08-22 14:18:12 +00:00
4cfe5a3a98
Disable discarding rows if DATA_YIELD_RATIO_THRESHOLD==0.
2022-08-19 13:10:56 +00:00
607da820f2
Configuration and cleaning changes
2022-08-18 14:21:05 +00:00
fb577bc9ad
Squashed commit of the following:
...
commit 43ecc243cb62bb31eed85cb477ca4131555c7fe7
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 22 15:26:09 2022 +0000
Adding TODO comments
commit 2df1ebf90c3a93812b112b8ed0ee4e23cd74533f
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Jul 21 13:59:23 2022 +0000
README update
commit 5182c2b16dff3537aad42984b8ea5214743cdb32
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Jul 21 11:03:01 2022 +0000
Few corrections for all_cleaning
commit 3d9254c1b3bed6e95e631d4e0402548830a19534
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Jul 21 10:28:05 2022 +0000
Adding the min overlap for corr threshold and preservation of esm cols.
commit e27c49cc8fa4c51f9fe8e593a8d25e9a032ab393
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Jul 21 09:02:00 2022 +0000
Commenting and cleaning.
commit 31a47a5ee4569264e39d7c445525a6e64bb7700a
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Jul 20 13:49:22 2022 +0000
Environment version change.
commit 5b274ed8993f58e783bda6d82fce936764209c28
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Jul 19 16:10:07 2022 +0000
Enabled cleaning for all participants + standardization files.
commit 203fdb31e0f3c647ef8c8a60cb9531831b7ab924
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Jul 19 14:14:51 2022 +0000
Features cleaning fixes after testing. Visualization script for phone features values.
commit 176178d73b154c30b9eb9eb4a67514f00d6a924e
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Jul 19 09:05:14 2022 +0000
Revert "Necessary config changes."
This reverts commit 6ec1ef50430d2e1f5ce4670d505d5e84ac47f0a0.
commit 26ea6512c9d512f95837e7b047fe510c1d196403
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 18 13:19:47 2022 +0000
Adding cleaning function condition and cleaning functionality.
commit 575c29eef9c21e6f2d7832871e73bc0941643734
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 18 12:51:56 2022 +0000
Translation of the cleaning individual RAPIDS function from R to py.
commit 6ec1ef50430d2e1f5ce4670d505d5e84ac47f0a0
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 18 12:02:18 2022 +0000
Necessary config changes.
commit b5669f51612fbd8378848615d639677851ab032f
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 15 15:26:00 2022 +0000
Modified snakemake rule to dynamically choose script extention.
commit 66636be1e8ae4828228b37c59b9df1faf3fc3d3d
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 15 14:43:08 2022 +0000
Trying to modify the snakefile rule to execute scripts in two languages depended on the provider.
commit 574778b00f3cbb368ef4bc74de15cf5070c65ea9
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 15 09:49:41 2022 +0000
gitignore: adding required files so that RAPIDS can be run successfully.
commit 71018ab178256970535e78961602ab8c7f0ebb14
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 15 08:34:19 2022 +0000
Standardization bug fixes
commit 6253c470a624e6bfbb02e0c453b652452eb2dbbc
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Jul 14 15:28:02 2022 +0000
Seperate rules for empatica vs. nonempatica standardization.
Parameter in config that controls the creation of standardized merged files for individual and all participants..
commit 90f902778565e0896d3bae22ae8551be8b487e67
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Jul 12 14:23:03 2022 +0000
Preparing for final csvs' standardization.
commit d25dde3998786a9a582f5cda544ee104386778f9
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 11 12:08:47 2022 +0000
Revert "Changes in config to be reverted."
This reverts commit bea7608e7095021fb7c53a9afa07074448fe4313.
commit 6b23e70857e63deda98eb98d190af9090626c84b
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 11 12:08:26 2022 +0000
Enabled standardization for rest (previously active) phone features.
Testing still needed.
commit 8ec58a6f34ba3d42e5cc71d26e6d91837472ca5f
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 11 09:07:55 2022 +0000
Enabled standardization for phone calls.
All steps completed and tested.
commit bea7608e7095021fb7c53a9afa07074448fe4313
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 11 07:47:51 2022 +0000
Changes in config to be reverted.
commit 4e84ca0e51bf709bff56fd09437b95310ec6bedd
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 8 14:11:24 2022 +0000
Standardization for the rest of the features.
commit cc581aa788e3d5c17131af8f3d5dd6b0c3b5aff7
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 8 14:11:08 2022 +0000
README update again
2022-07-22 15:31:30 +00:00
6ba4a66deb
Squashed commit of the following:
...
commit 31a47a5ee4569264e39d7c445525a6e64bb7700a
Author: Primoz <sisko.primoz@gmail.com>
Date: Wed Jul 20 13:49:22 2022 +0000
Environment version change.
commit 5b274ed8993f58e783bda6d82fce936764209c28
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Jul 19 16:10:07 2022 +0000
Enabled cleaning for all participants + standardization files.
commit 203fdb31e0f3c647ef8c8a60cb9531831b7ab924
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Jul 19 14:14:51 2022 +0000
Features cleaning fixes after testing. Visualization script for phone features values.
commit 176178d73b154c30b9eb9eb4a67514f00d6a924e
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Jul 19 09:05:14 2022 +0000
Revert "Necessary config changes."
This reverts commit 6ec1ef50430d2e1f5ce4670d505d5e84ac47f0a0.
commit 26ea6512c9d512f95837e7b047fe510c1d196403
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 18 13:19:47 2022 +0000
Adding cleaning function condition and cleaning functionality.
commit 575c29eef9c21e6f2d7832871e73bc0941643734
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 18 12:51:56 2022 +0000
Translation of the cleaning individual RAPIDS function from R to py.
commit 6ec1ef50430d2e1f5ce4670d505d5e84ac47f0a0
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 18 12:02:18 2022 +0000
Necessary config changes.
commit b5669f51612fbd8378848615d639677851ab032f
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 15 15:26:00 2022 +0000
Modified snakemake rule to dynamically choose script extention.
commit 66636be1e8ae4828228b37c59b9df1faf3fc3d3d
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 15 14:43:08 2022 +0000
Trying to modify the snakefile rule to execute scripts in two languages depended on the provider.
commit 574778b00f3cbb368ef4bc74de15cf5070c65ea9
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 15 09:49:41 2022 +0000
gitignore: adding required files so that RAPIDS can be run successfully.
commit 71018ab178256970535e78961602ab8c7f0ebb14
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 15 08:34:19 2022 +0000
Standardization bug fixes
commit 6253c470a624e6bfbb02e0c453b652452eb2dbbc
Author: Primoz <sisko.primoz@gmail.com>
Date: Thu Jul 14 15:28:02 2022 +0000
Seperate rules for empatica vs. nonempatica standardization.
Parameter in config that controls the creation of standardized merged files for individual and all participants..
commit 90f902778565e0896d3bae22ae8551be8b487e67
Author: Primoz <sisko.primoz@gmail.com>
Date: Tue Jul 12 14:23:03 2022 +0000
Preparing for final csvs' standardization.
commit d25dde3998786a9a582f5cda544ee104386778f9
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 11 12:08:47 2022 +0000
Revert "Changes in config to be reverted."
This reverts commit bea7608e7095021fb7c53a9afa07074448fe4313.
commit 6b23e70857e63deda98eb98d190af9090626c84b
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 11 12:08:26 2022 +0000
Enabled standardization for rest (previously active) phone features.
Testing still needed.
commit 8ec58a6f34ba3d42e5cc71d26e6d91837472ca5f
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 11 09:07:55 2022 +0000
Enabled standardization for phone calls.
All steps completed and tested.
commit bea7608e7095021fb7c53a9afa07074448fe4313
Author: Primoz <sisko.primoz@gmail.com>
Date: Mon Jul 11 07:47:51 2022 +0000
Changes in config to be reverted.
commit 4e84ca0e51bf709bff56fd09437b95310ec6bedd
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 8 14:11:24 2022 +0000
Standardization for the rest of the features.
commit cc581aa788e3d5c17131af8f3d5dd6b0c3b5aff7
Author: Primoz <sisko.primoz@gmail.com>
Date: Fri Jul 8 14:11:08 2022 +0000
README update again
2022-07-20 13:51:22 +00:00
788ac31190
Bug fix: if df has no rows write an empty zscore file.
2022-07-08 10:40:45 +00:00
21eb2665d7
README: few changes.
2022-07-08 10:40:08 +00:00
a65a85cce9
Merge branch 'empatica_calculating_features'
2022-07-07 15:35:47 +00:00
fa961fe2f5
gitignore
2022-07-07 15:34:31 +00:00
6c8014ba8e
Updated R and Python package files. Updated README.
2022-07-07 15:30:07 +00:00
5a777ac79f
Working version that integrates both phone and empatica feature calculations.
2022-07-07 15:00:47 +00:00
0425403951
Merge branch 'master' of https://repo.ijs.si/junoslukan/rapids
2022-07-06 11:53:31 +00:00
887fd7dc72
Merge branch 'empatica_calculating_features'
2022-07-06 11:53:21 +00:00
5a4696c548
Misc. changes
2022-07-06 11:30:18 +00:00
d2758eef46
Set not NaN sum insted of 0 sum for HRV features windows.
2022-07-06 07:36:35 +00:00
2d5d23b615
Testing files change and remove standardization from hrv sensors main files.
2022-07-06 07:35:39 +00:00
a5480f1369
Few changes during addition to file structure.
2022-07-04 13:00:47 +00:00
505c3a86b9
Testing different EDA findPeaks parameters.
2022-06-30 15:15:37 +00:00
ce04394679
Merge commit 'c05b047c2d9452151553961928c846c01d7395bc'
2022-06-25 20:06:24 +02:00
c851ab0763
Fill EDA NaN values where numPeak is zero. Other small changes.
2022-06-21 14:09:49 +00:00
a8cd16f88c
Debugging eda_explorer
2022-06-16 11:32:59 +00:00
dda4554d46
Various small changes.
2022-06-15 13:57:46 +00:00
212cf300f8
Debugging EDA signal - preliminary step for imputation.
2022-06-14 15:09:14 +00:00
9ea39dc557
Standardization as a Snakefile's rule enabled for all E4 sensors.
2022-06-13 18:17:30 +00:00
402059871f
Making standardization as a rule. WIP: done only for BVP.
2022-06-13 14:12:03 +00:00
094743244d
Added SO feature for sum all rows that are non zero for BVP and IBI sensors.
2022-06-13 10:51:22 +00:00
e1d7607de4
Extraction of additional SO features. Min/max has been changed to nsmallest/nlargest means.
2022-06-10 12:34:48 +00:00
f371249b99
First order features standardization WIP
2022-06-09 13:35:15 +00:00
64e41cfa35
Second order features standardization in config.yaml.
2022-06-07 10:39:48 +00:00
2c7ac21465
Added standardization on SO features.
2022-06-06 13:51:15 +00:00
2acf6ff9fb
Exception handling in case of empty ibi. Changes of the method EDA uses in main.py. Other small corrections.
2022-06-03 12:34:36 +00:00
d300f0f8f0
Fixed RAPIDS bug: error when IBI.csv is empty.
2022-06-02 11:43:49 +00:00
fbf6a77dfc
Small misc changes
2022-06-02 06:41:53 +00:00
5532043b1f
Patching IBI with BVP - completed.
2022-05-25 19:39:47 +00:00
bb62497ba6
Patching IBI with BVP - selecting appropriate pipeline entry point. WIP
2022-05-24 11:07:18 +00:00
Primoz
junos