a6a37c7bd9
Drop NaN targets.
...
This mirrors INNER join in merge_features_and_targets_for_individual_model.py:
data = pd.concat([sensor_features, targets[["target"]]], axis=1, join="inner")
2022-04-12 17:01:49 +02:00
9f5edf1c2b
Revert "Add a rule for model baselines."
...
The example was for a classification rather than regression problem.
This reverts commit 9ab0c8f289
2022-04-12 16:59:42 +02:00
4ad261fae5
Rename baseline features AGAIN.
...
Correct other mistakes.
2022-04-12 16:55:01 +02:00
9ab0c8f289
Add a rule for model baselines.
...
Add baselines and helper functions to main models dir.
2022-04-12 14:23:58 +02:00
f5688f6154
Add a rule to merge sensor and baseline features.
...
And select target as before.
2022-04-08 15:42:04 +02:00
b1f356c3f7
Extract a function to be used elsewhere.
2022-04-08 15:36:32 +02:00
7ff3dcf5fc
Move and rename target variable.
2022-04-06 18:21:09 +02:00
50c0defca7
Select target columns (no parsing necessary).
2022-04-06 18:16:49 +02:00
ac86221662
[WIP] Add a rule to parse targets.
...
Does nothing for now.
2022-04-06 17:47:03 +02:00
d326a1b09d
Include the constant directly in main.py.
2022-04-05 19:08:43 +02:00
2e545e81f0
Include feature calculations for different scales.
2022-04-05 19:05:34 +02:00
cbc8ae4e03
Add necessary checks for empty data frames.
2022-04-05 18:58:09 +02:00
751b04f3f4
Pass scale names to Snakemake correctly.
2022-04-05 18:14:37 +02:00
99245afca3
Try a different approach for preprocessing ESMs.
...
It is important that this follows generic RAPIDS pattern.
In the subsequent step of calculating features,
there is an expected file and folder structure of data/interim.
See rules/common.smk/find_features_files()
2022-04-05 18:02:31 +02:00
ed298a9479
Implement the basic feature extraction steps.
2022-04-05 15:46:02 +02:00
3af8de6235
Create feature provider script.
2022-03-30 10:40:53 +02:00
9478dc94f2
Add an else.
...
This is to make sure that in case the reversing fails, we do not get any output items.
Snakemake will inform us of an error in this event.
2022-03-30 10:40:53 +02:00
c6144f8403
Reverse JCQ items.
2022-03-16 18:55:46 +01:00
23f0aaba3a
Get the name of the questionnaire from Snakefile.
2022-03-16 18:28:57 +01:00
679f00dc19
Enable selecting any questionnaire as target.
2022-03-16 17:55:44 +01:00
1374eda171
Flatten questionnaire ID dict.
2022-03-16 17:38:09 +01:00
19b9da0ba3
Separate function definitions from main.
2022-03-16 16:49:28 +01:00
ef57103bac
Add questionnaire ID key.
2022-03-15 13:41:33 +01:00
5f293211a7
Reformat.
2022-03-15 13:28:51 +01:00
d470eef27e
Add a rule to preprocess and clean ESM.
2022-03-09 18:38:46 +01:00
d4a4bbbff0
Remove unused columns.
2022-03-09 17:58:36 +01:00
085a6d144b
Add files to compute and create an empty script.
2022-03-09 17:32:02 +01:00
42d62f16d0
Add RAPIDS mandatory columns for ESM.
2022-03-09 17:31:37 +01:00
2bef86b1da
Add a format for ESM and add to config.
2022-03-08 15:43:25 +01:00
d8e9a309f7
Rename features and write baseline_interim.
2022-03-08 15:10:36 +01:00
a3a4f04ffe
Setting with : produces NaNs.
2022-03-01 12:02:57 +01:00
aedb8b6785
Write questionnaire data to data/interim.
2022-03-01 12:02:36 +01:00
b5a6317f4b
Calculate JCQ control and demand control ratio.
...
Include norms and corresponding quartile.
2022-02-28 18:51:47 +01:00
2fed962644
Calculate JCQ demand score.
...
Hardcode question IDs to be reversed.
2022-02-28 18:30:41 +01:00
30ac8b1cd5
Start calculating demand control features.
2022-02-23 19:08:10 +01:00
9a74e74d08
Add the baseline features rule to snakefile.
...
Correct age calculation for a single value instead of dataframe.
2022-02-23 18:15:26 +01:00
07da6be398
Add age, gender, and language as features.
...
Move calculation of age from merge_baseline_data.py to baseline_features.py.
2022-02-23 18:05:23 +01:00
176367631b
Prepare baseline feature rule.
2022-02-23 11:09:33 +01:00
bf9c764c97
Split baseline data to participants.
...
And some csv I/O settings.
2022-02-04 18:37:57 +01:00
16e608db74
First merge baseline datasets.
2022-02-04 18:21:42 +01:00
204f6f50b0
Read the relevant files.
2022-02-04 18:06:02 +01:00
685ed6a546
Set up demographic data download.
2022-02-04 17:37:00 +01:00
afa3b8546f
Mutate data in an R script.
...
The Python script did not read the timestamp correctly for some reason. All timestamps were 0.
2022-01-26 16:34:19 +01:00
b17a7eff1a
Deal with inexplicable snakemake failure.
2022-01-07 18:11:38 +01:00
e1499a5ae2
Account for missing device_ids.
2021-12-15 20:41:28 +01:00
5a9252e46e
Merge remote-tracking branch 'origin/master'
2021-12-15 18:32:36 +01:00
352598f3da
Use absolute path to avoid RuleException.
2021-12-15 17:27:13 +01:00
70cada8bb8
Consider a subset of columns when dropping.
2021-12-15 16:14:33 +01:00
d2ed73dccf
Debug ValueError for index.
...
See exploration/debug_heatmap.py for illustration.
2021-12-15 16:03:04 +01:00
6f451e05ac
Bring back application_name.
...
This column still needs to be in the data, so add it in app_add_name.py.
Later, join categories by package hash.
2021-12-15 12:58:27 +01:00
junos