Describe LOSO results.

ml_pipeline
junos 2023-01-04 20:38:05 +01:00
parent 37eada4a2e
commit 61d786b2ca
2 changed files with 25 additions and 1 deletions

View File

@ -3,5 +3,6 @@
<component name="VcsDirectoryMappings">
<mapping directory="$PROJECT_DIR$" vcs="Git" />
<mapping directory="$PROJECT_DIR$/rapids" vcs="Git" />
<mapping directory="$PROJECT_DIR$/rapids/calculatingfeatures" vcs="Git" />
</component>
</project>

View File

@ -12,6 +12,7 @@ knitr::opts_chunk$set(
```{r libraries, include=FALSE}
library(knitr)
library(kableExtra)
library(stringr)
library(RColorBrewer)
library(magrittr)
library(tidyverse)
@ -77,4 +78,26 @@ podatki %>%
# Problem description
We are trying to predict whether a stressful event occurred, i.e. stressfulness > 0, or not (stressfulness == 0).
First
First, set up a leave-one-subject-out validation and use original distribution of the class variable.
For this, the majority classifier has a mean accuracy of 0.85 (and median 0.90), while the F1-score, precision and recall are all 0.
We also have an option to validate the results differently, such as with "half-loso", i.e. leaving half of the subject's data in the training set and only use half for testing, or k-fold cross-validation.
Additionally, we can undersample the majority class to balance the dataset.
# Results
## Leave one subject out, original distribution
```{r event_detection}
scores <- read_csv("event_stressful_detection_loso.csv", col_types = "ffdd")
scores_wide <- scores %>%
select(!max) %>%
pivot_wider(names_from = metric,
names_sep = "_",
values_from = mean) %>%
rename_all(~str_replace(.,"^test_",""))
kable(scores_wide, digits = 2) %>%
column_spec(4, color = 'white', background = 'black') %>%
kable_styling(full_width = TRUE)
```