Describe LOSO results.
parent
37eada4a2e
commit
61d786b2ca
|
@ -3,5 +3,6 @@
|
||||||
<component name="VcsDirectoryMappings">
|
<component name="VcsDirectoryMappings">
|
||||||
<mapping directory="$PROJECT_DIR$" vcs="Git" />
|
<mapping directory="$PROJECT_DIR$" vcs="Git" />
|
||||||
<mapping directory="$PROJECT_DIR$/rapids" vcs="Git" />
|
<mapping directory="$PROJECT_DIR$/rapids" vcs="Git" />
|
||||||
|
<mapping directory="$PROJECT_DIR$/rapids/calculatingfeatures" vcs="Git" />
|
||||||
</component>
|
</component>
|
||||||
</project>
|
</project>
|
|
@ -12,6 +12,7 @@ knitr::opts_chunk$set(
|
||||||
```{r libraries, include=FALSE}
|
```{r libraries, include=FALSE}
|
||||||
library(knitr)
|
library(knitr)
|
||||||
library(kableExtra)
|
library(kableExtra)
|
||||||
|
library(stringr)
|
||||||
library(RColorBrewer)
|
library(RColorBrewer)
|
||||||
library(magrittr)
|
library(magrittr)
|
||||||
library(tidyverse)
|
library(tidyverse)
|
||||||
|
@ -77,4 +78,26 @@ podatki %>%
|
||||||
# Problem description
|
# Problem description
|
||||||
|
|
||||||
We are trying to predict whether a stressful event occurred, i.e. stressfulness > 0, or not (stressfulness == 0).
|
We are trying to predict whether a stressful event occurred, i.e. stressfulness > 0, or not (stressfulness == 0).
|
||||||
First
|
First, set up a leave-one-subject-out validation and use original distribution of the class variable.
|
||||||
|
|
||||||
|
For this, the majority classifier has a mean accuracy of 0.85 (and median 0.90), while the F1-score, precision and recall are all 0.
|
||||||
|
|
||||||
|
We also have an option to validate the results differently, such as with "half-loso", i.e. leaving half of the subject's data in the training set and only use half for testing, or k-fold cross-validation.
|
||||||
|
Additionally, we can undersample the majority class to balance the dataset.
|
||||||
|
|
||||||
|
# Results
|
||||||
|
## Leave one subject out, original distribution
|
||||||
|
|
||||||
|
```{r event_detection}
|
||||||
|
scores <- read_csv("event_stressful_detection_loso.csv", col_types = "ffdd")
|
||||||
|
scores_wide <- scores %>%
|
||||||
|
select(!max) %>%
|
||||||
|
pivot_wider(names_from = metric,
|
||||||
|
names_sep = "_",
|
||||||
|
values_from = mean) %>%
|
||||||
|
rename_all(~str_replace(.,"^test_",""))
|
||||||
|
kable(scores_wide, digits = 2) %>%
|
||||||
|
column_spec(4, color = 'white', background = 'black') %>%
|
||||||
|
kable_styling(full_width = TRUE)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue