Update docs

docs/citation.md

@@ -0,0 +1,29 @@
+# Cite RAPIDS and providers
+
+!!! done "RAPIDS and the community"
+    RAPIDS is a community effort and as such we want to continue recognizing the contributions from other researchers. Besides citing RAPIDS, we ask you to cite any of the authors listed below if you used those sensor providers in your analysis, thank you!
+
+## RAPIDS
+
+If you used RAPIDS, please cite [this paper](https://preprints.jmir.org/preprint/23246).
+
+!!! cite "RAPIDS et al. citation"
+    Vega J, Li M, Aguillera K, Goel N, Joshi E, Durica KC, Kunta AR, Low CA
+    RAPIDS: Reproducible Analysis Pipeline for Data Streams Collected with Mobile Devices
+    JMIR Preprints. 18/08/2020:23246
+    DOI: 10.2196/preprints.23246
+    URL: https://preprints.jmir.org/preprint/23246
+
+## Panda (accelerometer)
+
+If you computed accelerometer features using the provider  `[PHONE_ACCLEROMETER][PANDA]` cite [this paper](https://pubmed.ncbi.nlm.nih.gov/31657854/) in addition to RAPIDS.
+
+!!! cite "Panda et al. citation"
+    Panda N, Solsky I, Huang EJ, Lipsitz S, Pradarelli JC, Delisle M, Cusack JC, Gadd MA, Lubitz CC, Mullen JT, Qadan M, Smith BL, Specht M, Stephen AE, Tanabe KK, Gawande AA, Onnela JP, Haynes AB. Using Smartphones to Capture Novel Recovery Metrics After Cancer Surgery. JAMA Surg. 2020 Feb 1;155(2):123-129. doi: 10.1001/jamasurg.2019.4702. PMID: 31657854; PMCID: PMC6820047.
+
+## Stachl (applications foreground)
+
+If you computed applications foreground features using the app category (genre) catalogue in  `[PHONE_APPLICATIONS_FOREGROUND][RAPIDS]` cite [this paper](https://www.pnas.org/content/117/30/17680) in addition to RAPIDS.
+
+!!! cite "Stachl et al. citation"
+    Clemens Stachl, Quay Au, Ramona Schoedel, Samuel D. Gosling, Gabriella M. Harari, Daniel Buschek, Sarah Theres Völkel, Tobias Schuwerk, Michelle Oldemeier, Theresa Ullmann, Heinrich Hussmann, Bernd Bischl, Markus Bühner. Proceedings of the National Academy of Sciences Jul 2020, 117 (30) 17680-17687; DOI: 10.1073/pnas.1920484117 

docs/features/phone-accelerometer.md

@@ -0,0 +1,77 @@
+# Phone Accelerometer
+
+## RAPIDS provider
+
+!!! info "Available day segments and platforms"
+    - Available for all day segments
+    - Available for Android and iOS
+
+!!! info "File Sequence"
+    ```bash
+    - data/raw/{pid}/phone_accelerometer_raw.csv
+    - data/raw/{pid}/phone_accelerometer_with_datetime.csv
+    - data/interim/{pid}/phone_accelerometer_features/phone_accelerometer_{language}_{provider_key}.csv
+    - data/processed/features/{pid}/phone_accelerometer.csv
+    ```
+
+
+Parameters description for `[PHONE_ACCELEROMETER][PROVIDERS][RAPIDS]`:
+
+|Key                                         | Description |
+|----------------|-----------------------------------------------------------------------------------------------------------------------------------
+|`[COMPUTE]`| Set to `True` to extract `PHONE_ACCELEROMETER` features from the `RAPIDS` provider|
+|`[FEATURES]` |         Features to be computed, see table below
+
+
+Features description for `[PHONE_ACCELEROMETER][PROVIDERS][RAPIDS]`:
+
+|Feature                    |Units      |Description|
+|-------------------------- |---------- |---------------------------|
+|maxmagnitude      |m/s^2^    |The maximum magnitude of acceleration ($\|acceleration\| = \sqrt{x^2 + y^2 + z^2}$).
+|minmagnitude      |m/s^2^    |The minimum magnitude of acceleration.
+|avgmagnitude      |m/s^2^    |The average magnitude of acceleration.
+|medianmagnitude   |m/s^2^    |The median magnitude of acceleration.
+|stdmagnitude      |m/s^2^    |The standard deviation of acceleration.
+
+!!! note "Assumptions/Observations"
+    1. Analyzing accelerometer data is a memory intensive task. If RAPIDS crashes is likely because the accelerometer dataset for a participant is to big to fit in memory. We are considering different alternatives to overcome this problem.
+
+## PANDA provider
+
+These features are based on the work by [Panda et al](/citation#panda-accelerometer).
+
+!!! info "Available day segments and platforms"
+    - Available for all day segments
+    - Available for Android and iOS
+
+!!! info "File Sequence"
+    ```bash
+    - data/raw/{pid}/phone_accelerometer_raw.csv
+    - data/raw/{pid}/phone_accelerometer_with_datetime.csv
+    - data/interim/{pid}/phone_accelerometer_features/phone_accelerometer_{language}_{provider_key}.csv
+    - data/processed/features/{pid}/phone_accelerometer.csv
+    ```
+
+
+Parameters description for `[PHONE_ACCELEROMETER][PROVIDERS][PANDA]`:
+
+|Key                                         | Description |
+|----------------|-----------------------------------------------------------------------------------------------------------------------------------
+|`[COMPUTE]`| Set to `True` to extract `PHONE_ACCELEROMETER` features from the `PANDA` provider|
+|`[FEATURES]` |         Features to be computed for exertional and non-exertional activity episodes, see table below
+
+
+Features description for `[PHONE_ACCELEROMETER][PROVIDERS][PANDA]`:
+
+|Feature                    |Units      |Description|
+|-------------------------- |---------- |---------------------------|
+| sumduration    | minutes | Total duration of all exertional or non-exertional activity episodes.                     |
+| maxduration    | minutes | Longest duration of any exertional or non-exertional activity episode.                    |
+| minduration    | minutes | Shortest duration of any exertional or non-exertional activity episode.                   |
+| avgduration    | minutes | Average duration of any exertional or non-exertional activity episode.                    |
+| medianduration | minutes | Median duration of any exertional or non-exertional activity episode.                     |
+| stdduration    | minutes | Standard deviation of the duration of all exertional or non-exertional activity episodes. |
+
+!!! note "Assumptions/Observations"
+    1. Analyzing accelerometer data is a memory intensive task. If RAPIDS crashes is likely because the accelerometer dataset for a participant is to big to fit in memory. We are considering different alternatives to overcome this problem.
+    2. See [Panda et al](/citation#panda-accelerometer) for a definition of exertional and non-exertional activity episodes

docs/features/phone-applications-foreground.md

@@ -0,0 +1,56 @@
+# Phone Applications Foreground
+
+Sensor parameters description for `[PHONE_APPLICATIONS_FOREGROUND]` (these parameters are used by the only provider available at the moment, RAPIDS):
+
+|Key                                         | Description |
+|----------------|-----------------------------------------------------------------------------------------------------------------------------------
+|`[APPLICATION_CATEGORIES][CATALOGUE_SOURCE]` | `FILE` or `GOOGLE`. If `FILE`, app categories (genres) are read from `[CATALOGUE_FILE]`. If `[GOOGLE]`, app categories (genres) are scrapped from the Play Store
+|`[APPLICATION_CATEGORIES][CATALOGUE_FILE]` | CSV file with a `package_name` and `genre` column. By default we provide the catalogue created by [Stachl et al](/citation#stachl-application-foreground) in `data/external/stachl_application_genre_catalogue.csv`
+|`[APPLICATION_CATEGORIES][UPDATE_CATALOGUE_FILE]` | if `[CATALOGUE_SOURCE]` is equal to `FILE`, this flag signals whether or not to update `[CATALOGUE_FILE]`, if `[CATALOGUE_SOURCE]` is equal to `GOOGLE` all scraped genres will be saved to `[CATALOGUE_FILE]`
+|`[APPLICATION_CATEGORIES][SCRAPE_MISSING_CATEGORIES]` | This flag signals whether or not to scrape categories (genres) missing from the `[CATALOGUE_FILE]`. If `[CATALOGUE_SOURCE]` is equal to `GOOGLE`, all genres are scraped anyway (this flag is ignored)
+
+## RAPIDS provider
+
+The app category (genre) catalogue used in these features was originally created by [Stachl et al](/citation#stachl-application-foreground).
+
+!!! info "Available day segments and platforms"
+    - Available for all day segments
+    - Available for Android only
+
+!!! info "File Sequence"
+    ```bash
+    - data/raw/{pid}/phone_applications_foreground_raw.csv
+    - data/raw/{pid}/phone_applications_foreground_with_datetime.csv
+    - data/raw/{pid}/phone_applications_foreground_with_datetime_with_categories.csv
+    - data/interim/{pid}/phone_applications_foreground_features/phone_applications_foreground_{language}_{provider_key}.csv
+    - data/processed/features/{pid}/phone_applications_foreground.csv
+    ```
+
+
+Parameters description for `[PHONE_APPLICATIONS_FOREGROUND][PROVIDERS][RAPIDS]`:
+
+|Key                                                    | Description |
+|----------------|-----------------------------------------------------------------------------------------------------------------------------------
+|`[COMPUTE]`| Set to `True` to extract `PHONE_APPLICATIONS_FOREGROUND` features from the `RAPIDS` provider|
+|`[FEATURES]` |         Features to be computed, see table below
+|`[SINGLE_CATEGORIES]`     | An array of app categories to be *included* in the feature extraction computation. The special keyword `all` represents a category with all the apps from each participant. By default we use the category catalogue pointed by `[APPLICATION_CATEGORIES][CATALOGUE_FILE]` (see the Sensor parameters description table above)
+|`[MULTIPLE_CATEGORIES]`   | An array of collections representing meta-categories (a group of categories). They key of each element is the name of the `meta-category` and the value is an array of member app categories. By default we use the category catalogue pointed by `[APPLICATION_CATEGORIES][CATALOGUE_FILE]` (see the Sensor parameters description table above)
+|`[SINGLE_APPS]`           | An array of apps to be *included* in the feature extraction computation. Use their package name (e.g. `com.google.android.youtube`) or the reserved keyword `top1global` (the most used app by a participant over the whole monitoring study)
+|`[EXCLUDED_CATEGORIES]`   | An array of app categories to be *excluded* from the feature extraction computation. By default we use the category catalogue pointed by `[APPLICATION_CATEGORIES][CATALOGUE_FILE]` (see the Sensor parameters description table above)
+|`[EXCLUDED_APPS]`         | An array of apps to be excluded from the feature extraction computation. Use their package name, for example: `com.google.android.youtube`
+
+Features description for `[PHONE_APPLICATIONS_FOREGROUND][PROVIDERS][RAPIDS]`:
+
+|Feature                    |Units      |Description|
+|-------------------------- |---------- |---------------------------|
+|count              |apps      | Number of times a single app or apps within a category were used (i.e. they were brought to the foreground either by tapping their icon or switching to it from another app)
+|timeoffirstuse     |minutes   | The time in minutes between 12:00am (midnight) and the first use of a single app or apps within a category during a `day_segment`
+|timeoflastuse      |minutes   | The time in minutes between 12:00am (midnight) and the last use of a single app or apps within a category during a `day_segment`
+|frequencyentropy   |nats      | The entropy of the used apps within a category during a `day_segment` (each app is seen as a unique event, the more apps were used, the higher the entropy). This is especially relevant when computed over all apps. Entropy cannot be obtained for a single app
+
+!!! note "Assumptions/Observations"
+    Features can be computed by app, by apps grouped under a single category (genre) and by multiple categories grouped together (meta-categories). For example, we can get features for `Facebook` (single app), for `Social Network` apps (a category including Facebook and other social media apps) or for `Social` (a meta-category formed by `Social Network` and `Social Media Tools` categories).
+
+    Apps installed by default like YouTube are considered systems apps on some phones. We do an exact match to exclude apps where "genre" == `EXCLUDED_CATEGORIES` or "package_name" == `EXCLUDED_APPS`.
+
+    We provide three ways of classifying and app within a category (genre): a) by automatically scraping its official category from the Google Play Store, b) by using the catalogue created by Stachl et al. which we provide in RAPIDS (`data/external/stachl_application_genre_catalogue.csv`), or c) by manually creating a personalized catalogue. You can choose a, b or c by modifying `[APPLICATION_GENRES]` keys and values (see the Sensor parameters description table above).

docs/file-structure.md

@@ -1,5 +1,8 @@
 # File Structure
 
+!!! tip
+    Read this page if you want to learn more about how RAPIDS is structured. If you want to start using it go to [Installation](https://www.rapids.science/setup/installation/) and then to [Initial Configuration](https://www.rapids.science/setup/configuration/)
+
 All paths mentioned in this page are relative to RAPIDS' root folder.
 
 If you want to extract the behavioral features that RAPIDS offers, you will only have to create or modify the [`.env` file](https://www.rapids.science/setup/configuration/#database-credentials), [participants files](https://www.rapids.science/setup/configuration/#participant-files), [day segment files](https://www.rapids.science/setup/configuration/#day-segments), and the `config.yaml` file. The `config.yaml` file is the heart of RAPIDS and includes parameters to manage participants, data sources, sensor data, visualizations and more.

docs/setup/installation.md

@@ -24,11 +24,16 @@ You can install RAPIDS using Docker (the fastest), or native instructions for Ma
         git pull
         ```
     
-    5.  Check that RAPIDS is working
+    5. Make RAPIDS script executable
+        ```bash
+        chmod +x rapids
+        ```
+    
+    6.  Check that RAPIDS is working
         ``` bash
         ./rapids -j1
         ```
-    6.  *Optional*. You can edit RAPIDS files with vim but we recommend using Visual Studio Code and its Remote Containers extension
+    7.  *Optional*. You can edit RAPIDS files with vim but we recommend using Visual Studio Code and its Remote Containers extension
 
         ??? info "How to configure Remote Containers extension"
 
@@ -91,6 +96,11 @@ You can install RAPIDS using Docker (the fastest), or native instructions for Ma
         !!! note
             This step could take several minutes to complete, especially if you have less than 3Gb of RAM or packages need to be compiled from source. Please be patient and let it run until completion.
     
+    5. Make RAPIDS script executable
+        ```bash
+        chmod +x rapids
+        ```
+
     8.  Check that RAPIDS is working
         ``` bash
         ./rapids -j1
@@ -182,6 +192,11 @@ You can install RAPIDS using Docker (the fastest), or native instructions for Ma
         !!! note
             This step could take several minutes to complete, especially if you have less than 3Gb of RAM or packages need to be compiled from source. Please be patient and let it run until completion.
 
+    5. Make RAPIDS script executable
+        ```bash
+        chmod +x rapids
+        ```
+
     8.  Check that RAPIDS is working
         ``` bash
         ./rapids -j1

mkdocs.yml

@@ -9,7 +9,8 @@ markdown_extensions:
     - codehilite:
         linenums: True
     # - urlize # requires: pip install git+https://github.com/r0wb0t/markdown-urlize.git
-    - pymdownx.arithmatex
+    - pymdownx.arithmatex:
+        generic: true
     - pymdownx.betterem:
         smart_enable: all
     - pymdownx.caret
@@ -34,6 +35,11 @@ extra:
   social:
     - icon: fontawesome/brands/twitter
       link: 'https://twitter.com/julio_ui'
+extra_javascript:
+  - javascripts/config.js
+  - https://polyfill.io/v3/polyfill.min.js?features=es6
+  - https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js
+
 repo_name: 'carissalow/rapids'
 repo_url: 'https://github.com/carissalow/rapids'
 copyright: 'Released under AGPL'
@@ -48,8 +54,8 @@ theme:
     
 nav:
   - Home: 'index.md'
-  - File Structure: file-structure.md
   - Setup:
+    - File Structure: file-structure.md
     - Installation: 'setup/installation.md'
     - Initial Configuration: setup/configuration.md
     - Execution: setup/execution.md
@@ -64,4 +70,7 @@ nav:
         - Phone Bluetooth: features/phone-bluetooth.md
         - Phone WiFI Visible: features/phone-wifi-visible.md
         - Phone WiFI Connected: features/phone-wifi-connected.md
+        - Phone Accelerometer: features/phone-accelerometer.md
+        - Phone Applications Foreground: features/phone-applications-foreground.md
   - Frequently Asked Questions: faq.md
+  - Citation: citation.md