junos
/
rapids
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Add FAQ

pull/103/head
JulioV 2020-11-04 16:41:39 -05:00
parent ce55cbda4a
commit 231a5d4265
3 changed files with 215 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

195
docs/faq.md 100644
View File

@ -0,0 +1,195 @@
# Frequently Asked Questions
## Cannot connect to your MySQL server
???+ failure "Problem"
```bash
**Error in .local(drv, \...) :** **Failed to connect to database: Error:
Can\'t initialize character set unknown (path: compiled\_in)** :
Calls: dbConnect -> dbConnect -> .local -> .Call
Execution halted
[Tue Mar 10 19:40:15 2020]
Error in rule download_dataset:
jobid: 531
output: data/raw/p60/locations_raw.csv
RuleException:
CalledProcessError in line 20 of /home/ubuntu/rapids/rules/preprocessing.snakefile:
Command 'set -euo pipefail; Rscript --vanilla /home/ubuntu/rapids/.snakemake/scripts/tmp_2jnvqs7.download_dataset.R' returned non-zero exit status 1.
File "/home/ubuntu/rapids/rules/preprocessing.snakefile", line 20, in __rule_download_dataset
File "/home/ubuntu/anaconda3/envs/moshi-env/lib/python3.7/concurrent/futures/thread.py", line 57, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
```
???+ done "Solution"
Please make sure the `DATABASE_GROUP` in `config.yaml` matches your DB credentials group in `.env`.
---
## Cannot start mysql in linux via `brew services start mysql`
???+ failure "Problem"
Cannot start mysql in linux via `brew services start mysql`
???+ done "Solution"
Use `mysql.server start`
---
## Every time I run force the download_dataset rule all rules are executed
???+ failure "Problem"
When running `snakemake -j1 -R download_phone_data` or `./rapids -j1 -R download_phone_data` all the rules and files are re-computed
???+ done "Solution"
This is expected behavior. The advantage of using `snakemake` under the hood is that every time a file containing data is modified every rule that depends on that file will be re-executed to update their results. In this case, since `download_dataset` updates all the raw data, and you are forcing the rule with the flag `-R` every single rule that depends on those raw files will be executed.
---
## Error `Table XXX doesn't exist` while running the `download_phone_data` or `download_fitbit_data` rule.
???+ failure "Problem"
```bash
Error in .local(conn, statement, ...) :
could not run statement: Table 'db_name.table_name' doesn't exist
Calls: colnames ... .local -> dbSendQuery -> dbSendQuery -> .local -> .Call
Execution halted
```
???+ done "Solution"
Please make sure the sensors listed in `[PHONE_VALID_SENSED_BINS][PHONE_SENSORS]` and the `[TABLE]` of each sensor you activated in `config.yaml` match your database tables.
---
## How do I install RAPIDS on Ubuntu 16.04
???+ done "Solution"
1. Install dependencies (Homebrew - if not installed):
- `sudo apt-get install libmariadb-client-lgpl-dev libxml2-dev libssl-dev`
- Install [brew](https://docs.brew.sh/Homebrew-on-Linux) for linux and add the following line to `~/.bashrc`: `export PATH=$HOME/.linuxbrew/bin:$PATH`
- `source ~/.bashrc`
1. Install MySQL
- `brew install mysql`
- `brew services start mysql`
2. Install R, pandoc and rmarkdown:
- `brew install r`
- `brew install gcc@6` (needed due to this [bug](https://github.com/Homebrew/linuxbrew-core/issues/17812))
- `HOMEBREW_CC=gcc-6 brew install pandoc`
3. Install miniconda using these [instructions](https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html)
4. Clone our repo:
- `git clone https://github.com/carissalow/rapids`
5. Create a python virtual environment:
- `cd rapids`
- `conda env create -f environment.yml -n MY_ENV_NAME`
- `conda activate MY_ENV_NAME`
6. Install R packages and virtual environment:
- `snakemake renv_install`
- `snakemake renv_init`
- `snakemake renv_restore`
This step could take several minutes to complete. Please be patient and let it run until completion.
---
## `mysql.h` cannot be found
???+ failure "Problem"
```bash
--------------------------[ ERROR MESSAGE ]----------------------------
<stdin>:1:10: fatal error: mysql.h: No such file or directory
compilation terminated.
-----------------------------------------------------------------------
ERROR: configuration failed for package 'RMySQL'
```
???+ done "Solution"
```bash
sudo apt install libmariadbclient-dev
```
---
## No package `libcurl` found
???+ failure "Problem"
`libcurl` cannot be found
???+ done "Solution"
Install `libcurl`
```bash
sudo apt install libcurl4-openssl-dev
```
---
## Configuration failed because `openssl` was not found.
???+ failure "Problem"
`openssl` cannot be found
???+ done "Solution"
Install `openssl`
```bash
sudo apt install libssl-dev
```
---
## Configuration failed because `libxml-2.0` was not found
???+ failure "Problem"
`libxml-2.0` cannot be found
???+ done "Solution"
Install `libxml-2.0`
```bash
sudo apt install libxml2-dev
```
---
## SSL connection error when running RAPIDS
???+ failure "Problem"
You are getting the following error message when running RAPIDS:
```bash
Error: Failed to connect: SSL connection error: error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol.
```
???+ done "Solution"
This is a bug in Ubuntu 20.04 when trying to connect to an old MySQL server with MySQL client 8.0. You should get the same error message if you try to connect from the command line. There you can add the option `--ssl-mode=DISABLED` but we can\'t do this from the R connector.
If you can\'t update your server, the quickest solution would be to import your database to another server or to a local environment. Alternatively, you could replace `mysql-client` and `libmysqlclient-dev` with `mariadb-client` and `libmariadbclient-dev` and reinstall renv. More info about this issue [here](https://bugs.launchpad.net/ubuntu/+source/mysql-8.0/+bug/1872541)
---
## `DB_TABLES` key not found
???+ failure "Problem"
If you get the following error `KeyError in line 43 of preprocessing.smk: 'PHONE_SENSORS'`, it means that the indentation of the key `[PHONE_SENSORS]` is not matching the other child elements of `PHONE_VALID_SENSED_BINS`
???+ done "Solution"
You need to add or remove any leading whitespaces as needed on that line.
```yaml
PHONE_VALID_SENSED_BINS:
COMPUTE: False # This flag is automatically ignored (set to True) if you are extracting PHONE_VALID_SENSED_DAYS or screen or Barnett's location features
BIN_SIZE: &bin_size 5 # (in minutes)
PHONE_SENSORS: []
```
---
## Error while updating your conda environment in Ubuntu
???+ failure "Problem"
You get the following error:
```bash
CondaMultiError: CondaVerificationError: The package for tk located at /home/ubuntu/miniconda2/pkgs/tk-8.6.9-hed695b0_1003
appears to be corrupted. The path 'include/mysqlStubs.h'
specified in the package manifest cannot be found.
ClobberError: This transaction has incompatible packages due to a shared path.
packages: conda-forge/linux-64::llvm-openmp-10.0.0-hc9558a2_0, anaconda/linux-64::intel-openmp-2019.4-243
path: 'lib/libiomp5.so'
```
???+ done "Solution"
Reinstall conda

20
docs/setup/execution.md
View File

@ -9,9 +9,25 @@ After you have [installed](/setup/installation) and [configured](/setup/configur
!!! info
The script `#!bash ./rapids` is a wrapper around Snakemake so you can pass any parameters that Snakemake accepts (e.g. `-j1`).
Any changes to the `config.yaml` file will be applied automatically and only the relevant files will be updated.
!!! hint "Updating RAPIDS output after modifying `config.yaml`"
Any changes to the `config.yaml` file will be applied automatically and only the relevant files will be updated. This means that after modifying the features list for `PHONE_MESSAGE` for example, RAPIDS will update the output file with the correct features.
!!! hint "Multi-core"
You can run RAPIDS over multiple cores by modifying the `-j` argument (e.g. use `-j8` to use 8 cores). **However**, take into account that this means multiple sensor datasets for different participants will be load in memory at the same time. If RAPIDS crashes because it ran out of memory reduce the number of cores and try again.
As reference, we have run RAPIDS over 12 cores and 32 Gb of RAM without problems for a study with 200 participants with 14 days of low-frequency smartphone data (no accelerometer, gyroscope, or magnetometer).
As reference, we have run RAPIDS over 12 cores and 32 Gb of RAM without problems for a study with 200 participants with 14 days of low-frequency smartphone data (no accelerometer, gyroscope, or magnetometer).
!!! hint "Forcing a complete rerun"
If you want to update your data from your database or rerun the whole pipeline from scratch run one or both of the following commands depending on the devices you are using:
```bash
./rapids -j1 -R download_phone_data
./rapids -j1 -R download_fitbit_data
```
!!! hint "Deleting RAPIDS output"
If you want to delete all the output files RAPIDS produces you can execute the following command (the content of these folders will be deleted: `data/raw`, `data/interim`, `data/processed`, `reports/figures`, and `reports/compliance`)
```bash
./rapids -j1 -R clean
```

3
mkdocs.yml
View File

@ -52,4 +52,5 @@ pages:
- Initial Configuration: setup/configuration.md
- Execution: setup/execution.md
- Example Workflows:
- Minimal: workflow-examples/minimal.md
- Minimal: workflow-examples/minimal.md
- Frequently Asked Questions: faq.md