Frequently Asked Questions¶

Cannot connect to your MySQL server¶

Problem

**Error in .local(drv, \...) :** **Failed to connect to database: Error:
Can\'t initialize character set unknown (path: compiled\_in)** :

Calls: dbConnect -> dbConnect -> .local -> .Call
Execution halted
[Tue Mar 10 19:40:15 2020]
Error in rule download_dataset:
    jobid: 531
    output: data/raw/p60/locations_raw.csv

RuleException:
CalledProcessError in line 20 of /home/ubuntu/rapids/rules/preprocessing.snakefile:
Command 'set -euo pipefail;  Rscript --vanilla /home/ubuntu/rapids/.snakemake/scripts/tmp_2jnvqs7.download_dataset.R' returned non-zero exit status 1.
File "/home/ubuntu/rapids/rules/preprocessing.snakefile", line 20, in __rule_download_dataset
File "/home/ubuntu/anaconda3/envs/moshi-env/lib/python3.7/concurrent/futures/thread.py", line 57, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

Solution

Please make sure the DATABASE_GROUP in config.yaml matches your DB credentials group in .env.

Cannot start mysql in linux via `brew services start mysql`¶

Problem

Cannot start mysql in linux via brew services start mysql

Solution

Use mysql.server start

Every time I run force the download_dataset rule all rules are executed¶

Problem

When running snakemake -j1 -R download_phone_data or ./rapids -j1 -R download_phone_data all the rules and files are re-computed

Solution

This is expected behavior. The advantage of using snakemake under the hood is that every time a file containing data is modified every rule that depends on that file will be re-executed to update their results. In this case, since download_dataset updates all the raw data, and you are forcing the rule with the flag -R every single rule that depends on those raw files will be executed.

Error `Table XXX doesn't exist` while running the `download_phone_data` or `download_fitbit_data` rule.¶

Problem

Error in .local(conn, statement, ...) : 
  could not run statement: Table 'db_name.table_name' doesn't exist
Calls: colnames ... .local -> dbSendQuery -> dbSendQuery -> .local -> .Call
Execution halted

Solution

Please make sure the sensors listed in [PHONE_VALID_SENSED_BINS][PHONE_SENSORS] and the [TABLE] of each sensor you activated in config.yaml match your database tables.

How do I install RAPIDS on Ubuntu 16.04¶

Solution

Install dependencies (Homebrew - if not installed):
- sudo apt-get install libmariadb-client-lgpl-dev libxml2-dev libssl-dev
- Install brew for linux and add the following line to ~/.bashrc: export PATH=$HOME/.linuxbrew/bin:$PATH
- source ~/.bashrc
Install MySQL
- brew install mysql
- brew services start mysql
Install R, pandoc and rmarkdown:
- brew install r
- brew install gcc@6 (needed due to this bug)
- HOMEBREW_CC=gcc-6 brew install pandoc
Install miniconda using these instructions
Clone our repo:
- git clone https://github.com/carissalow/rapids
Create a python virtual environment:
- cd rapids
- conda env create -f environment.yml -n MY_ENV_NAME
- conda activate MY_ENV_NAME
Install R packages and virtual environment:
- snakemake renv_install
- snakemake renv_init
- snakemake renv_restore
This step could take several minutes to complete. Please be patient and let it run until completion.

`mysql.h` cannot be found¶

Problem

--------------------------[ ERROR MESSAGE ]----------------------------
<stdin>:1:10: fatal error: mysql.h: No such file or directory
compilation terminated.
-----------------------------------------------------------------------
ERROR: configuration failed for package 'RMySQL'

Solution

sudo apt install libmariadbclient-dev

No package `libcurl` found¶

Problem

libcurl cannot be found

Solution

Install libcurl

sudo apt install libcurl4-openssl-dev

Configuration failed because `openssl` was not found.¶

Problem

openssl cannot be found

Solution

Install openssl

sudo apt install libssl-dev

Configuration failed because `libxml-2.0` was not found¶

Problem

libxml-2.0 cannot be found

Solution

Install libxml-2.0

sudo apt install libxml2-dev

SSL connection error when running RAPIDS¶

Problem

You are getting the following error message when running RAPIDS:

Error: Failed to connect: SSL connection error: error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol.

Solution

This is a bug in Ubuntu 20.04 when trying to connect to an old MySQL server with MySQL client 8.0. You should get the same error message if you try to connect from the command line. There you can add the option --ssl-mode=DISABLED but we can't do this from the R connector.

If you can't update your server, the quickest solution would be to import your database to another server or to a local environment. Alternatively, you could replace mysql-client and libmysqlclient-dev with mariadb-client and libmariadbclient-dev and reinstall renv. More info about this issue here

`DB_TABLES` key not found¶

Problem

If you get the following error KeyError in line 43 of preprocessing.smk: 'PHONE_SENSORS', it means that the indentation of the key [PHONE_SENSORS] is not matching the other child elements of PHONE_VALID_SENSED_BINS

Solution

You need to add or remove any leading whitespaces as needed on that line.

PHONE_VALID_SENSED_BINS:
    COMPUTE: False # This flag is automatically ignored (set to True) if you are extracting PHONE_VALID_SENSED_DAYS or screen or Barnett's location features
    BIN_SIZE: &bin_size 5 # (in minutes)
    PHONE_SENSORS: []

Error while updating your conda environment in Ubuntu¶

Problem

You get the following error:

CondaMultiError: CondaVerificationError: The package for tk located at /home/ubuntu/miniconda2/pkgs/tk-8.6.9-hed695b0_1003
    appears to be corrupted. The path 'include/mysqlStubs.h'
    specified in the package manifest cannot be found.
ClobberError: This transaction has incompatible packages due to a shared path.
    packages: conda-forge/linux-64::llvm-openmp-10.0.0-hc9558a2_0, anaconda/linux-64::intel-openmp-2019.4-243
    path: 'lib/libiomp5.so'

Solution

Reinstall conda

Embedded nul in string¶

Problem

You get the following error when downloading sensor data:

Error in result_fetch(res@ptr, n = n) : 
  embedded nul in string:

Solution

This problem is due to the way RMariaDB handles a mismatch between data types in R and MySQL (see this issue). Since it seems this problem won’t be handled by RMariaDB, you have two options:

If it’s only a few rows that are causing this problem, remove the the null character from the conflictive table cell.
If it’s not feasible to modify your data you can try swapping RMariaDB with RMySQL. Just have in mind you might have problems connecting to modern MySQL servers running in Liunx:
- Add RMySQL to the renv environment by running the following command in a terminal open on RAPIDS root folder
```
R -e 'renv::install("RMySQL")'
```
- Go to src/data/download_phone_data.R and replace library(RMariaDB) with library(RMySQL)
- In the same file replace dbEngine <- dbConnect(MariaDB(), default.file = "./.env", group = group) with dbEngine <- dbConnect(MySQL(), default.file = "./.env", group = group)

Frequently Asked Questions¶

Cannot connect to your MySQL server¶

Cannot start mysql in linux via brew services start mysql¶

Every time I run force the download_dataset rule all rules are executed¶

Error Table XXX doesn't exist while running the download_phone_data or download_fitbit_data rule.¶

How do I install RAPIDS on Ubuntu 16.04¶

mysql.h cannot be found¶

No package libcurl found¶

Configuration failed because openssl was not found.¶

Configuration failed because libxml-2.0 was not found¶

SSL connection error when running RAPIDS¶

DB_TABLES key not found¶

Error while updating your conda environment in Ubuntu¶

Embedded nul in string¶

Cannot start mysql in linux via `brew services start mysql`¶

Error `Table XXX doesn't exist` while running the `download_phone_data` or `download_fitbit_data` rule.¶

`mysql.h` cannot be found¶

No package `libcurl` found¶

Configuration failed because `openssl` was not found.¶

Configuration failed because `libxml-2.0` was not found¶

`DB_TABLES` key not found¶