rapids/docs/develop/testing.rst

68 lines
4.0 KiB
ReStructuredText
Raw Normal View History

2020-05-15 21:58:33 +02:00
Testing
==========
2020-05-18 17:38:39 +02:00
The following is a simple guide to testing RAPIDS. All files necessary for testing are stored in the ``tests`` directory:
2020-05-15 21:58:33 +02:00
::
├── tests
2020-06-25 22:32:45 +02:00
│ ├── data <- Replica of the project root data directory for testing.
│ │ ├── external <- Contains the fake testing participant files.
2020-05-15 21:58:33 +02:00
│ │ ├── interim <- The expected intermediate data that has been transformed.
2020-06-25 22:32:45 +02:00
│ │ ├── processed <- The expected final data, canonical data sets for modeling used to test/validate feature calculations.
│ │ └── raw <- The specially created raw input datasets (fake data) that will be used for testing.
2020-05-15 21:58:33 +02:00
│ │
│ ├── scripts <- Scripts for testing. Add test scripts in this directory.
2020-06-25 22:32:45 +02:00
│ │ ├── run_tests.sh <- The shell script to runs RAPIDS pipeline test data and test the results
│ │ ├── test_sensor_features.py <- The default test script for testing RAPIDS builting sensor features.
2020-05-15 21:58:33 +02:00
│ │ └── utils.py <- Contains any helper functions and methods.
│ │
│ ├── settings <- The directory contains the config and settings files for testing snakemake.
│ │ ├── config.yaml <- Defines the testing profile configurations for running snakemake.
│ │ └── testing_config.yaml <- Contains the actual snakemake configuration settings for testing.
│ │
│ └── Snakefile <- The Snakefile for testing only. It contains the rules that you would be testing.
2020-05-18 17:38:39 +02:00
2020-06-25 22:32:45 +02:00
Steps for Testing
""""""""""""""""""
2020-05-18 17:38:39 +02:00
2020-06-25 22:32:45 +02:00
#. To begin testing RAPIDS place the fake raw input data ``csv`` files in ``tests/data/raw/``. The fake participant files should be placed in ``tests/data/external/``. The expected output files of RAPIDS after processing the input data should be placed in ``tests/data/processesd/``.
2020-05-15 21:58:33 +02:00
2020-06-25 22:32:45 +02:00
#. The Snakemake rule(s) that are to be tested must be placed in the ``tests/Snakemake`` file. The current ``tests/Snakemake`` is a good example of how to define them. (At the time of writing this documentation the snakefile contains rules messages (SMS), calls and screen)
2020-05-15 21:58:33 +02:00
2020-06-25 22:32:45 +02:00
#. Edit the ``tests/settings/config.yaml``. Add and/or remove the rules to be run for testing from the ``forcerun`` list.
2020-05-15 21:58:33 +02:00
2020-06-25 22:32:45 +02:00
#. Edit the ``tests/settings/testing_config.yaml`` with the necessary configuration settings for running the rules to be tested.
2020-05-15 21:58:33 +02:00
2020-06-25 22:32:45 +02:00
#. Add any additional testscripts in ``tests/scripts``.
2020-05-15 21:58:33 +02:00
2020-06-25 22:32:45 +02:00
#. Uncomment or comment off lines in the testing shell script ``tests/scripts/run_tests.sh``.
2020-05-15 21:58:33 +02:00
2020-06-25 22:32:45 +02:00
#. Run the testing shell script.
2020-05-15 21:58:33 +02:00
::
2020-06-25 22:32:45 +02:00
$ tests/scripts/run_tests.sh
2020-05-15 21:58:33 +02:00
2020-05-18 17:38:39 +02:00
The following is a snippet of the output you should see after running your test.
2020-05-15 21:58:33 +02:00
::
test_sensors_files_exist (test_sensor_features.TestSensorFeatures) ... ok
test_sensors_features_calculations (test_sensor_features.TestSensorFeatures) ... FAIL
======================================================================
FAIL: test_sensors_features_calculations (test_sensor_features.TestSensorFeatures)
----------------------------------------------------------------------
2020-05-18 17:38:39 +02:00
The results above show that the first test ``test_sensors_files_exist`` passed while ``test_sensors_features_calculations`` failed. In addition you should get the traceback of the failure (not shown here). For more information on how to implement test scripts and use unittest please see `Unittest Documentation`_
2020-05-15 21:58:33 +02:00
2020-06-25 22:32:45 +02:00
Testing of the RAPIDS sensors and features is a work-in-progess. Please see :ref:`test-cases` for a list of sensors and features that have testing currently available.
Currently the repository is set up to test a number of senssors out of the box by simply running the ``tests/scripts/run_tests.sh`` command once the RAPIDS python environment is active.
2020-05-15 21:58:33 +02:00
.. _`Unittest Documentation`: https://docs.python.org/3.7/library/unittest.html#command-line-interface