273 lines
25 KiB
Markdown
273 lines
25 KiB
Markdown
# Testing
|
|
|
|
The following is a simple guide to run RAPIDS' tests. All files necessary for testing are stored in the `./tests/` directory
|
|
|
|
## Steps for Testing
|
|
|
|
1. **Add raw data.**
|
|
1. Add the raw data to the corresponding sensor CSV file in `tests/data/external/aware_csv`. Create the CSV if it does not exist.
|
|
2. **Link raw data.**
|
|
1. Make sure that you link the new raw data to a participant by using the same `device_id` in the data and in `[DEVICE_IDS]` inside their participant file (`tests/data/external/participant_files/testXX.yaml`).
|
|
2. Create the participant file if it does not exist, and don't forget to edit `[PIDS]` in the config file of the time segments you are testing (see below). For simplicity, we use a participant's id (`testXX`) as their `device_id`.
|
|
3. **Edit the config file.**
|
|
1. Activate the sensor provider you are testing if it isn't already. Set `[SENSOR][PROVIDER][COMPUTE]` to `TRUE` in the `config.yaml` of the time segments you are testing:
|
|
```yaml
|
|
- tests/settings/frequency_config.yaml # For frequency time segments
|
|
- tests/settings/periodic_config.yaml # For periodic time segments
|
|
# We have not tested events time segments yet
|
|
```
|
|
4. **Run the pipeline and tests.**
|
|
1. You can run all time segments pipelines and their tests
|
|
```bash
|
|
tests/scripts/run_tests.sh -t all
|
|
```
|
|
2. You can run only the pipeline of a specific time segment and its tests
|
|
```bash
|
|
tests/scripts/run_tests.sh -t frequency -a both
|
|
```
|
|
2. Or, if you are working on your tests and you want to run a pipeline and its tests independently
|
|
```bash
|
|
tests/scripts/run_tests.sh -t frequency -a run
|
|
tests/scripts/run_tests.sh -t frequency -a test
|
|
```
|
|
|
|
## Output example
|
|
The following is a snippet of the output you should see after running your test.
|
|
|
|
```bash
|
|
test_sensors_files_exist (test_sensor_features.TestSensorFeatures) ... periodic
|
|
ok
|
|
test_sensors_features_calculations (test_sensor_features.TestSensorFeatures) ... periodic
|
|
ok
|
|
|
|
test_sensors_files_exist (test_sensor_features.TestSensorFeatures) ... frequency
|
|
ok
|
|
test_sensors_features_calculations (test_sensor_features.TestSensorFeatures) ... frequency
|
|
FAIL
|
|
```
|
|
|
|
The results above show that the for periodic both `test_sensors_files_exist` and `test_sensors_features_calculations` passed while for frequency first test `test_sensors_files_exist` passed while `test_sensors_features_calculations` failed. Additionally, you should get the traceback of the failure (not shown here). For more information on how to implement test scripts and use unittest please see [Unittest Documentation](https://docs.python.org/3.7/library/unittest.html#command-line-interface)
|
|
|
|
Testing of the RAPIDS sensors and features is a work-in-progress. Please see [Test Cases](../test-cases) for a list of sensors and features that have testing currently available.
|
|
|
|
## How do we execute the tests?
|
|
This bash script `tests/scripts/run_tests.sh` executes one or all pipelines for different time segment types (`frequency`, `periodic`, and `events`) as well as their tests (see below).
|
|
|
|
This python script `tests/scripts/run_tests.py` runs the tests. It parses the involved participants and active sensor providers in the `config.yaml` file of the time segment type being tested. We test that the output file we expect exists and that its content matches the expected values.
|
|
|
|
??? example "Example of raw data for PHONE_APPLICATIONS_FOREGROUND testing"
|
|
```json hl_lines="1 2 4" linenums="1"
|
|
--8<---- "tests/data/external/aware_csv/phone_applications_foreground_raw.csv"
|
|
```
|
|
|
|
## What cases do we test?
|
|
The sample data includes 7 tests cases. Take phone battery as an example, on this platform, battery status 2 represents `charging` and battery status 4 represents `discharge`.
|
|
|
|
??? "1. A daily segment instance with no battery episodes"
|
|
|
|
??? "Example"
|
|
|
|
Input time segments:
|
|
|
|
| timestamp | device_id | battery_status | battery_level | battery_scale | battery_voltage | battery_temperature | battery_adaptor | battery_health | battery_technology |
|
|
|---|---|---|---|---|---|---|---|---|---|
|
|
| 00:08:10.415 | per_ios | 4 | 80 | 100 | 4170 | 23 | 0 | 2 | Li-ion |
|
|
| 00:17:38.602 | per_ios | 4 | 77 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 03:20:30.415 | per_ios | 2 | 77 | 100 | 4170 | 23 | 0 | 2 | Li-ion |
|
|
| 03:30:35.875 | per_ios | 2 | 80 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
|
|
Output results
|
|
|
|
| local_segment | local_segment_label | local_segment_start_datetime | local_segment_end_datetime | phone_battery_rapids_countdischarge | phone_battery_rapids_sumdurationdischarge | phone_battery_rapids_avgconsumptionrate | phone_battery_rapids_maxconsumptionrate | phone_battery_rapids_countcharge | phone_battery_rapids_sumdurationcharge |
|
|
| --- |---|---|---|---|---|---|---|---|---|
|
|
| 00:00:00,00:29:59 | thirtyminutes0000 | 2020-07-01 00:00:00 | 2020-07-01 00:29:59 | 1 | 21.8259833333333 | 0.137450851775292 | 0.137450851775292 | 0 | 0 |
|
|
| 00:03:00,03:29:59 | thirtyminutes0006 | 2020-07-01 03:00:00 | 2020-07-01 03:29:59 | 0 | 0 | 0 | 0 | 1 | 9.49288333333333 |
|
|
|
|
Since there is no battery episode between 00:00:30 and 03:00:00, no result will be generated for this epoch.
|
|
|
|
??? "2. A daily segment instance with two battery episodes (one charging, one discharge)"
|
|
|
|
??? "Periodic (daily)"
|
|
|
|
Input time segments:
|
|
|
|
| timestamp | device_id | battery_status | battery_level | battery_scale | battery_voltage | battery_temperature | battery_adaptor | battery_health | battery_technology |
|
|
|-----------------------|-----------|----------------|---------------|---------------|-----------------|---------------------|-----------------|----------------|--------------------|
|
|
| 17:59:41.434 | per_ios | 4 | 59 | 100 | 4094 | 23 | 0 | 2 | Li-ion |
|
|
| 18:04:14.321 | per_ios | 4 | 58 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 18:07:24.456 | per_ios | 4 | 57 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 20:03:03.415 | per_ios | 2 | 72 | 100 | 4170 | 23 | 0 | 2 | Li-ion |
|
|
| 20:05:12.434 | per_ios | 2 | 73 | 100 | 4094 | 23 | 0 | 2 | Li-ion |
|
|
| 20:07:24.678 | per_ios | 2 | 74 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 20:10:34.875 | per_ios | 2 | 75 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 21:30:04.415 | per_ios | 4 | 74 | 100 | 4170 | 23 | 0 | 2 | Li-ion |
|
|
| 21:32:14.434 | per_ios | 4 | 73 | 100 | 4094 | 23 | 0 | 2 | Li-ion |
|
|
| 21:35:23.678 | per_ios | 4 | 72 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 21:37:47.875 | per_ios | 4 | 71 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
|
|
Output results:
|
|
|
|
| local_segment | local_segment_label | local_segment_start_datetime | local_segment_end_datetime | phone_battery_rapids_countdischarge | phone_battery_rapids_sumdurationdischarge | phone_battery_rapids_avgconsumptionrate | phone_battery_rapids_maxconsumptionrate | phone_battery_rapids_countcharge | phone_battery_rapids_sumdurationcharge |
|
|
| --- |---|---|---|---|---|---|---|---|---|
|
|
| 18:00:00,23:59:59 | evening | 2020-07-01 18:00:00 | 2020-07-01 23:59:59 | 2 | 75.1306166666666 | 0.0664958369201784 | 0.079525673538274 | 1 | 37.5236666666667 |
|
|
|
|
|
|
??? "Frequency (30 mins)"
|
|
|
|
Input time segments:
|
|
|
|
| timestamp | device_id | battery_status | battery_level | battery_scale | battery_voltage | battery_temperature | battery_adaptor | battery_health | battery_technology |
|
|
|---|---|---|---|---|---|---|---|---|---|
|
|
| 20:10:34.875 | fre_ios | 2 | 75 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 20:20:17.171 | fre_ios | 4 | 74 | 100 | 4170 | 23 | 0 | 2 | Li-ion |
|
|
|
|
Output results
|
|
|
|
| local_segment | local_segment_label | local_segment_start_datetime | local_segment_end_datetime | phone_battery_rapids_countdischarge | phone_battery_rapids_sumdurationdischarge | phone_battery_rapids_avgconsumptionrate | phone_battery_rapids_maxconsumptionrate | phone_battery_rapids_countcharge | phone_battery_rapids_sumdurationcharge |
|
|
| --- |---|---|---|---|---|---|---|---|---|
|
|
| 20:00:00,20:29:59 | thirtyminutes0040 | 2020-07-01 20:00:00 | 2020-07-01 20:29:59 | 1 | 14.6351666666667 | 0.0683285693136395 | 0.0683285693136395 | 1 | 12.3074 |
|
|
|
|
|
|
??? "3. A daily segment instance with a charging episode that spans to the next daily instance"
|
|
|
|
??? "Periodic (daily)"
|
|
|
|
Input time segments:
|
|
|
|
| timestamp | device_id | battery_status | battery_level | battery_scale | battery_voltage | battery_temperature | battery_adaptor | battery_health | battery_technology |
|
|
|---|---|---|---|---|---|---|---|---|---|
|
|
| 11:59:28.434 | per_ios | 2 | 63 | 100 | 4094 | 23 | 0 | 2 | Li-ion |
|
|
| 12:04:37.678 | per_ios | 2 | 64 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
|
|
??? "Frequency (30 mins)"
|
|
|
|
Input time segements:
|
|
|
|
| timestamp | device_id | battery_status | battery_level | battery_scale | battery_voltage | battery_temperature | battery_adaptor | battery_health | battery_technology |
|
|
|---|---|---|---|---|---|---|---|---|---|
|
|
| 11:59:28.434 | fre_ios | 2 | 63 | 100 | 4094 | 23 | 0 | 2 | Li-ion |
|
|
| 12:04:37.678 | fre_ios | 2 | 64 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
|
|
??? "4. A daily segment instance with a discharge episode that spans to the next daily instance"
|
|
|
|
??? "Periodic (daily)"
|
|
|
|
Input time segements:
|
|
|
|
| timestamp | device_id | battery_status | battery_level | battery_scale | battery_voltage | battery_temperature | battery_adaptor | battery_health | battery_technology |
|
|
|---|---|---|---|---|---|---|---|---|---|
|
|
| 05:59:49.434 | per_ios | 4 | 79 | 100 | 4094 | 23 | 0 | 2 | Li-ion |
|
|
| 06:02:19.321 | per_ios | 4 | 78 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
|
|
??? "Frequency (30 mins)"
|
|
|
|
Input time segements:
|
|
|
|
| timestamp | device_id | battery_status | battery_level | battery_scale | battery_voltage | battery_temperature | battery_adaptor | battery_health | battery_technology |
|
|
|---|---|---|---|---|---|---|---|---|---|
|
|
| 17:59:41.434 | fre_ios | 4 | 59 | 100 | 4094 | 23 | 0 | 2 | Li-ion |
|
|
| 18:04:14.321 | fre_ios | 4 | 58 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
|
|
??? "5. Three-day segments that repeat everyday"
|
|
|
|
[Time segment tested:](../setup/configuration.md#time-segments)
|
|
|
|
| label | start_time | length | repeats_on | repeats_value |
|
|
|---|---|---|---|---|
|
|
| daily | 00:00:00 | 23H 59M 59S | every_day | 0 |
|
|
|
|
|
|
Data tested:
|
|
|
|
We test 14 segments, one at the beginning of the first day, one at the end of the last day
|
|
|
|
| timestamp | device_id | battery_status | battery_level | battery_scale | battery_voltage | battery_temperature | battery_adaptor | battery_health | battery_technology |
|
|
|-----------------------|-----------|----------------|---------------|---------------|-----------------|---------------------|-----------------|----------------|--------------------|
|
|
| 2020-07-02 00:03:47.875 | per_and | 3 | 63 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-02 00:05:47.875 | per_and | 3 | 62 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-02 23:55:47.875 | per_and | 3 | 55 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-02 23:59:47.875 | per_and | 3 | 54 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-03 00:06:47.875 | per_and | 3 | 53 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-03 00:09:47.875 | per_and | 3 | 52 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-03 23:47:05.000 | per_and | 3 | 60 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-03 23:55:05.000 | per_and | 3 | 59 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-04 00:15:05.000 | per_and | 3 | 58 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-04 00:18:05.000 | per_and | 3 | 57 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-04 23:51:00.000 | per_and | 3 | 41 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-04 23:57:00.000 | per_and | 3 | 40 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-05 00:21:00.000 | per_and | 3 | 39 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-05 00:23:00.000 | per_and | 3 | 38 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
|
|
|
|
Output results:
|
|
|
|
| local_segment | local_segment_label | local_segment_start_datetime | local_segment_end_datetime | phone_battery_rapids_countdischarge | phone_battery_rapids_sumdurationdischarge | phone_battery_rapids_avgconsumptionrate | phone_battery_rapids_maxconsumptionrate | phone_battery_rapids_countcharge | phone_battery_rapids_sumdurationcharge |
|
|
|--------------------------------------------------|---------------------|------------------------------|----------------------------|-------------------------------------|-------------------------------------------|-----------------------------------------|-----------------------------------------|----------------------------------|----------------------------------------|
|
|
| threeday#2020-07-02 00:00:00,2020-07-04 23:59:59 | threeday | 2020-07-02 00:00:00 | 2020-07-04 23:59:59 | 4 | 149.7954 | 0.0710868450815781 | 0.111113168762384 | 0 | 0 |
|
|
| threeday#2020-07-03 00:00:00,2020-07-05 23:59:59 | threeday | 2020-07-03 00:00:00 | 2020-07-05 23:59:59 | 3 | 162.7952 | 0.0492745931499224 | 0.0502547286558745 | 0 | 0 |
|
|
| threeday#2020-07-04 00:00:00,2020-07-06 23:59:59 | threeday | 2020-07-04 00:00:00 | 2020-07-06 23:59:59 | 2 | 110.0815 | 0.0449915246814979 | 0.0483879032392475 | 0 | 0 |
|
|
| threeday#2020-07-05 00:00:00,2020-07-07 23:59:59 | threeday | 2020-07-05 00:00:00 | 2020-07-07 23:59:59 | 1 | 52.9991166666667 | 0.0377364779979038 | 0.0377364779979038 | 0 | 0 |
|
|
|
|
??? "6. A three-day segment that repeats on a fixed day"
|
|
|
|
[Time segment tested:](../setup/configuration.md#time-segments)
|
|
|
|
| label | start_time | length | repeats_on | repeats_value |
|
|
|---|---|---|---|---|
|
|
| weekends | 00:00:00 | 2D 23H 59M 59S | wday | 5 |
|
|
|
|
Data tested:
|
|
|
|
We test 10 segments, one at the beginning of the first day, one at the end of the last day
|
|
|
|
| timestamp | device_id | battery_status | battery_level | battery_scale | battery_voltage | battery_temperature | battery_adaptor | battery_health | battery_technology |
|
|
|-----------------------|-----------|----------------|---------------|---------------|-----------------|---------------------|-----------------|----------------|--------------------|
|
|
| 2020-07-03 00:06:47.875 | per_and | 3 | 53 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-03 00:09:47.875 | per_and | 3 | 52 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-03 23:47:05.000 | per_and | 3 | 60 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-03 23:55:05.000 | per_and | 3 | 59 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-04 00:15:05.000 | per_and | 3 | 58 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-04 00:18:05.000 | per_and | 3 | 57 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-04 23:51:00.000 | per_and | 3 | 41 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-04 23:57:00.000 | per_and | 3 | 40 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-05 00:21:00.000 | per_and | 3 | 39 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-07-05 00:23:00.000 | per_and | 3 | 38 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
|
|
Output results:
|
|
|
|
| local_segment | local_segment_label | local_segment_start_datetime | local_segment_end_datetime | phone_battery_rapids_countdischarge | phone_battery_rapids_sumdurationdischarge | phone_battery_rapids_avgconsumptionrate | phone_battery_rapids_maxconsumptionrate | phone_battery_rapids_countcharge | phone_battery_rapids_sumdurationcharge |
|
|
|--------------------------------------------------|---------------------|------------------------------|----------------------------|-------------------------------------|-------------------------------------------|-----------------------------------------|-----------------------------------------|----------------------------------|----------------------------------------|
|
|
| weekends#2020-07-03 00:00:00,2020-07-05 23:59:59 | weekends | 2020-07-03 00:00:00 | 2020-07-05 23:59:59 | 3 | 162.7952 | 0.0492745931499224 | 0.0502547286558745 | 0 | 0 |
|
|
|
|
??? "7. Event segements"
|
|
|
|
[Time segments tested:](../setup/configuration.md#time-segments)
|
|
|
|
| label | event_timestamp | length | shift | shift_direction | device_id |
|
|
|---|---|---|---|---|---|
|
|
| survey1 | 1587661220000 | 10H | 10H | -1 | a748ee1a-1d0b-4ae9-9074-279a2b6ba524 |
|
|
| survey2 | 1587661220000 | 10H | 5H | -1 | a748ee1a-1d0b-4ae9-9074-279a2b6ba524 |
|
|
| survey3 | 1587661220000 | 10H | 0H | 1 | a748ee1a-1d0b-4ae9-9074-279a2b6ba524 |
|
|
|
|
Data tested:
|
|
|
|
We test 7 segments, one at the beginning of the first day, one at the end of the last day
|
|
|
|
| timestamp | device_id | battery_status | battery_level | battery_scale | battery_voltage | battery_temperature | battery_adaptor | battery_health | battery_technology |
|
|
|-----------------------|--------------------------------------|----------------|---------------|---------------|-----------------|---------------------|-----------------|----------------|--------------------|
|
|
| 2020-04-23 03:15:00.000 | a748ee1a-1d0b-4ae9-9074-279a2b6ba524 | 3 | 90 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-04-23 03:21:00.000 | a748ee1a-1d0b-4ae9-9074-279a2b6ba524 | 3 | 89 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-04-23 07:50:00.000 | a748ee1a-1d0b-4ae9-9074-279a2b6ba524 | 3 | 80 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-04-23 08:05:00.000 | a748ee1a-1d0b-4ae9-9074-279a2b6ba524 | 3 | 79 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-04-23 08:12:00.000 | a748ee1a-1d0b-4ae9-9074-279a2b6ba524 | 3 | 78 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-04-23 22:50:00.000 | a748ee1a-1d0b-4ae9-9074-279a2b6ba524 | 3 | 50 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
| 2020-04-23 22:53:00.000 | a748ee1a-1d0b-4ae9-9074-279a2b6ba524 | 3 | 49 | 100 | 4157 | 23 | 0 | 2 | Li-ion |
|
|
|
|
Output results:
|
|
|
|
| local_segment | local_segment_label | local_segment_start_datetime | local_segment_end_datetime | phone_battery_rapids_sumdurationcharge | phone_battery_rapids_countdischarge | phone_battery_rapids_sumdurationdischarge | phone_battery_rapids_maxconsumptionrate | phone_battery_rapids_avgconsumptionrate | phone_battery_rapids_countcharge |
|
|
|-------------------------------------------------|---------------------|------------------------------|----------------------------|----------------------------------------|-------------------------------------|-------------------------------------------|-----------------------------------------|-----------------------------------------|----------------------------------|
|
|
| survey1#2020-04-23 03:00:20,2020-04-23 13:00:20 | survey1 | 2020-04-23 03:00:20 | 2020-04-23 13:00:20 | 0 | 2 | 87.9985333333333 | 0.0384621794978634 | 0.0331202101231602 | 0 |
|
|
| survey2#2020-04-23 08:00:20,2020-04-23 18:00:20 | survey2 | 2020-04-23 08:00:20 | 2020-04-23 18:00:20 | 0 | 1 | 41.6659833333333 | 0.0480007872129103 | 0.0480007872129103 | 0 |
|
|
| survey3#2020-04-23 13:00:20,2020-04-23 23:00:20 | survey3 | 2020-04-23 13:00:20 | 2020-04-23 23:00:20 | 0 | 1 | 10.3498 | 0.0966202245454018 | 0.0966202245454018 | 0 |
|