Add bluetooth, calls, sms and wifi docs

pull/103/head
JulioV 2020-11-04 19:37:24 -05:00
parent 2aa44fc504
commit 432f11fbcb
10 changed files with 1250 additions and 4 deletions

View File

@ -65,8 +65,8 @@ PHONE_VALID_SENSED_BINS:
PHONE_VALID_SENSED_DAYS:
COMPUTE: False
MIN_VALID_HOURS_PER_DAY: &min_valid_hours_per_day [16] # (out of 24) MIN_HOURS_PER_DAY
MIN_VALID_BINS_PER_HOUR: &min_valid_bins_per_hour [6] # (out of 60min/BIN_SIZE bins)
MIN_VALID_HOURS_PER_DAY: &min_valid_hours_per_day [16] # (out of 24) MIN_HOURS_PER_DAY
# Communication SMS features config, TYPES and FEATURES keys need to match
PHONE_MESSAGES:

View File

@ -0,0 +1,21 @@
# Behavioral Features Introduction
Every phone or Fitbit sensor has a corresponding config section e.g. `[PHONE_CALLS]` in `config.yaml` and 0, 1 or more **providers**. A provider is a script created by the core RAPIDS team or other researchers that extracts behavioral features for that sensor.
If you want to extract features from any sensor, set the corresponding `[PROVIDER][COMPUTE]` variable to `TRUE`, the `[TABLE]` variable to the sensor's table name in your database, and change any other parameters as [desired](https://www.rapids.science/setup/configuration/#sensor-and-features-to-process), and [execute](https://www.rapids.science/setup/execution/) RAPIDS:
!!! example
In this example the `config.yaml` file has been configured to extract `PHONE_CALLS` features from a table called `calls`:
```yaml
PHONE_CALLS:
TABLE: calls
PROVIDERS:
RAPIDS:
COMPUTE: True
...
```
!!! hint
Every time you change any sensor parameters, all the necessary files will be updated as soon as you execute RAPIDS. Some sensors will have specific attributes (like `MESSAGES_TYPES`) so refer to each sensor documentation.
Click on the left-hand side menu for the full feature catalogue per sensor.

View File

@ -0,0 +1,35 @@
# Phone Bluetooth
## RAPIDS provider
!!! info "Available day segments and platforms"
- Available for all day segments
- Available for Android only
!!! info "File Sequence"
```bash
- data/raw/{pid}/phone_bluetooth_raw.csv
- data/raw/{pid}/phone_bluetooth_with_datetime.csv
- data/interim/{pid}/phone_bluetooth_features/phone_bluetooth_{language}_{provider_key}.csv
- data/processed/features/{pid}/phone_bluetooth.csv"
```
Parameters description for `[PHONE_BLUETOOTH][PROVIDERS][RAPIDS]`:
|Key                              | Description |
|----------------|-----------------------------------------------------------------------------------------------------------------------------------
|`[COMPUTE]`| Set to `True` to extract `PHONE_BLUETOOTH` features from the `RAPIDS` provider|
|`[FEATURES]` | Features to be computed, see table below
Features description for `[PHONE_BLUETOOTH][PROVIDERS][RAPIDS]`:
|Feature |Units |Description|
|-------------------------- |---------- |---------------------------|
| countscans | devices | Number of scanned devices during a `day_segment`, a device can be detected multiple times over time and these appearances are counted separately |
| uniquedevices | devices | Number of unique devices during a `day_segment` as identified by their hardware (`bt_address`) address |
| countscansmostuniquedevice | scans | Number of scans of the most scanned device during a `day_segment` across the whole monitoring period |
!!! note "Assumptions/Observations"
NA

View File

@ -0,0 +1,58 @@
# Phone Calls
## RAPIDS Provider
!!! info "Available day segments and platforms"
- Available for all day segments
- Available for Android and iOS
!!! info "File Sequence"
```bash
- data/raw/{pid}/phone_calls_raw.csv
- data/raw/{pid}/phone_calls_with_datetime.csv
- data/raw/{pid}/phone_calls_with_datetime_unified.csv
- data/interim/{pid}/phone_calls_features/phone_calls_{language}_{provider_key}.csv
- data/processed/features/{pid}/phone_calls.csv
```
Parameters description for `[PHONE_CALLS][PROVIDERS][RAPIDS]`:
| Key                        | Description |
|-------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|`[COMPUTE]`| Set to `True` to extract `PHONE_CALLS` features from the `RAPIDS` provider|
| `[CALL_TYPES]` | The particular call_type that will be analyzed. The options for this parameter are incoming, outgoing or missed. |
| `[FEATURES]` | Features to be computed for `outgoing`, `incoming`, and `missed` calls. Note that the same features are available for both incoming and outgoing calls, while missed calls has its own set of features. See the tables below. |
Features description for `[PHONE_CALLS][PROVIDERS][RAPIDS]` incoming and outgoing calls:
|Feature |Units |Description|
|-------------------------- |---------- |---------------------------|
|count |calls |Number of calls of a particular `call_type` occurred during a particular `day_segment`.
|distinctcontacts |contacts |Number of distinct contacts that are associated with a particular `call_type` for a particular `day_segment`
|meanduration |seconds |The mean duration of all calls of a particular `call_type` during a particular `day_segment`.
|sumduration |seconds |The sum of the duration of all calls of a particular `call_type` during a particular `day_segment`.
|minduration |seconds |The duration of the shortest call of a particular `call_type` during a particular `day_segment`.
|maxduration |seconds |The duration of the longest call of a particular `call_type` during a particular `day_segment`.
|stdduration |seconds |The standard deviation of the duration of all the calls of a particular `call_type` during a particular `day_segment`.
|modeduration |seconds |The mode of the duration of all the calls of a particular `call_type` during a particular `day_segment`.
|entropyduration |nats |The estimate of the Shannon entropy for the the duration of all the calls of a particular `call_type` during a particular `day_segment`.
|timefirstcall |minutes |The time in minutes between 12:00am (midnight) and the first call of `call_type`.
|timelastcall |minutes |The time in minutes between 12:00am (midnight) and the last call of `call_type`.
|countmostfrequentcontact |calls |The number of calls of a particular `call_type` during a particular `day_segment` of the most frequent contact throughout the monitored period.
Features description for `[PHONE_CALLS][PROVIDERS][RAPIDS]` missed calls:
|Feature |Units |Description|
|-------------------------- |---------- |---------------------------|
|count |calls |Number of `missed` calls that occurred during a particular `day_segment`.
|distinctcontacts |contacts |Number of distinct contacts that are associated with `missed` calls for a particular `day_segment`
|timefirstcall |minutes |The time in hours from 12:00am (Midnight) that the first `missed` call occurred.
|timelastcall |minutes |The time in hours from 12:00am (Midnight) that the last `missed` call occurred.
|countmostfrequentcontact |calls |The number of `missed` calls during a particular `day_segment` of the most frequent contact throughout the monitored period.
!!! note "Assumptions/Observations"
1. Traces for iOS calls are unique even for the same contact calling a participant more than once which renders `countmostfrequentcontact` meaningless and `distinctcontacts` equal to the total number of traces.
2. `[CALL_TYPES]` and `[FEATURES]` keys in `config.yaml` need to match. For example, `[CALL_TYPES]` `outgoing` matches the `[FEATURES]` key `outgoing`
3. iOS calls data is transformed to match Android calls data format. See our [algorithm](algorithms/phone-algorithms.md#phone-calls)

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,40 @@
# Phone Messages
## RAPIDS provider
!!! info "Available day segments and platforms"
- Available for all day segments
- Available for Android only
!!! info "File Sequence"
```bash
- data/raw/{pid}/phone_messages_raw.csv
- data/raw/{pid}/phone_messages_with_datetime.csv
- data/interim/{pid}/phone_messages_features/phone_messages_{language}_{provider_key}.csv
- data/processed/features/{pid}/phone_messages.csv
```
Parameters description for `[PHONE_MESSAGES][PROVIDERS][RAPIDS]`:
|Key                              | Description |
|----------------|-----------------------------------------------------------------------------------------------------------------------------------
|`[COMPUTE]`| Set to `True` to extract `PHONE_MESSAGES` features from the `RAPIDS` provider|
|`[MESSAGES_TYPES]` | The `messages_type` that will be analyzed. The options for this parameter are `received` or `sent`.
|`[FEATURES]` | Features to be computed, see table below for `[MESSAGES_TYPES]` `received` and `sent`
Features description for `[PHONE_MESSAGES][PROVIDERS][RAPIDS]`:
|Feature |Units |Description|
|-------------------------- |---------- |---------------------------|
|count |messages |Number of messages of type `messages_type` that occurred during a particular `day_segment`.
|distinctcontacts |contacts |Number of distinct contacts that are associated with a particular `messages_type` during a particular `day_segment`.
|timefirstmessages |minutes |Number of minutes between 12:00am (midnight) and the first `message` of a particular `messages_type` during a particular `day_segment`.
|timelastmessages |minutes |Number of minutes between 12:00am (midnight) and the last `message` of a particular `messages_type` during a particular `day_segment`.
|countmostfrequentcontact |messages |Number of messages from the contact with the most messages of `messages_type` during a `day_segment` throughout the whole dataset of each participant.
!!! note "Assumptions/Observations"
1. `[MESSAGES_TYPES]` and `[FEATURES]` keys in `config.yaml` need to match. For example, `[MESSAGES_TYPES]` `sent` matches the `[FEATURES]` key `sent`

View File

@ -0,0 +1,36 @@
# Phone WiFi Connected
## RAPIDS provider
!!! info "Available day segments and platforms"
- Available for all day segments
- Available for Android and iOS
!!! info "File Sequence"
```bash
- data/raw/{pid}/phone_wifi_connected_raw.csv
- data/raw/{pid}/phone_wifi_connected_with_datetime.csv
- data/interim/{pid}/phone_wifi_connected_features/phone_wifi_connected_{language}_{provider_key}.csv
- data/processed/features/{pid}/phone_wifi_connected.csv
```
Parameters description for `[PHONE_WIFI_CONNECTED][PROVIDERS][RAPIDS]`:
|Key                              | Description |
|----------------|-----------------------------------------------------------------------------------------------------------------------------------
|`[COMPUTE]`| Set to `True` to extract `PHONE_WIFI_CONNECTED` features from the `RAPIDS` provider|
|`[FEATURES]` | Features to be computed, see table below
Features description for `[PHONE_WIFI_CONNECTED][PROVIDERS][RAPIDS]`:
|Feature |Units |Description|
|-------------------------- |---------- |---------------------------|
| countscans | devices | Number of scanned WiFi access points connected during a day_segment, an access point can be detected multiple times over time and these appearances are counted separately |
| uniquedevices | devices | Number of unique access point during a day_segment as identified by their hardware address |
| countscansmostuniquedevice | scans | Number of scans of the most scanned access point during a day_segment across the whole monitoring period |
!!! note "Assumptions/Observations"
1. A connected WiFI access point is one that a phone was connected to.
2. By default AWARE stores this data in the `sensor_wifi` table.

View File

@ -0,0 +1,36 @@
# Phone WiFi Visible
## RAPIDS provider
!!! info "Available day segments and platforms"
- Available for all day segments
- Available for Android only
!!! info "File Sequence"
```bash
- data/raw/{pid}/phone_wifi_visible_raw.csv
- data/raw/{pid}/phone_wifi_visible_with_datetime.csv
- data/interim/{pid}/phone_wifi_visible_features/phone_wifi_visible_{language}_{provider_key}.csv
- data/processed/features/{pid}/phone_wifi_visible.csv
```
Parameters description for `[PHONE_WIFI_VISIBLE][PROVIDERS][RAPIDS]`:
|Key                              | Description |
|----------------|-----------------------------------------------------------------------------------------------------------------------------------
|`[COMPUTE]`| Set to `True` to extract `PHONE_WIFI_VISIBLE` features from the `RAPIDS` provider|
|`[FEATURES]` | Features to be computed, see table below
Features description for `[PHONE_WIFI_VISIBLE][PROVIDERS][RAPIDS]`:
|Feature |Units |Description|
|-------------------------- |---------- |---------------------------|
| countscans | devices | Number of scanned WiFi access points visible during a day_segment, an access point can be detected multiple times over time and these appearances are counted separately |
| uniquedevices | devices | Number of unique access point during a day_segment as identified by their hardware address |
| countscansmostuniquedevice | scans | Number of scans of the most scanned access point during a day_segment across the whole monitoring period |
!!! note "Assumptions/Observations"
1. A visible WiFI access point is one that a phone sensed around itself but that it was not connected to. Due to API restrictions, this sensor is not available on iOS.
2. By default AWARE stores this data in the `wifi` table.

View File

@ -371,7 +371,7 @@ DEVICE_DATA:
Finally, you need to modify the `config.yaml` of the sensors you want to process. All sensors follow the same naming nomenclature `DEVICE_SENSOR` and have the following basic attributes (we will use `PHONE_MESSAGES` as an example).
!!! hint
Every time you change any sensor parameter, all the necessary files will be updated as soon as you execute RAPIDS. Some sensors will have specific attributes (like `MESSAGES_TYPES`) so refer to each sensor documentation.
Every time you change any sensor parameters, all the necessary files will be updated as soon as you execute RAPIDS. Some sensors will have specific attributes (like `MESSAGES_TYPES`) so refer to each sensor documentation.
```yaml
PHONE_MESSAGES:

View File

@ -43,8 +43,10 @@ theme:
primary: blue
icon:
logo: material/air-filter
features:
- navigation.sections
pages:
nav:
- Home: 'index.md'
- File Structure: file-structure.md
- Setup:
@ -53,4 +55,13 @@ pages:
- Execution: setup/execution.md
- Example Workflows:
- Minimal: workflow-examples/minimal.md
- Behavioral Features:
- Introduction: features/feature-introduction.md
- Phone:
- Phone Data Quality: features/phone-data-quality.md
- Phone Messages: features/phone-messages.md
- Phone Calls: features/phone-calls.md
- Phone Bluetooth: features/phone-bluetooth.md
- Phone WiFI Visible: features/phone-wifi-visible.md
- Phone WiFI Connected: features/phone-wifi-connected.md
- Frequently Asked Questions: faq.md