junos
/
rapids
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Deployed 73090e4 to dev with MkDocs 1.1.2 and mike 1.0.0

gh-pages
github-actions 2021-04-27 19:26:54 +00:00
parent 0498ec3ee9
commit 7f8bc0dc8a
18 changed files with 433 additions and 299 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
dev/change-log/index.html
View File

@ -1656,6 +1656,13 @@
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#v120" class="md-nav__link">
v1.2.0
</a>
</li>
<li class="md-nav__item">
<a href="#v111" class="md-nav__link">
v1.1.1
@ -1828,6 +1835,13 @@
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#v120" class="md-nav__link">
v1.2.0
</a>
</li>
<li class="md-nav__item">
<a href="#v111" class="md-nav__link">
v1.1.1
@ -1979,6 +1993,12 @@
<h1 id="change-log">Change Log<a class="headerlink" href="#change-log" title="Permanent link">&para;</a></h1>
<h2 id="v120">v1.2.0<a class="headerlink" href="#v120" title="Permanent link">&para;</a></h2>
<ul>
<li>Sleep summary and intraday features are more consistent.</li>
<li>Add wake and bedtime features for sleep summary data.</li>
<li>Fix bugs with sleep PRICE features.</li>
</ul>
<h2 id="v111">v1.1.1<a class="headerlink" href="#v111" title="Permanent link">&para;</a></h2>
<ul>
<li>Fix length of periodic segments on days with DLS</li>

19
dev/datastreams/fitbitjson-csv/index.html
View File

@ -1822,7 +1822,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_heartrate_summary_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_heartrate_summary_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note<p>All columns except <code>DEVICE_ID</code> are parsed from <code>JSON_FITBIT_COLUMN</code>. <code>JSON_FITBIT_COLUMN</code> is a string column containing the JSON objects returned by Fitbit&rsquo;s API. See an example of the raw data RAPIDS expects for this data stream:</p>
@ -1902,7 +1903,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_heartrate_intraday_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_heartrate_intraday_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note<p>All columns except <code>DEVICE_ID</code> are parsed from <code>JSON_FITBIT_COLUMN</code>. <code>JSON_FITBIT_COLUMN</code> is a string column containing the JSON objects returned by Fitbit&rsquo;s API. See an example of the raw data RAPIDS expects for this data stream:</p>
@ -2018,7 +2020,9 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_sleep_summary_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_sleep_summary_json.py
- src/data/streams/mutations/fitbit/add_local_date_time.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
@ -2115,7 +2119,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_sleep_intraday_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_sleep_intraday_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
@ -2196,7 +2201,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_steps_summary_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_steps_summary_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note<p><code>TIMESTAMP</code>, <code>LOCAL_DATE_TIME</code>, and <code>STEPS</code> are parsed from <code>JSON_FITBIT_COLUMN</code>. <code>JSON_FITBIT_COLUMN</code> is a string column containing the JSON objects returned by Fitbit&rsquo;s API. See an example of the raw data RAPIDS expects for this data stream:</p>
@ -2276,7 +2282,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_steps_intraday_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_steps_intraday_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note<p><code>TIMESTAMP</code>, <code>LOCAL_DATE_TIME</code>, and <code>STEPS</code> are parsed from <code>JSON_FITBIT_COLUMN</code>. <code>JSON_FITBIT_COLUMN</code> is a string column containing the JSON objects returned by <a href="https://dev.fitbit.com/build/reference/web-api/activity/#get-activity-intraday-time-series">Fitbit&rsquo;s API</a>. See an example of the raw data RAPIDS expects for this data stream:</p>

19
dev/datastreams/fitbitjson-mysql/index.html
View File

@ -1814,7 +1814,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_heartrate_summary_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_heartrate_summary_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note<p>All columns except <code>DEVICE_ID</code> are parsed from <code>JSON_FITBIT_COLUMN</code>. <code>JSON_FITBIT_COLUMN</code> is a string column containing the JSON objects returned by Fitbit&rsquo;s API. See an example of the raw data RAPIDS expects for this data stream:</p>
@ -1894,7 +1895,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_heartrate_intraday_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_heartrate_intraday_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note<p>All columns except <code>DEVICE_ID</code> are parsed from <code>JSON_FITBIT_COLUMN</code>. <code>JSON_FITBIT_COLUMN</code> is a string column containing the JSON objects returned by Fitbit&rsquo;s API. See an example of the raw data RAPIDS expects for this data stream:</p>
@ -2010,7 +2012,9 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_sleep_summary_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_sleep_summary_json.py
- src/data/streams/mutations/fitbit/add_local_date_time.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
@ -2107,7 +2111,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_sleep_intraday_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_sleep_intraday_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
@ -2188,7 +2193,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_steps_summary_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_steps_summary_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note<p><code>TIMESTAMP</code>, <code>LOCAL_DATE_TIME</code>, and <code>STEPS</code> are parsed from <code>JSON_FITBIT_COLUMN</code>. <code>JSON_FITBIT_COLUMN</code> is a string column containing the JSON objects returned by Fitbit&rsquo;s API. See an example of the raw data RAPIDS expects for this data stream:</p>
@ -2268,7 +2274,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_steps_intraday_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_steps_intraday_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note<p><code>TIMESTAMP</code>, <code>LOCAL_DATE_TIME</code>, and <code>STEPS</code> are parsed from <code>JSON_FITBIT_COLUMN</code>. <code>JSON_FITBIT_COLUMN</code> is a string column containing the JSON objects returned by <a href="https://dev.fitbit.com/build/reference/web-api/activity/#get-activity-intraday-time-series">Fitbit&rsquo;s API</a>. See an example of the raw data RAPIDS expects for this data stream:</p>

5
dev/datastreams/fitbitparsed-csv/index.html
View File

@ -1965,7 +1965,7 @@
</tr>
<tr>
<td>LOCAL_DATE_TIME</td>
<td>local_date_time</td>
<td>FLAG_TO_MUTATE</td>
</tr>
<tr>
<td>LOCAL_START_DATE_TIME</td>
@ -2020,7 +2020,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong> </p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/add_zero_timestamp.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/add_local_date_time.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
</li>
</ul>

5
dev/datastreams/fitbitparsed-mysql/index.html
View File

@ -1954,7 +1954,7 @@
</tr>
<tr>
<td>LOCAL_DATE_TIME</td>
<td>local_date_time</td>
<td>FLAG_TO_MUTATE</td>
</tr>
<tr>
<td>LOCAL_START_DATE_TIME</td>
@ -2009,7 +2009,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong> </p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/add_zero_timestamp.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/add_local_date_time.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
</li>
</ul>

2
dev/datastreams/mandatory-fitbit-format/index.html
View File

@ -1774,7 +1774,7 @@
</tr>
<tr>
<td>LOCAL_DATE_TIME</td>
<td>Date time string with format <code>yyyy-mm-dd hh:mm:ss</code>, this either is a copy of LOCAL_START_DATE_TIME or LOCAL_END_DATE_TIME depending on which column is used to assign an episode to a specific day</td>
<td>Date time string with format <code>yyyy-mm-dd 00:00:00</code>, the date is the same as the start date of a daily sleep episode if its time is after SLEEP_SUMMARY_LAST_NIGHT_END, otherwise it is the day before the start date of that sleep episode</td>
</tr>
<tr>
<td>LOCAL_START_DATE_TIME</td>

181
dev/developers/test-cases/index.html
View File

@ -1611,6 +1611,20 @@
Fitbit Calories Intraday
</a>
</li>
<li class="md-nav__item">
<a href="#fitbit-sleep-summary" class="md-nav__link">
Fitbit Sleep Summary
</a>
</li>
<li class="md-nav__item">
<a href="#fitbit-sleep-intraday" class="md-nav__link">
Fitbit Sleep Intraday
</a>
</li>
</ul>
@ -1847,6 +1861,20 @@
Fitbit Calories Intraday
</a>
</li>
<li class="md-nav__item">
<a href="#fitbit-sleep-summary" class="md-nav__link">
Fitbit Sleep Summary
</a>
</li>
<li class="md-nav__item">
<a href="#fitbit-sleep-intraday" class="md-nav__link">
Fitbit Sleep Intraday
</a>
</li>
</ul>
@ -2000,6 +2028,13 @@
<td>N</td>
</tr>
<tr>
<td>Fitbit Calories Intraday</td>
<td>RAPIDS</td>
<td>Y</td>
<td>Y</td>
<td>Y</td>
</tr>
<tr>
<td>Fitbit Data Yield</td>
<td>RAPIDS</td>
<td>N</td>
@ -2028,6 +2063,20 @@
<td>N</td>
</tr>
<tr>
<td>Fitbit Sleep Intraday</td>
<td>RAPIDS</td>
<td>Y</td>
<td>Y</td>
<td>Y</td>
</tr>
<tr>
<td>Fitbit Sleep Intraday</td>
<td>PRICE</td>
<td>Y</td>
<td>Y</td>
<td>Y</td>
</tr>
<tr>
<td>Fitbit Steps Summary</td>
<td>RAPIDS</td>
<td>N</td>
@ -2293,6 +2342,138 @@
</tr>
</tbody>
</table>
<h2 id="fitbit-sleep-summary">Fitbit Sleep Summary<a class="headerlink" href="#fitbit-sleep-summary" title="Permanent link">&para;</a></h2>
<p>Description</p>
<ul>
<li>A main sleep episode that starts on Fri 20:00:00 and ends on Sat 02:00:00. This episode starts after 11am (Last Night End) which will be considered as today&rsquo;s (Fri) data.</li>
<li>A nap that starts on Sat 04:00:00 and ends on Sat 06:00:00. This episode starts before 11am (Last Night End) which will be considered as yesterday&rsquo;s (Fri) data.</li>
<li>A nap that starts on Sat 13:00:00 and ends on Sat 15:00:00. This episode starts after 11am (Last Night End) which will be considered as today&rsquo;s (Sat) data.</li>
<li>A main sleep that starts on Sun 01:00:00 and ends on Sun 12:00:00. This episode starts before 11am (Last Night End) which will be considered as yesterday&rsquo;s (Sat) data.</li>
<li>A main sleep that starts on Sun 23:00:00 and ends on Mon 07:00:00. This episode starts after 11am (Last Night End) which will be considered as today&rsquo;s (Sun) data.</li>
<li>Any segment shorter than one day will be ignored for sleep RAPIDS features.</li>
</ul>
<p>Checklist</p>
<table>
<thead>
<tr>
<th>time segment</th>
<th>single tz</th>
<th>multi tz</th>
<th>platform</th>
</tr>
</thead>
<tbody>
<tr>
<td>30min</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
<tr>
<td>morning</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
<tr>
<td>daily</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
<tr>
<td>threeday</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
<tr>
<td>weekend</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
<tr>
<td>beforeMarchEvent</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
<tr>
<td>beforeNovemberEvent</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
</tbody>
</table>
<h2 id="fitbit-sleep-intraday">Fitbit Sleep Intraday<a class="headerlink" href="#fitbit-sleep-intraday" title="Permanent link">&para;</a></h2>
<p>Description</p>
<ul>
<li>A five-minute main sleep episode with asleep-classic level on Fri 11:00:00.</li>
<li>An eight-hour main sleep episode on Fri 17:00:00. It is split into 2 parts for daily segment: a seven-hour sleep episode on Fri 17:00:00 and an one-hour sleep episode on Sat 00:00:00.</li>
<li>A two-hour nap on Sat 01:00:00 that will be ignored for main sleep features.</li>
<li>An one-hour nap on Sat 13:00:00 that will be ignored for main sleep features.</li>
<li>An eight-hour main sleep episode on Sat 22:00:00. This episode ends on Sun 08:00:00 (NY) for March and Sun 06:00:00 (NY) for Novembers due to daylight savings. It will be considered for <code>beforeMarchEvent</code> segment and ignored for <code>beforeNovemberEvent</code> segment.</li>
<li>A nine-hour main sleep episode on Sun 11:00:00. Start time will be assigned as NY time zone and converted to 14:00:00.</li>
<li>A seven-hour main sleep episode on Mon 06:00:00. This episode will be split into two parts: a five-hour sleep episode on Mon 06:00:00 and a two-hour sleep episode on Mon 11:00:00. The first part will be discarded as it is before 11am (Last Night End)</li>
<li>Any segment shorter than one day will be ignored for sleep PRICE features.</li>
</ul>
<p>Checklist</p>
<table>
<thead>
<tr>
<th>time segment</th>
<th>single tz</th>
<th>multi tz</th>
<th>platform</th>
</tr>
</thead>
<tbody>
<tr>
<td>30min</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
<tr>
<td>morning</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
<tr>
<td>daily</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
<tr>
<td>threeday</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
<tr>
<td>weekend</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
<tr>
<td>beforeMarchEvent</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
<tr>
<td>beforeNovemberEvent</td>
<td>OK</td>
<td>OK</td>
<td>fitbit</td>
</tr>
</tbody>
</table>

243
dev/features/fitbit-sleep-intraday/index.html
View File

@ -1759,6 +1759,10 @@
</tbody>
</table>
<h2 id="rapids-provider">RAPIDS provider<a class="headerlink" href="#rapids-provider" title="Permanent link">&para;</a></h2>
<div class="admonition hint">
<p class="admonition-title">Understanding RAPIDS features</p>
<p><a href="../../img/sleep_intraday_rapids.png">This diagram</a> will help you understand how sleep episodes are chunked and grouped within time segments for the RAPIDS provider.</p>
</div>
<div class="admonition info">
<p class="admonition-title">Available time segments</p>
<ul>
@ -1795,19 +1799,11 @@
</tr>
<tr>
<td><code>[SLEEP_LEVELS]</code></td>
<td>Fitbit’s sleep API Version 1 only provides <code>CLASSIC</code> records. However, Version 1.2 provides 2 types of records: <code>CLASSIC</code> and <code>STAGES</code>. <code>STAGES</code> is only available in devices with a heart rate sensor and even those devices will fail to report it if the battery is low or the device is not tight enough. While <code>CLASSIC</code> contains 3 sleep levels (<code>awake</code>, <code>restless</code>, and <code>asleep</code>), <code>STAGES</code> contains 4 sleep levels (<code>wake</code>, <code>deep</code>, <code>light</code>, <code>rem</code>). To make it consistent, RAPIDS grouped them into 2 <code>UNIFIED</code> sleep levels: <code>awake</code> (<code>CLASSIC</code>: <code>awake</code> and <code>restless</code>; <code>STAGES</code>: <code>wake</code>) and <code>asleep</code> (<code>CLASSIC</code>: <code>asleep</code>; <code>STAGES</code>: <code>deep</code>, <code>light</code>, and <code>rem</code>).</td>
<td>Fitbit’s sleep API Version 1 only provides <code>CLASSIC</code> records. However, Version 1.2 provides 2 types of records: <code>CLASSIC</code> and <code>STAGES</code>. <code>STAGES</code> is only available in devices with a heart rate sensor and even those devices will fail to report it if the battery is low or the device is not tight enough. While <code>CLASSIC</code> contains 3 sleep levels (<code>awake</code>, <code>restless</code>, and <code>asleep</code>), <code>STAGES</code> contains 4 sleep levels (<code>wake</code>, <code>deep</code>, <code>light</code>, <code>rem</code>). To make it consistent, RAPIDS groups them into 2 <code>UNIFIED</code> sleep levels: <code>awake</code> (<code>CLASSIC</code>: <code>awake</code> and <code>restless</code>; <code>STAGES</code>: <code>wake</code>) and <code>asleep</code> (<code>CLASSIC</code>: <code>asleep</code>; <code>STAGES</code>: <code>deep</code>, <code>light</code>, and <code>rem</code>). In this section, there is a boolean flag named <code>INCLUDE_ALL_GROUPS</code> that if set to TRUE, computes LEVELS_AND_TYPES features grouping all levels together in a single <code>all</code> category.</td>
</tr>
<tr>
<td><code>[SLEEP_TYPES]</code></td>
<td>Types of sleep to be included in the feature extraction computation. Fitbit provides 2 types of sleep: <code>main</code>, <code>nap</code>.</td>
</tr>
<tr>
<td><code>[INCLUDE_SLEEP_LATER_THAN]</code></td>
<td>All resampled sleep rows (bin interval: one minute) that started after this time will be included in the feature computation. It is a number ranging from 0 (midnight) to 1439 (23:59) which denotes the number of minutes after midnight. If a segment is longer than one day, this value is for every day.</td>
</tr>
<tr>
<td><code>[REFERENCE_TIME]</code></td>
<td>The reference point from which the <code>[ROUTINE]</code> features are to be computed. Chosen from <code>MIDNIGHT</code> and <code>START_OF_THE_SEGMENT</code>, default is <code>MIDNIGHT</code>. If you have multiple time segments per day it might be more informative to set this flag to <code>START_OF_THE_SEGMENT</code>.</td>
<td>Types of sleep to be included in the feature extraction computation. There are three sleep types: <code>main</code>, <code>nap</code>, and <code>all</code>. The <code>all</code> type means both main sleep and naps are considered.</td>
</tr>
</tbody>
</table>
@ -1824,37 +1820,37 @@
<tr>
<td>countepisode<code>[LEVEL][TYPE]</code></td>
<td>episodes</td>
<td>Number of <code>[LEVEL][TYPE]</code>sleep episodes. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). Both <code>[LEVEL]</code>and <code>[TYPE]</code> can also be <code>all</code> when <code>LEVELS_AND_TYPES_COMBINING_ALL</code> is True, which ignores the levels and groups by sleep types.</td>
<td>Number of <code>[LEVEL][TYPE]</code>sleep episodes. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). <code>[LEVEL]</code> can also be <code>all</code> when <code>INCLUDE_ALL_GROUPS</code> is True, which ignores the levels and groups by sleep types.</td>
</tr>
<tr>
<td>sumduration<code>[LEVEL][TYPE]</code></td>
<td>minutes</td>
<td>Total duration of all <code>[LEVEL][TYPE]</code>sleep episodes. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). Both <code>[LEVEL]</code> and <code>[TYPE]</code>can also be <code>all</code> when <code>LEVELS_AND_TYPES_COMBINING_ALL</code> is True, which ignores the levels and groups by sleep types.</td>
<td>Total duration of all <code>[LEVEL][TYPE]</code>sleep episodes. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). <code>[LEVEL]</code> can also be <code>all</code> when <code>INCLUDE_ALL_GROUPS</code> is True, which ignores the levels and groups by sleep types.</td>
</tr>
<tr>
<td>maxduration<code>[LEVEL][TYPE]</code></td>
<td>minutes</td>
<td>Longest duration of any <code>[LEVEL][TYPE]</code>sleep episode. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). Both <code>[LEVEL]</code> and <code>[TYPE]</code>can also be <code>all</code> when <code>LEVELS_AND_TYPES_COMBINING_ALL</code> is True, which ignores the levels and groups by sleep types.</td>
<td>Longest duration of any <code>[LEVEL][TYPE]</code>sleep episode. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). <code>[LEVEL]</code> can also be <code>all</code> when <code>INCLUDE_ALL_GROUPS</code> is True, which ignores the levels and groups by sleep types.</td>
</tr>
<tr>
<td>minduration<code>[LEVEL][TYPE]</code></td>
<td>minutes</td>
<td>Shortest duration of any <code>[LEVEL][TYPE]</code>sleep episode. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). Both <code>[LEVEL]</code> and <code>[TYPE]</code>can also be <code>all</code> when <code>LEVELS_AND_TYPES_COMBINING_ALL</code> is True, which ignores the levels and groups by sleep types.</td>
<td>Shortest duration of any <code>[LEVEL][TYPE]</code>sleep episode. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). <code>[LEVEL]</code> can also be <code>all</code> when <code>INCLUDE_ALL_GROUPS</code> is True, which ignores the levels and groups by sleep types.</td>
</tr>
<tr>
<td>avgduration<code>[LEVEL][TYPE]</code></td>
<td>minutes</td>
<td>Average duration of all <code>[LEVEL][TYPE]</code>sleep episodes. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). Both <code>[LEVEL]</code> and <code>[TYPE]</code>can also be <code>all</code> when <code>LEVELS_AND_TYPES_COMBINING_ALL</code> is True, which ignores the levels and groups by sleep types.</td>
<td>Average duration of all <code>[LEVEL][TYPE]</code>sleep episodes. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). <code>[LEVEL]</code> can also be <code>all</code> when <code>INCLUDE_ALL_GROUPS</code> is True, which ignores the levels and groups by sleep types.</td>
</tr>
<tr>
<td>medianduration<code>[LEVEL][TYPE]</code></td>
<td>minutes</td>
<td>Median duration of all <code>[LEVEL][TYPE]</code>sleep episodes. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). Both <code>[LEVEL]</code> and <code>[TYPE]</code>can also be <code>all</code> when <code>LEVELS_AND_TYPES_COMBINING_ALL</code> is True, which ignores the levels and groups by sleep types.</td>
<td>Median duration of all <code>[LEVEL][TYPE]</code>sleep episodes. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). <code>[LEVEL]</code> can also be <code>all</code> when <code>INCLUDE_ALL_GROUPS</code> is True, which ignores the levels and groups by sleep types.</td>
</tr>
<tr>
<td>stdduration<code>[LEVEL][TYPE]</code></td>
<td>minutes</td>
<td>Standard deviation duration of all <code>[LEVEL][TYPE]</code>sleep episodes. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). Both <code>[LEVEL]</code> and <code>[TYPE]</code>can also be <code>all</code> when <code>LEVELS_AND_TYPES_COMBINING_ALL</code> is True, which ignores the levels and groups by sleep types.</td>
<td>Standard deviation duration of all <code>[LEVEL][TYPE]</code>sleep episodes. <code>[LEVEL]</code>is one of <code>[SLEEP_LEVELS]</code> (e.g. awake-classic or rem-stages) and <code>[TYPE]</code> is one of <code>[SLEEP_TYPES]</code> (e.g. main). <code>[LEVEL]</code> can also be <code>all</code> when <code>INCLUDE_ALL_GROUPS</code> is True, which ignores the levels and groups by sleep types.</td>
</tr>
</tbody>
</table>
@ -1913,12 +1909,12 @@
</thead>
<tbody>
<tr>
<td>ratiocount<code>[TYPE]</code>within<code>[LEVEL]</code></td>
<td>ratiocountmainwithin<code>[LEVEL]</code></td>
<td>-</td>
<td>Ratio between the <strong>count</strong> of episodes of a single sleep <code>[LEVEL]</code> during <code>main</code> sleep divided by the <strong>count</strong> of episodes of a single sleep <code>[LEVEL]</code> during <code>main</code> <strong>and</strong> <code>nap</code>. This answers the question: are <code>rem</code> episodes more frequent during <code>main</code> than <code>nap</code> sleep? We do not provide the ratio for <code>nap</code> because is complementary. (<span class="arithmatex">\(countepisode[remstages][main] / countepisode[remstages][all]\)</span>)</td>
</tr>
<tr>
<td>ratioduration<code>[TYPE]</code>within<code>[LEVEL]</code></td>
<td>ratiodurationmainwithin<code>[LEVEL]</code></td>
<td>-</td>
<td>Ratio between the <strong>duration</strong> of episodes of a single sleep <code>[LEVEL]</code> during <code>main</code> sleep divided by the <strong>duration</strong> of episodes of a single sleep <code>[LEVEL]</code> during <code>main</code> <strong>and</strong> <code>nap</code>. This answers the question: is <code>rem</code> time more frequent during <code>main</code> than <code>nap</code> sleep? We do not provide the ratio for <code>nap</code> because is complementary. (<span class="arithmatex">\(countepisode[remstages][main] / countepisode[remstages][all]\)</span>)</td>
</tr>
@ -1946,46 +1942,21 @@
</tr>
</tbody>
</table>
<p>Features description for <code>[FITBIT_SLEEP_INTRADAY][PROVIDERS][RAPIDS][ROUTINE]</code>:</p>
<table>
<thead>
<tr>
<th>Feature</th>
<th>Units</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>starttimefirstmainsleep</td>
<td>minutes</td>
<td>Start time (in minutes since <code>REFERENCE_TIME</code>) of the first main sleep episode after <code>INCLUDE_EPISODES_LATER_THAN</code>.</td>
</tr>
<tr>
<td>endtimelastmainsleep</td>
<td>minutes</td>
<td>End time (in minutes since <code>REFERENCE_TIME</code>) of the last main sleep episode after <code>INCLUDE_EPISODES_LATER_THAN</code>.</td>
</tr>
<tr>
<td>starttimefirstnap</td>
<td>minutes</td>
<td>Start time (in minutes since <code>REFERENCE_TIME</code>) of the first nap episode after <code>INCLUDE_EPISODES_LATER_THAN</code>.</td>
</tr>
<tr>
<td>endtimelastnap</td>
<td>minutes</td>
<td>End time (in minutes since <code>REFERENCE_TIME</code>) of the last nap episode after <code>INCLUDE_EPISODES_LATER_THAN</code>.</td>
</tr>
</tbody>
</table>
<div class="admonition note">
<p class="admonition-title">Assumptions/Observations</p>
<ol>
<li>Deleting values from <code>[SLEEP_LEVELS]</code> or <code>[SLEEP_TYPES]</code> will only change the features you receive from <code>[LEVELS_AND_TYPES]</code>. For example if <code>STAGES</code> only contains <code>[rem, light]</code> you will not receive <code>countepisode[wake|deep][TYPE]</code> or sum, max, min, avg, median, or std <code>duration</code>. These values will not influence <code>RATIOS</code> or <code>ROUTINE</code> features.</li>
<li>Any <code>[LEVEL]</code> grouping is done within the elements of each class <code>CLASSIC</code>, <code>STAGES</code>, and <code>UNIFIED</code>. That is, we never combine <code>CLASSIC</code> or <code>STAGES</code> types to compute features when <code>LEVELS_AND_TYPES_COMBINING_ALL</code> is True or when computing <code>RATIOS</code>.</li>
<li><a href="../../img/sleep_intraday_rapids.png">This diagram</a> will help you understand how sleep episodes are chunked and grouped within time segments for the RAPIDS provider.</li>
<li>Features listed in <code>[LEVELS_AND_TYPES]</code> are computed for any levels and types listed in <code>[SLEEP_LEVELS]</code> or <code>[SLEEP_TYPES]</code>. For example if <code>STAGES</code> only contains <code>[rem, light]</code> you will not get <code>countepisode[wake|deep][TYPE]</code> or sum, max, min, avg, median, or std <code>duration</code>. Levels or types in these lists do not influence <code>RATIOS</code> or <code>ROUTINE</code> features.</li>
<li>Any <code>[LEVEL]</code> grouping is done within the elements of each class <code>CLASSIC</code>, <code>STAGES</code>, and <code>UNIFIED</code>. That is, we never combine <code>CLASSIC</code> or <code>STAGES</code> types to compute features.</li>
<li>The categories for <code>all</code> levels (when <code>INCLUDE_ALL_GROUPS</code> is <code>True</code>) and <code>all</code> <code>SLEEP_TYPES</code> are not considered for <code>RATIOS</code> features as they are always 1.</li>
<li>These features can be computed in time segments of any length, but only the 1-minute sleep chunks within each segment instance will be used.</li>
</ol>
</div>
<h2 id="price-provider">PRICE provider<a class="headerlink" href="#price-provider" title="Permanent link">&para;</a></h2>
<div class="admonition hint">
<p class="admonition-title">Understanding PRICE features</p>
<p><a href="../../img/sleep_intraday_price.png">This diagram</a> will help you understand how sleep episodes are chunked and grouped within time segments and <code>LNE-LNE</code> intervals for the PRICE provider.</p>
</div>
<div class="admonition info">
<p class="admonition-title">Available time segments</p>
<ul>
@ -2021,15 +1992,15 @@
</tr>
<tr>
<td><code>[SLEEP_LEVELS]</code></td>
<td>Fitbit’s sleep API Version 1 only provides <code>CLASSIC</code> records. However, Version 1.2 provides 2 types of records: <code>CLASSIC</code> and <code>STAGES</code>. <code>STAGES</code> is only available in devices with a heart rate sensor and even those devices will fail to report it if the battery is low or the device is not tight enough. While <code>CLASSIC</code> contains 3 sleep levels (<code>awake</code>, <code>restless</code>, and <code>asleep</code>), <code>STAGES</code> contains 4 sleep levels (<code>wake</code>, <code>deep</code>, <code>light</code>, <code>rem</code>). To make it consistent, RAPIDS grouped them into 2 <code>UNIFIED</code> sleep levels: <code>awake</code> (<code>CLASSIC</code>: <code>awake</code> and <code>restless</code>; <code>STAGES</code>: <code>wake</code>) and <code>asleep</code> (<code>CLASSIC</code>: <code>asleep</code>; <code>STAGES</code>: <code>deep</code>, <code>light</code>, and <code>rem</code>).</td>
<td>Fitbit’s sleep API Version 1 only provides <code>CLASSIC</code> records. However, Version 1.2 provides 2 types of records: <code>CLASSIC</code> and <code>STAGES</code>. <code>STAGES</code> is only available in devices with a heart rate sensor and even those devices will fail to report it if the battery is low or the device is not tight enough. While <code>CLASSIC</code> contains 3 sleep levels (<code>awake</code>, <code>restless</code>, and <code>asleep</code>), <code>STAGES</code> contains 4 sleep levels (<code>wake</code>, <code>deep</code>, <code>light</code>, <code>rem</code>). To make it consistent, RAPIDS groups them into 2 <code>UNIFIED</code> sleep levels: <code>awake</code> (<code>CLASSIC</code>: <code>awake</code> and <code>restless</code>; <code>STAGES</code>: <code>wake</code>) and <code>asleep</code> (<code>CLASSIC</code>: <code>asleep</code>; <code>STAGES</code>: <code>deep</code>, <code>light</code>, and <code>rem</code>). In this section, there is a boolean flag named <code>INCLUDE_ALL_GROUPS</code> that if set to TRUE, computes avgdurationallmain<code>[DAY_TYPE]</code> features grouping all levels together in a single <code>all</code> category.</td>
</tr>
<tr>
<td><code>[DAY_TYPE]</code></td>
<td>The features of this provider can be computed using daily averages/standard deviations that were extracted on <code>WEEKEND</code> days only, <code>WEEK</code> days only, or <code>ALL</code> days</td>
</tr>
<tr>
<td><code>[GROUP_EPISODES_WITHIN]</code></td>
<td>This parameter contains 2 values: <code>[START_TIME]</code> and <code>[LENGTH]</code>. Only <code>main</code> sleep episodes that intersect or contain the period between [<code>START_TIME</code>, <code>START_TIME</code> + <code>LENGTH</code>] are taken into account to compute the features described below. Both <code>[START_TIME]</code> and <code>[LENGTH]</code> are in minutes. <code>[START_TIME]</code> is a number ranging from 0 (midnight) to 1439 (23:59) which denotes the number of minutes after midnight. <code>[LENGTH]</code> is a number smaller than 1440 (24 hours).</td>
<td><code>[LAST_NIGHT_END]</code></td>
<td>Only <code>main</code> sleep episodes that start within the <code>LNE-LNE</code> interval [<code>LAST_NIGHT_END</code>, <code>LAST_NIGHT_END</code> + 23H 59M 59S] are taken into account to compute the features described below. <code>[LAST_NIGHT_END]</code> is a number ranging from 0 (midnight) to 1439 (23:59).</td>
</tr>
</tbody>
</table>
@ -2046,182 +2017,92 @@
<tr>
<td>avgduration<code>[LEVEL]</code>main<code>[DAY_TYPE]</code></td>
<td>minutes</td>
<td>Average duration of daily <code>LEVEL</code> sleep episodes. You can include daily average that were computed on weekend days, week days or both depending on the value of the <code>DAY_TYPE</code> flag.</td>
<td>Average duration of daily sleep chunks of a <code>LEVEL</code>. Use the <code>DAY_TYPE</code> flag to include daily durations from weekend days only, weekdays, or both. Use <code>[LEVEL]</code> to group all levels in a single <code>all</code> category.</td>
</tr>
<tr>
<td>avgratioduration<code>[LEVEL]</code>withinmain<code>[DAY_TYPE]</code></td>
<td>-</td>
<td>Average ratio between daily <code>LEVEL</code> time and in-bed time inferred from <code>main</code> sleep episodes. <code>LEVEL</code> is one of <code>SLEEP_LEVELS</code> (e.g. awake-classic or rem-stages). In-bed time is the total duration of all <code>main</code> sleep episodes for each day. You can include daily ratios that were computed on weekend days, week days or both depending on the value of the <code>DAY_TYPE</code> flag.</td>
<td>Average of the daily ratio between the duration of sleep chunks of a <code>LEVEL</code> and total duration of all <code>main</code> sleep episodes in a day. When <code>INCLUDE_ALL_GROUPS</code> is <code>True</code> the <code>all</code> <code>LEVEL</code> is ignored since this feature is always 1. Use the <code>DAY_TYPE</code> flag to include start times from weekend days only, weekdays, or both.</td>
</tr>
<tr>
<td>avgstarttimeofepisodemain<code>[DAY_TYPE]</code></td>
<td>minutes</td>
<td>Average start time of the first <code>main</code> sleep episode of each day in a time segment. You can include daily start times from episodes detected on weekend days, week days or both depending on the value of the <code>DAY_TYPE</code> flag.</td>
<td>Average of all start times of the first <code>main</code> sleep episode within each <code>LNE-LNE</code> interval in a time segment. Use the <code>DAY_TYPE</code> flag to include start times from <code>LNE-LNE</code> intervals that start on weekend days only, weekdays, or both.</td>
</tr>
<tr>
<td>avgendtimeofepisodemain<code>[DAY_TYPE]</code></td>
<td>minutes</td>
<td>Average end time of the last <code>main</code> sleep episode of each day in a time segment. You can include daily end times from episodes detected on weekend days, week days or both depending on the value of the <code>DAY_TYPE</code> flag.</td>
<td>Average of all end times of the last <code>main</code> sleep episode within each <code>LNE-LNE</code> interval in a time segment. Use the <code>DAY_TYPE</code> flag to include end times from <code>LNE-LNE</code> intervals that start on weekend days only, weekdays, or both.</td>
</tr>
<tr>
<td>avgmidpointofepisodemain<code>[DAY_TYPE]</code></td>
<td>minutes</td>
<td>Average mid time between the start of the first <code>main</code> sleep episode and the end of the last <code>main</code> sleep episode of each day in a time segment. You can include episodes detected on weekend days, week days or both depending on the value of the <code>DAY_TYPE</code> flag.</td>
<td>Average of all the differences between <code>avgendtime...</code> and <code>avgstarttime..</code> in a time segment. Use the <code>DAY_TYPE</code> flag to include end times from <code>LNE-LNE</code> intervals that start on weekend days only, weekdays, or both.</td>
</tr>
<tr>
<td>stdstarttimeofepisodemain<code>[DAY_TYPE]</code></td>
<td>minutes</td>
<td>Standard deviation of start time of the first <code>main</code> sleep episode of each day in a time segment. You can include daily start times from episodes detected on weekend days, week days or both depending on the value of the <code>DAY_TYPE</code> flag.</td>
<td>Standard deviation of all start times of the first <code>main</code> sleep episode within each <code>LNE-LNE</code> interval in a time segment. Use the <code>DAY_TYPE</code> flag to include start times from <code>LNE-LNE</code> intervals that start on weekend days only, weekdays, or both.</td>
</tr>
<tr>
<td>stdendtimeofepisodemain<code>[DAY_TYPE]</code></td>
<td>minutes</td>
<td>Standard deviation of end time of the last <code>main</code> sleep episode of each day in a time segment. You can include daily end times from episodes detected on weekend days, week days or both depending on the value of the <code>DAY_TYPE</code> flag.</td>
<td>Standard deviation of all end times of the last <code>main</code> sleep episode within each <code>LNE-LNE</code> interval in a time segment. Use the <code>DAY_TYPE</code> flag to include end times from <code>LNE-LNE</code> intervals that start on weekend days only, weekdays, or both.</td>
</tr>
<tr>
<td>stdmidpointofepisodemain<code>[DAY_TYPE]</code></td>
<td>minutes</td>
<td>Standard deviation of mid time between the start of the first <code>main</code> sleep episode and the end of the last <code>main</code> sleep episode of each day in a time segment. You can include episodes detected on weekend days, week days or both depending on the value of the <code>DAY_TYPE</code> flag.</td>
<td>Standard deviation of all the differences between <code>avgendtime...</code> and <code>avgstarttime..</code> in a time segment. Use the <code>DAY_TYPE</code> flag to include end times from <code>LNE-LNE</code> intervals that start on weekend days only, weekdays, or both.</td>
</tr>
<tr>
<td>socialjetlag</td>
<td>minutes</td>
<td>Difference in minutes between the avgmidpointofepisodemain (average mid time between bedtime and wake time) of weekends and weekdays.</td>
<td>Difference in minutes between the avgmidpointofepisodemain of weekends and weekdays that belong to each time segment instance. If your time segment does not contain at least one week day and one weekend day this feature will be NA.</td>
</tr>
<tr>
<td>meanssdstarttimeofepisodemain</td>
<td>minutes squared</td>
<td>Same as <code>avgstarttimeofepisodemain[DAY_TYPE]</code> but the average is computed over the squared differences of each pair of consecutive start times.</td>
<td>rmssdmeanstarttimeofepisodemain</td>
<td>minutes</td>
<td>Square root of the <strong>mean</strong> squared successive difference (RMSSD) between today&rsquo;s and yesterday&rsquo;s <code>starttimeofepisodemain</code> values across the entire participant&rsquo;s sleep data grouped per time segment instance. It represents the mean of how someone&rsquo;s <code>starttimeofepisodemain</code> (bedtime) changed from night to night.</td>
</tr>
<tr>
<td>meanssdendtimeofepisodemain</td>
<td>minutes squared</td>
<td>Same as <code>avgendtimeofepisodemain[DAY_TYPE]</code> but the average is computed over the squared differences of each pair of consecutive end times.</td>
<td>rmssdmeanendtimeofepisodemain</td>
<td>minutes</td>
<td>Square root of the <strong>mean</strong> squared successive difference (RMSSD) between today&rsquo;s and yesterday&rsquo;s <code>endtimeofepisodemain</code> values across the entire participant&rsquo;s sleep data grouped per time segment instance. It represents the mean of how someone&rsquo;s <code>endtimeofepisodemain</code> (wake time) changed from night to night.</td>
</tr>
<tr>
<td>meanssdmidpointofepisodemain</td>
<td>minutes squared</td>
<td>Same as <code>avgmidpointofepisodemain[DAY_TYPE]</code> but the average is computed over the squared differences of each pair of consecutive mid times.</td>
<td>rmssdmeanmidpointofepisodemain</td>
<td>minutes</td>
<td>Square root of the <strong>mean</strong> squared successive difference (RMSSD) between today&rsquo;s and yesterday&rsquo;s <code>midpointofepisodemain</code> values across the entire participant&rsquo;s sleep data grouped per time segment instance. It represents the mean of how someone&rsquo;s <code>midpointofepisodemain</code> (mid time between bedtime and wake time) changed from night to night.</td>
</tr>
<tr>
<td>medianssdstarttimeofepisodemain</td>
<td>minutes squared</td>
<td>Same as <code>avgstarttimeofepisodemain[DAY_TYPE]</code> but the median is computed over the squared differences of each pair of consecutive start times.</td>
<td>rmssdmedianstarttimeofepisodemain</td>
<td>minutes</td>
<td>Square root of the <strong>median</strong> squared successive difference (RMSSD) between today&rsquo;s and yesterday&rsquo;s <code>starttimeofepisodemain</code> values across the entire participant&rsquo;s sleep data grouped per time segment instance. It represents the median of how someone&rsquo;s <code>starttimeofepisodemain</code> (bedtime) changed from night to night.</td>
</tr>
<tr>
<td>medianssdendtimeofepisodemain</td>
<td>minutes squared</td>
<td>Same as <code>avgendtimeofepisodemain[DAY_TYPE]</code> but the median is computed over the squared differences of each pair of consecutive end times.</td>
<td>rmssdmedianendtimeofepisodemain</td>
<td>minutes</td>
<td>Square root of the <strong>median</strong> squared successive difference (RMSSD) between today&rsquo;s and yesterday&rsquo;s <code>endtimeofepisodemain</code> values across the entire participant&rsquo;s sleep data grouped per time segment instance. It represents the median of how someone&rsquo;s <code>endtimeofepisodemain</code> (wake time) changed from night to night.</td>
</tr>
<tr>
<td>medianssdmidpointofepisodemain</td>
<td>minutes squared</td>
<td>Same as <code>avgmidpointofepisodemain[DAY_TYPE]</code> but the median is computed over the squared differences of each pair of consecutive mid times.</td>
<td>rmssdmedianmidpointofepisodemain</td>
<td>minutes</td>
<td>Square root of the <strong>median</strong> squared successive difference (RMSSD) between today&rsquo;s and yesterday&rsquo;s <code>midpointofepisodemain</code> values across the entire participant&rsquo;s sleep data grouped per time segment instance. It represents the median of how someone&rsquo;s <code>midpointofepisodemain</code> (average mid time between bedtime and wake time) changed from night to night.</td>
</tr>
</tbody>
</table>
<div class="admonition note">
<p class="admonition-title">Assumptions/Observations</p>
<ol>
<li>These features are based on descriptive statistics computed across daily values (start/end/mid times of sleep episodes). This is the reason why they are only available on time segments that are longer than 24 hours (we need at least 1 day to get the average).</li>
<li><a href="../../img/sleep_intraday_price.png">This diagram</a> will help you understand how sleep episodes are chunked and grouped within time segments and <code>LNE-LNE</code> intervals for the PRICE provider.</li>
<li>We recommend you use periodic segments that start in the morning so RAPIDS can chunk and group sleep episodes overnight. Shifted segments (as any other segments) are labelled based on their start and end date times.</li>
<li><code>avgstarttime...</code> and <code>avgendtime...</code> are roughly equivalent to an average bed and awake time only if you are using shifted segments.</li>
<li>The features of this provider are only available on time segments that are longer than 24 hours because they are based on descriptive statistics computed across daily values.</li>
<li>Even though Fitbit provides 2 types of sleep episodes (<code>main</code> and <code>nap</code>), only <code>main</code> sleep episodes are considered.</li>
<li>
<p>How do we assign sleep episodes to specific dates?</p>
<p><code>START_TIME</code> and <code>LENGTH</code> control the dates that sleep episodes belong to. For a pair of <code>[START_TIME]</code> and <code>[LENGTH]</code>, sleep episodes (blue boxes) can only be placed at the following places:</p>
<p><figure>
<img src="../../img/features_fitbit_sleep_intraday.png" max-width="100%" />
<figcaption>Relationship between sleep episodes and the given times<code>([START_TIME], [LENGTH])</code></figcaption>
</figure></p>
<ul>
<li>
<p>If the end time of a sleep episode is before <code>[START_TIME]</code>, it will belong to the day before its start date (e.g. sleep episode #1).</p>
</li>
<li>
<p>if (1) the start time or the end time of a sleep episode are between (overlap) <code>[START_TIME]</code> and <code>[START_TIME] + [LENGTH]</code> or (2) the start time is before <code>[START_TIME]</code> and the end time is after <code>[START_TIME] + [LENGTH]</code>, it will belong to its start date (e.g. sleep episode #2, #3, #4, #5).</p>
</li>
<li>
<p>If the start time of a sleep episode is after <code>START_TIME] + [LENGTH]</code>, it will belong to the day after its start date (e.g. sleep episode #6).</p>
</li>
</ul>
<p>Only <code>main</code> sleep episodes that intersect or contain the period between <code>[START_TIME]</code> and <code>[START_TIME] + [LENGTH]</code> will be included in the feature computation. If we process the following <code>main</code> sleep episodes: </p>
<table>
<thead>
<tr>
<th>episode</th>
<th>start</th>
<th>end</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>2021-02-01 12:00</td>
<td>2021-02-01 15:00</td>
</tr>
<tr>
<td>2</td>
<td>2021-02-01 21:00</td>
<td>2021-02-02 03:00</td>
</tr>
<tr>
<td>3</td>
<td>2021-02-02 05:00</td>
<td>2021-02-02 08:00</td>
</tr>
<tr>
<td>4</td>
<td>2021-02-02 11:00</td>
<td>2021-02-02 14:00</td>
</tr>
<tr>
<td>5</td>
<td>2021-02-02 19:00</td>
<td>2021-02-03 06:00</td>
</tr>
</tbody>
</table>
<p>And our parameters:</p>
<ul>
<li>
<p><code>[INCLUDE_EPISODES_INTERSECTING][START_TIME]</code> = 1320 (today&rsquo;s 22:00)</p>
</li>
<li>
<p><code>[INCLUDE_EPISODES_INTERSECTING][LENGTH]</code> = 720 (tomorrow&rsquo;s 10:00, or 22:00 + 12 hours)</p>
</li>
</ul>
<p>Only sleep episodes 2, 3,and 5 would be considered.</p>
</li>
<li>
<p>Time related features represent the number of minutes between the start/end/midpoint of sleep episodes and the assigned day&rsquo;s midnight.</p>
</li>
<li>
<p>All <code>main</code> sleep episodes are chunked within the requested <a href="../../setup/configuration/#time-segments">time segments</a> which need to be at least 24 hours or more long (1, 2, 3, 7 days, etc.). Then, daily features will be extracted and averaged across the length of the time segment, for example:</p>
<p>The daily features extracted on 2021-02-01 will be:</p>
<ul>
<li>
<p>starttimeofepisodemain (bedtime) is <code>21 * 60</code> (episode 2 start time 2021-02-01 21:00)</p>
</li>
<li>
<p>endtimeofepisodemain (wake time) is <code>32 * 60</code>(episode 3 end time 2021-02-02 08:00 + 24)</p>
</li>
<li>
<p>midpointofepisodemain (midpoint sleep) is <code>[(21 * 60) + (32 * 60)] / 2</code></p>
</li>
</ul>
<p>The daily features extracted on 2021-02-02 will be:</p>
<ul>
<li>
<p>starttimeofepisodemain (bedtime) is <code>19 * 60</code> (episode 5 start time 2021-02-01 19:00)</p>
</li>
<li>
<p>endtimeofepisodemain (wake time) is <code>30 * 60</code>(episode 5 end time 2021-02-03 06:00 + 24)</p>
</li>
<li>
<p>midpointofepisodemain (midpoint sleep) is <code>[(19 * 60) + (30 * 60)] / 2</code></p>
</li>
</ul>
<p>And <code>avgstarttimeofepisodemain[DAY_TYPE]</code> will be <code>([21 * 60] + [19 * 60]) / 2</code> </p>
</li>
<li>The reference point for all times is 00:00 of the first day in the LNE-LNE interval.</li>
<li>Sleep episodes are formed by 1-minute chunks that we group overnight starting from today’s LNE and ending on tomorrow’s LNE or the end of that segment (whatever is first). </li>
<li>The features <code>avgstarttime...</code> and <code>avgendtime...</code> are the average of the first and last sleep episode across every LNE-LNE interval within a segment (<code>avgmidtime...</code> is the mid point between start and end). Therefore, only segments longer than 24hrs will be averaged across more than one LNE-LNE interval.</li>
<li><code>socialjetlag</code> is only available on segment instances equal or longer than 48hrs that contain at least one weekday day and one weekend day, for example seven-day (weekly) segments.</li>
</ol>
</div>

48
dev/features/fitbit-sleep-summary/index.html
View File

@ -1745,6 +1745,10 @@
</tbody>
</table>
<h2 id="rapids-provider">RAPIDS provider<a class="headerlink" href="#rapids-provider" title="Permanent link">&para;</a></h2>
<div class="admonition hint">
<p class="admonition-title">Understanding RAPIDS features</p>
<p><a href="../../img/sleep_summary_rapids.png">This diagram</a> will help you understand how sleep episodes are chunked and grouped within time segments using <code>SLEEP_SUMMARY_LAST_NIGHT_END</code> for the RAPIDS provider.</p>
</div>
<div class="admonition info">
<p class="admonition-title">Available time segments</p>
<ul>
@ -1774,12 +1778,16 @@
</tr>
<tr>
<td><code>[SLEEP_TYPES]</code></td>
<td>Types of sleep to be included in the feature extraction computation. Fitbit provides 3 types of sleep: <code>main</code>, <code>nap</code>, <code>all</code>.</td>
<td>Types of sleep to be included in the feature extraction computation. There are three sleep types: <code>main</code>, <code>nap</code>, and <code>all</code>. The <code>all</code> type means both main sleep and naps are considered.</td>
</tr>
<tr>
<td><code>[FEATURES]</code></td>
<td>Features to be computed from sleep summary data, see table below</td>
</tr>
<tr>
<td><code>[FITBIT_DATA_STREAMS][data stream][SLEEP_SUMMARY_LAST_NIGHT_END]</code></td>
<td>As an exception, the <code>LAST_NIGHT_END</code> parameter for this provider is in the data stream configuration section. This parameter controls how sleep episodes are assigned to different days and affects wake and bedtimes.</td>
</tr>
</tbody>
</table>
<p>Features description for <code>[FITBIT_SLEEP_SUMMARY][PROVIDERS][RAPIDS]</code>:</p>
@ -1793,6 +1801,26 @@
</thead>
<tbody>
<tr>
<td>firstwaketimeTYPE</td>
<td>minutes</td>
<td>First wake time for a certain sleep type during a time segment. Wake time is number of minutes after midnight of a sleep episode&rsquo;s end time.</td>
</tr>
<tr>
<td>lastwaketimeTYPE</td>
<td>minutes</td>
<td>Last wake time for a certain sleep type during a time segment. Wake time is number of minutes after midnight of a sleep episode&rsquo;s end time.</td>
</tr>
<tr>
<td>firstbedtimeTYPE</td>
<td>minutes</td>
<td>First bedtime for a certain sleep type during a time segment. Bedtime is number of minutes after midnight of a sleep episode&rsquo;s start time.</td>
</tr>
<tr>
<td>lastbedtimeTYPE</td>
<td>minutes</td>
<td>Last bedtime for a certain sleep type during a time segment. Bedtime is number of minutes after midnight of a sleep episode&rsquo;s start time.</td>
</tr>
<tr>
<td>countepisodeTYPE</td>
<td>episodes</td>
<td>Number of sleep episodes for a certain sleep type during a time segment.</td>
@ -1857,18 +1885,18 @@
<div class="admonition note">
<p class="admonition-title">Assumptions/Observations</p>
<ol>
<li>
<p>There are three sleep types (TYPE): <code>main</code>, <code>nap</code>, <code>all</code>. The <code>all</code> type contains both main sleep and naps.</p>
</li>
<li>
<p>There are two versions of Fitbit’s sleep API (<a href="https://dev.fitbit.com/build/reference/web-api/sleep-v1/">version 1</a> and <a href="https://dev.fitbit.com/build/reference/web-api/sleep/">version 1.2</a>), and each provides raw sleep data in a different format:</p>
<ul>
<li><a href="../../img/sleep_summary_rapids.png">This diagram</a> will help you understand how sleep episodes are chunked and grouped within time segments using <code>LNE</code> for the RAPIDS provider.</li>
<li>There are three sleep types (TYPE): <code>main</code>, <code>nap</code>, <code>all</code>. The <code>all</code> type groups both <code>main</code> sleep and <code>naps</code>. All types are based on Fitbit&rsquo;s labels.</li>
<li>There are two versions of Fitbit’s sleep API (<a href="https://dev.fitbit.com/build/reference/web-api/sleep-v1/">version 1</a> and <a href="https://dev.fitbit.com/build/reference/web-api/sleep/">version 1.2</a>), and each provides raw sleep data in a different format:<ul>
<li><em>Count &amp; duration summaries</em>. <code>v1</code> contains <code>count_awake</code>, <code>duration_awake</code>, <code>count_awakenings</code>, <code>count_restless</code>, and <code>duration_restless</code> fields for every sleep record but <code>v1.2</code> does not.</li>
</ul>
</li>
<li>
<p><em>API columns</em>. Features are computed based on the values provided by Fitbit’s API: <code>efficiency</code>, <code>minutes_after_wakeup</code>, <code>minutes_asleep</code>, <code>minutes_awake</code>, <code>minutes_to_fall_asleep</code>, <code>minutes_in_bed</code>, <code>is_main_sleep</code> and <code>type</code>.</p>
</li>
<li><em>API columns</em>. Most features are computed based on the values provided by Fitbit’s API: <code>efficiency</code>, <code>minutes_after_wakeup</code>, <code>minutes_asleep</code>, <code>minutes_awake</code>, <code>minutes_to_fall_asleep</code>, <code>minutes_in_bed</code>, <code>is_main_sleep</code> and <code>type</code>.</li>
<li>Bed time and sleep duration are based on episodes that started between today’s LNE and tomorrow’s LNE while awake time is based on the episodes that started between yesterday’s LNE and today’s LNE</li>
<li>The reference point for bed/awake times is today’s 00:00. You can have bedtimes larger than 24 and awake times smaller than 0</li>
<li>These features are only available for time segments that span midnight to midnight of the same or different day.</li>
<li>We include first and last wake and bedtimes because, when <code>LAST_NIGHT_END</code> is 10 am, the first bedtime could match a nap at 2 pm, and the last bedtime could match a main overnight sleep episode that starts at 10pm.</li>
<li>Set the value for <code>SLEEP_SUMMARY_LAST_NIGHT_END</code> int the config parameter [FITBIT_DATA_STREAMS][data stream][SLEEP_SUMMARY_LAST_NIGHT_END].</li>
</ol>
</div>

BIN
dev/img/sleep_intraday_price.png 100644
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 840 KiB

BIN
dev/img/sleep_intraday_rapids.png 100644
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 230 KiB

BIN
dev/img/sleep_summary_rapids.png 100644
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 294 KiB

24
dev/setup/configuration/index.html
View File

@ -2587,19 +2587,19 @@ fitbit999, Europe/Amsterdam, 0
<span class="c1"># AVAILABLE:</span>
<span class="nt">fitbitjson_mysql</span><span class="p">:</span>
<span class="nt">DATABASE_GROUP</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">MY_GROUP</span>
<span class="nt">SLEEP_SUMMARY_EPISODE_DAY_ANCHOR</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">False</span>
<span class="nt">SLEEP_SUMMARY_LAST_NIGHT_END</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">660</span>
<span class="nt">fitbitjson_csv</span><span class="p">:</span>
<span class="nt">FOLDER</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">data/external/fitbit_csv</span>
<span class="nt">SLEEP_SUMMARY_EPISODE_DAY_ANCHOR</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">False</span>
<span class="nt">SLEEP_SUMMARY_LAST_NIGHT_END</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">660</span>
<span class="nt">fitbitparsed_mysql</span><span class="p">:</span>
<span class="nt">DATABASE_GROUP</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">MY_GROUP</span>
<span class="nt">SLEEP_SUMMARY_EPISODE_DAY_ANCHOR</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">False</span>
<span class="nt">SLEEP_SUMMARY_LAST_NIGHT_END</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">660</span>
<span class="nt">fitbitparsed_csv</span><span class="p">:</span>
<span class="nt">FOLDER</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">data/external/fitbit_csv</span>
<span class="nt">SLEEP_SUMMARY_EPISODE_DAY_ANCHOR</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">False</span>
<span class="nt">SLEEP_SUMMARY_LAST_NIGHT_END</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">660</span>
</code></pre></div>
<div class="tabbed-set" data-tabs="7:4"><input checked="checked" id="__tabbed_7_1" name="__tabbed_7" type="radio" /><label for="__tabbed_7_1">fitbitjson_mysql</label><div class="tabbed-content">
<p>This data stream process Fitbit data inside a JSON column as obtained from the Fitbit API and stored in a MySQL database. Read more about its column mappings and mutations in <a href="../../datastreams/fitbitjson-mysql#format"><code>fitbitjson_mysql</code></a>.</p>
@ -2616,8 +2616,8 @@ fitbit999, Europe/Amsterdam, 0
<td>A database credentials group. Read the instructions below to set it up</td>
</tr>
<tr>
<td><code>[SLEEP_SUMMARY_EPISODE_DAY_ANCHOR]</code></td>
<td>One of <code>start</code> or <code>end</code>. Summary sleep episodes are considered as events based on either the start timestamp or end timestamp (they will belong to the day where they start or end).</td>
<td><code>[SLEEP_SUMMARY_LAST_NIGHT_END]</code></td>
<td>Segments are assigned based on this parameter. Any sleep episodes starts between today&rsquo;s SLEEP_SUMMARY_LAST_NIGHT_END (LNE) and tomorrow&rsquo;s LNE is regarded as today&rsquo;s sleep episode. While today&rsquo;s bedtime is based on today&rsquo;s sleep episodes, today&rsquo;s wake time is based on yesterday&rsquo;s sleep episodes.</td>
</tr>
</tbody>
</table>
@ -2700,8 +2700,8 @@ fitbit999, Europe/Amsterdam, 0
<td>Folder where you have to place a CSV file <strong>per</strong> Fitbit sensor. Each file has to contain all the data from every participant you want to process.</td>
</tr>
<tr>
<td><code>[SLEEP_SUMMARY_EPISODE_DAY_ANCHOR]</code></td>
<td>One of <code>start</code> or <code>end</code>. Summary sleep episodes are considered as events based on either the start timestamp or end timestamp (they will belong to the day where they start or end).</td>
<td><code>[SLEEP_SUMMARY_LAST_NIGHT_END]</code></td>
<td>Segments are assigned based on this parameter. Any sleep episodes starts between today&rsquo;s SLEEP_SUMMARY_LAST_NIGHT_END (LNE) and tomorrow&rsquo;s LNE is regarded as today&rsquo;s sleep episode. While today&rsquo;s bedtime is based on today&rsquo;s sleep episodes, today&rsquo;s wake time is based on yesterday&rsquo;s sleep episodes.</td>
</tr>
</tbody>
</table>
@ -2721,8 +2721,8 @@ fitbit999, Europe/Amsterdam, 0
<td>A database credentials group. Read the instructions below to set it up</td>
</tr>
<tr>
<td><code>[SLEEP_SUMMARY_EPISODE_DAY_ANCHOR]</code></td>
<td>One of <code>start</code> or <code>end</code>. Summary sleep episodes are considered as events based on either the start timestamp or end timestamp (they will belong to the day where they start or end).</td>
<td><code>[SLEEP_SUMMARY_LAST_NIGHT_END]</code></td>
<td>Segments are assigned based on this parameter. Any sleep episodes starts between today&rsquo;s SLEEP_SUMMARY_LAST_NIGHT_END (LNE) and tomorrow&rsquo;s LNE is regarded as today&rsquo;s sleep episode. While today&rsquo;s bedtime is based on today&rsquo;s sleep episodes, today&rsquo;s wake time is based on yesterday&rsquo;s sleep episodes.</td>
</tr>
</tbody>
</table>
@ -2805,8 +2805,8 @@ fitbit999, Europe/Amsterdam, 0
<td>Folder where you have to place a CSV file <strong>per</strong> Fitbit sensor. Each file has to contain all the data from every participant you want to process.</td>
</tr>
<tr>
<td><code>[SLEEP_SUMMARY_EPISODE_DAY_ANCHOR]</code></td>
<td>One of <code>start</code> or <code>end</code>. Summary sleep episodes are considered as events based on either the start timestamp or end timestamp (they will belong to the day where they start or end).</td>
<td><code>[SLEEP_SUMMARY_LAST_NIGHT_END]</code></td>
<td>Segments are assigned based on this parameter. Any sleep episodes starts between today&rsquo;s SLEEP_SUMMARY_LAST_NIGHT_END (LNE) and tomorrow&rsquo;s LNE is regarded as today&rsquo;s sleep episode. While today&rsquo;s bedtime is based on today&rsquo;s sleep episodes, today&rsquo;s wake time is based on yesterday&rsquo;s sleep episodes.</td>
</tr>
</tbody>
</table>

140
dev/sitemap.xml
View File

@ -1,283 +1,283 @@
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"><url>
<loc>https://www.rapids.science/dev/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/setup/overview/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/workflow-examples/minimal/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/setup/installation/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/setup/configuration/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/setup/execution/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/citation/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/data-streams-introduction/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/aware-mysql/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/aware-csv/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/aware-influxdb/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/mandatory-phone-format/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/fitbitjson-mysql/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/fitbitjson-csv/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/fitbitparsed-mysql/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/fitbitparsed-csv/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/mandatory-fitbit-format/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/empatica-zip/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/mandatory-empatica-format/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/datastreams/add-new-data-streams/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/feature-introduction/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-accelerometer/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-activity-recognition/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-applications-crashes/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-applications-foreground/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-applications-notifications/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-battery/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-bluetooth/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-calls/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-conversation/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-data-yield/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-keyboard/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-light/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-locations/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-log/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-messages/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-screen/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-wifi-connected/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/phone-wifi-visible/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/fitbit-calories-intraday/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/fitbit-data-yield/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/fitbit-heartrate-summary/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/fitbit-heartrate-intraday/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/fitbit-sleep-summary/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/fitbit-sleep-intraday/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/fitbit-steps-summary/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/fitbit-steps-intraday/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/empatica-accelerometer/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/empatica-heartrate/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/empatica-temperature/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/empatica-electrodermal-activity/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/empatica-blood-volume-pulse/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/empatica-inter-beat-interval/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/empatica-tags/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/features/add-new-features/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/visualizations/data-quality-visualizations/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/visualizations/feature-visualizations/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/workflow-examples/analysis/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/developers/git-flow/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/developers/remote-support/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/developers/virtual-environments/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/developers/documentation/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/developers/testing/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/developers/test-cases/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/developers/validation-schema-config/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/migrating-from-old-versions/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/code_of_conduct/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/common-errors/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/team/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url><url>
<loc>https://www.rapids.science/dev/change-log/</loc>
<lastmod>2021-04-22</lastmod>
<lastmod>2021-04-27</lastmod>
<changefreq>daily</changefreq>
</url>
</urlset>

BIN
dev/sitemap.xml.gz
View File

Binary file not shown.

19
dev/snippets/jsonfitbit_format/index.html
View File

@ -1716,7 +1716,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_heartrate_summary_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_heartrate_summary_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note<p>All columns except <code>DEVICE_ID</code> are parsed from <code>JSON_FITBIT_COLUMN</code>. <code>JSON_FITBIT_COLUMN</code> is a string column containing the JSON objects returned by Fitbit&rsquo;s API. See an example of the raw data RAPIDS expects for this data stream:</p>
@ -1796,7 +1797,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_heartrate_intraday_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_heartrate_intraday_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note<p>All columns except <code>DEVICE_ID</code> are parsed from <code>JSON_FITBIT_COLUMN</code>. <code>JSON_FITBIT_COLUMN</code> is a string column containing the JSON objects returned by Fitbit&rsquo;s API. See an example of the raw data RAPIDS expects for this data stream:</p>
@ -1912,7 +1914,9 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_sleep_summary_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_sleep_summary_json.py
- src/data/streams/mutations/fitbit/add_local_date_time.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
@ -2009,7 +2013,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_sleep_intraday_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_sleep_intraday_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
@ -2090,7 +2095,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_steps_summary_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_steps_summary_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note<p><code>TIMESTAMP</code>, <code>LOCAL_DATE_TIME</code>, and <code>STEPS</code> are parsed from <code>JSON_FITBIT_COLUMN</code>. <code>JSON_FITBIT_COLUMN</code> is a string column containing the JSON objects returned by Fitbit&rsquo;s API. See an example of the raw data RAPIDS expects for this data stream:</p>
@ -2170,7 +2176,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong></p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/parse_steps_intraday_json.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/parse_steps_intraday_json.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note<p><code>TIMESTAMP</code>, <code>LOCAL_DATE_TIME</code>, and <code>STEPS</code> are parsed from <code>JSON_FITBIT_COLUMN</code>. <code>JSON_FITBIT_COLUMN</code> is a string column containing the JSON objects returned by <a href="https://dev.fitbit.com/build/reference/web-api/activity/#get-activity-intraday-time-series">Fitbit&rsquo;s API</a>. See an example of the raw data RAPIDS expects for this data stream:</p>

5
dev/snippets/parsedfitbit_format/index.html
View File

@ -1852,7 +1852,7 @@
</tr>
<tr>
<td>LOCAL_DATE_TIME</td>
<td>local_date_time</td>
<td>FLAG_TO_MUTATE</td>
</tr>
<tr>
<td>LOCAL_START_DATE_TIME</td>
@ -1907,7 +1907,8 @@
</li>
<li>
<p><strong>SCRIPTS</strong> </p>
<div class="highlight"><pre><span></span><code>src/data/streams/mutations/fitbit/add_zero_timestamp.py
<div class="highlight"><pre><span></span><code>- src/data/streams/mutations/fitbit/add_local_date_time.py
- src/data/streams/mutations/fitbit/add_zero_timestamp.py
</code></pre></div>
</li>
</ul>

2
dev/workflow-examples/analysis/index.html
View File

@ -1837,7 +1837,7 @@
<details class="info"><summary>5. Data visualization.</summary><p>At this point the user can use the five plots RAPIDS provides (or implement new ones) to explore and understand the quality of the raw data and extracted features and decide what sensors, days, or participants to include and exclude. Refer to <code>rules/reports.smk</code> to find the rules that generate these plots.</p>
</details>
<details class="info"><summary>6. Feature cleaning.</summary><p>In this stage we perform four steps to clean our sensor feature file. First, we discard days with a data yield hour ratio less than or equal to 0.75, i.e. we include days with at least 18 hours of data. Second, we drop columns (features) with more than 30% of missing rows. Third, we drop columns with zero variance. Fourth, we drop rows (days) with more than 30% of missing columns (features). In this cleaning stage several parameters are created and exposed in <code>example_profile/example_config.yaml</code>. </p>
<p>After this step, we kept 161 features over 11 days for the individual model of p01, 101 features over 12 days for the individual model of p02 and 107 features over 20 days for the population model. Note that the difference in the number of features between p01 and p02 is mostly due to iOS restrictions that stops researchers from collecting the same number of sensors than in Android phones. </p>
<p>After this step, we kept 158 features over 11 days for the individual model of p01, 101 features over 12 days for the individual model of p02 and 106 features over 20 days for the population model. Note that the difference in the number of features between p01 and p02 is mostly due to iOS restrictions that stops researchers from collecting the same number of sensors than in Android phones. </p>
<p>Feature cleaning for the individual models is done in the <code>clean_sensor_features_for_individual_participants</code> rule and for the population model in the <code>clean_sensor_features_for_all_participants</code> rule in <code>rules/models.smk</code>.</p>
</details>
<details class="info"><summary>7. Merge features and targets.</summary><p>In this step we merge the cleaned features and target labels for our individual models in the <code>merge_features_and_targets_for_individual_model</code> rule in <code>rules/models.smk</code>. Additionally, we merge the cleaned features, target labels, and demographic features of our two participants for the population model in the <code>merge_features_and_targets_for_population_model</code> rule in <code>rules/models.smk</code>. These two merged files are the input for our individual and population models. </p>