Introduction
Physical inactivity, in particular sedentary behavior, is a growing concern in many countries (World Health Organization 2014). Over the last decade, the time that people daily spent doing moderate-to-vigorous physical activity has decreased, while the prevalence of obesity has increased (Guthold et al. 2018; Kohl et al. 2012; Lee et al. 2012). Insight in a populations’ physical activity (PA) levels is needed to inform governments and policy makers. Questionnaires, currently the predominant form of PA measurement, are known to suffer from measurement error due to recall bias, telescoping and social desirability (Adams et al. 2005; Ferrari, Friedenreich, and Matthews 2007; Fruin and Rankin 2004; Helmerhorst et al. 2012; Sallis and Saelens 2000; Shephard 2003; Welk et al. 2007; Wijndaele et al. 2015). These disadvantages can potentially be overcome by using activity trackers to measure PA (Kapteyn et al. 2018).
Activity trackers are designed to measure motion and derived features such as step count and active minutes (Ward et al. 2005). Trackers vary strongly in accuracy, usage and costs. To measure PA amongst the population objectively, one could distribute either commercial or research-grade activity trackers amongst willing respondents. Yet, this would require intensive logistics and comes with high costs (de Wolf et al. 2024).
An alternative approach would be to rely on personal activity trackers owned by respondents, especially since more and more people have a device that measures their PA. Respondents can be requested to extract the readily collected data from their personal devices and share it with researchers. This method of data collection is generally referred to as a form of ‘data donation’, which encompasses both the use of data that respondents (passively) collect themselves (e.g., through wearables) as well as data that is being collected from respondents by third parties (e.g., companies logging digital trace data from social media platforms). Data donation poses the opportunity for respondents to share data that was gathered over a past period, overcoming the challenges of a questionnaire while still building on retrospective data.
Prior research on data donation of PA data has investigated the (hypothetical) willingness to donate data on a small scale (Toepoel, Luiten, and Zandvliet 2021; Kraakman et al. 2023). Half of the respondents owned a personal tracker in these studies, and willingness to donate data differed greatly per task. Data for 1 day was copied by 86% of the respondents, but hypothetical willingness for copying data of 1 week was only 53%, and for uploading a spreadsheet only 38%. The small pilot described in Kraakman et al. (2023) showed that, dependent on the indicator, only very few respondents copied useful data over 1 week (ranging between 7 to 25 respondents).
In the current study, actual donation of PA data is examined at a larger scale with the aim of evaluating its feasibility to set-up a national surveillance system to measure PA with donated data from personal activity trackers. In this paper, we further extend the research area of data donation of PA data by exploring the promise of two different methods of data donation from personal activity trackers to measure PA: (1) uploading exported spreadsheets and (2) manually copying data into questionnaires. We study both the number and characteristics of respondents who are willing and able to provide data using each of these methods. In addition to measuring response rates and the representativity of willingness to donate, we compare the substantive outcomes of these methods with those from the Dutch PA questionnaire to determine the extent to which survey data and personal tracker data match. This comparison will provide valuable insights into how personal tracker data donations compare to self-reported data in terms of feasibility and data quality, informing the design of future national PA monitoring systems.
Method
The data used in this article were gathered within a larger research project about measurement of PA (see Appendix A for full description of this project). All data were gathered amongst participants of a Dutch probability-based panel (LISS panel; more information in Appendix B). The data collection procedures were approved by the Internal Review Board of Centerdata. Informed consent was obtained from all respondents digitally.
Study design
Three different methods of data donation (copy data of 1 day, copy data of 1 week and data upload) were offered to respondents at different stages throughout the study. Respondents could use multiple methods to donate data. Figure 1 shows the different stages and resulting data donation groups.
In October 2021, a pre questionnaire was fielded to 3874 panel members, of which 3370 responded. In addition to questions on PA, respondents were invited to participate in a PA study where they would wear a research-grade activity tracker (ActivPAL) for a week. Based on the indication of willingness to wear a research-grade device (and availability of the devices), 615 of the 3370 pre-questionnaire respondents were invited to the ActivPAL study, in which they were asked to wear an ActivPAL for 8 days; 503 respondents finally did so.
All respondents who wore the ActivPAL (n = 503) were invited for an evaluation questionnaire (in August 2022). In this questionnaire, 221 indicated that they had worn their personal activity tracker (no phone measuring PA) during the study’s fieldwork. Subsequently they were asked to manually copy PA indicators from their device into the survey for the same week they wore the ActivPAL (group Copy data of 1 week in Figure 1). Sixty-one of these 221 people with a personal tracker owned a Fitbit and were additionally asked to export the PA spreadsheet file from their device from the same week they wore the ActivPAL and donate this file by uploading it within the survey (group Data upload in Figure 1).
In July 2022, a post questionnaire was fielded to the same group of panel members as the pre questionnaire was fielded amongst. The sample was slightly reduced (n = 3628) due to panel members dropping out of the panel and response was also lower (n = 2929) than in the pre questionnaire. In this questionnaire, the same PA questions as in the pre questionnaire were posed, supplemented with the question to all personal activity tracker owners (n = 626) to donate the PA indicators of 1 day (group Copy data of 1 day in Figure 1).
Donating a spreadsheet versus manual copying
To ensure a successful data donation process, detailed instructions had to be developed for each of the personal activity tracker brands. Extensive manuals containing step-by-step screenshots were developed for the most popular brands[1] (Apple, Garmin, Samsung, and Fitbit) for manually copying data to the survey. However, during the development of the instructions for the spreadsheets, it became evident that for three of the four brands it would become too technically challenging for the majority of participants to successfully export these files. For Fitbit users, instructions were relatively straightforward, which led to the decision to focus only on Fitbit for exporting and donating the spreadsheet files (see Appendix C for screenshots of the instructions).
Measures
Questionnaire
The pre and post questionnaire consisted of the Dutch PA questionnaire: the Short QUestionnaire to ASsess Health-enhancing physical activity (SQUASH; Wendel-Vos and Schuit 2002), and several questions on sedentary behavior (National Institute for Public Health and the Environment (RIVM) 2020). Furthermore, respondents were asked about their willingness to donate data and to wear a research-grade device, followed by questions about privacy, technology and general health.
Donated data
For the groups manually copying the data into the questionnaire, PA indicators available for most brands were requested: number of steps, total distance travelled, number of active minutes, number of calories burned, average heart rate and time asleep (see Appendix C).
In the post questionnaire, all 626 respondents who indicated that they own a personal activity tracker were asked to copy the values of the available indicators, describing their activity on the prior day. Respondents could also indicate that they did not want to copy the data.
All respondents who wore an ActivPAL and a personal activity tracker (n = 221) were asked to copy the values of the indicators named above for the entire week that they wore the ActivPAL to allow comparison between the two devices. In the evaluation questionnaire, respondents were first asked about their willingness to donate data. If they were willing, they were subsequently forwarded to a screen to enter the requested values per day (see Appendix C).
Additionally, the 61 respondents who wore the ActivPAL and owned a Fitbit were asked in the evaluation questionnaire whether they were willing to upload a spreadsheet with all their data from the same week that they wore the ActivPAL. Willing and doubting respondents were provided with step-by-step screenshot instructions on how to download the data from their Fitbit account and upload the spreadsheet in the questionnaire (Appendix C). The spreadsheet of Fitbit contains data on weight, calories, sleep, steps and active minutes.
Adherence to PA guidelines
To meet the Dutch PA guidelines, respondents have to engage in at least 150 minutes of moderate to high intensity PA per week and do muscle strengthening activities at least twice a week (RIVM 2020). Therefore, this is the main target variable that is used in national surveillance systems to measure PA and thus in this article.
Here, we only report adherence to the first aspect as the trackers only assess PA intensity and cannot assess whether muscle strengthening activities were performed. Standardized methods developed by the Dutch National Institute for Public Health and the Environment were used to compute this parameter from the SQUASH (Wendel-Vos et al. 2020).
For each method of data donation, the number of active minutes per week was used to assess the adherence to the PA guidelines. If the number of active minutes was not provided by the respondent, no assessment of the adherence could be made. The number of active minutes per week was calculated by multiplying the active minutes (of 1 day) by seven for the Copy data of 1 day group. For the Copy data of 1 week group the number of active minutes reported for each day were summed. For the Data upload group, the ‘fairly active’ and ‘very active’ minutes were summed, as Fitbit distinguishes activity levels as ‘sedentary’, ‘slightly active’, ‘fairly active’, and ‘very active’. If these Fitbit respondents uploaded more or less than 1 week, the total number of active minutes was interpolated or extrapolated to 1 week.
Analysis
Using logistic regression models, we examined selectivity in ownership of personal activity trackers within pre questionnaire respondents and selectivity in donating data conditional on tracker ownership. In these models, adherence to the PA guidelines according to the SQUASH was included as well as several demographic characteristics.
Adherence to the PA guidelines in the group that copied data of 1 day could only be determined for 313 respondents (61.7%), as not all respondents who copied data of 1 day provided the number of active minutes. In addition, nine outliers were removed using the Interquartile Range (IQR) method of outlier detection, resulting in 304 (60.0%) respondents in this analysis. For the group that copied data of 1 week, 99 respondents provided the number of active minutes (63.9%) and seven outliers were excluded using the IQR method of outlier detection, resulting in 92 (59.4%) included respondents. For uploading Fitbit data, no outliers were identified due to the small sample size, yet three respondents uploaded an empty file and were thus removed, leading to 31 respondents (50.8%) in the analysis.
Based on these reduced datasets, the number of active minutes per week and day were calculated. Subsequently, adherence to PA guidelines according to the different data donation sources and the SQUASH was determined and compared.
Results
Response
In the post questionnaire, 507 respondents copied data of 1 day (Table 1). Out of the 221 respondents who wore an ActivPAL and a personal activity tracker, 151 respondents reported they were willing to copy 1 week of data, 18 respondents were in doubt and 52 respondents said ‘No’. From the willing and doubting respondents, 155 copied data of 1 week. However, only 38 respondents completed all requested activity indicators. This can (partly) be explained by the fact that not all activity trackers measure all requested activity indicators.
Of the 61 respondents who wore an ActivPAL and a Fitbit, 43 respondents said ‘Yes’ to donating a spreadsheet, two respondents were in doubt and 16 respondents refused to donate their PA spreadsheet. Of the willing and doubting respondents, 34 respondents uploaded a file.
Representativeness
Table 2 shows the results of the conditional logistic regression models used to test the difference in demographic characteristics between respondents who donated data and the respondents who could but did not copy/upload data (see Appendix D for the demographic composition of the groups). Younger and higher educated respondents were overrepresented amongst respondents owning a device, compared to respondents without a device. Moreover, respondents owning a personal activity tracker also are more likely to adhere to the PA guidelines. The additional bias for copying 1 day of data is limited to higher educated respondents being more likely to donate this type of data. Regarding copying 1 week of data (either by copying or by uploading) age is an important factor. Amongst respondents owning a personal tracker, the younger respondents are more likely to share 1 week worth of data. Remarkably, for uploading Fitbit data, respondents adhering to the guidelines according to the SQUASH are less likely to export and donate their spreadsheet. However, the small sample size of this group may have impacted this result.
Adherence to the PA guidelines
Respondents were ‘very active’ according to their own personal trackers. According to these devices, the majority of respondents adhered to the PA guidelines (i.e., 150 minutes of moderate to vigorous PA per week) as shown in Table 3. The median number of active minutes per day and week are also clearly above the threshold.
In Figure 2, the comparison of adherence according to the SQUASH versus adherence according to each data donation method is shown. This comparison is visualized separately for various groups of respondents based on the number of methods that a respondent used to donate data. Adherence according to the 1-day data is consistently closest to the adherence according to the SQUASH. Adherence according to the week of data and Fitbit are both higher and closer to one other.
The differences in demographic characteristics between respondents who donated data and those who could but did not copy/upload data in Table 2, along with the analysis of adherence data shown in Table 3 and the comparative analysis of data donation methods in Figure 2, suggest that data donations from personal activity trackers are likely to produce unrepresentative outcomes for adherence to PA guidelines, the main variable of interest. Furthermore, the low number of responses in most subgroups shown in Table 3 dramatically reduces the generalizability of the study’s findings.
Discussion
In this study, we compared three data donation methods to collect data from personal activity trackers to examine the feasibility of using donated data from these trackers to set-up a device-based PA surveillance system. Data donation could be the cost-efficient alternative for distributing commercial or research-grade activity trackers amongst willing respondents. To evaluate the feasibility of uploading exported spreadsheets and manually copying data into questionnaires, we examined differences in response, representativeness, and substantive outcomes.
From the different data donation methods offered in this study, copying data into the questionnaire was done more often than uploading a spreadsheet, presumably due to its relative simplicity. Overall, the willingness to donate data was high amongst the LISS panel respondents who owned a personal activity tracker. Contrary to expectations, these percentages were higher than the willingness percentages found in earlier studies conducted in a different panel (Toepoel, Luiten, and Zandvliet 2021; Kraakman et al. 2023), but it has to be noted that respondents from the LISS panel are regularly invited to participate in innovative data collection studies, making them potentially more willing to donate data. The generalizability of these results to a fresh (non-panel) sample of respondents is therefore questionable and lower willingness percentages may be foreseen in such a group.
Another challenge in data donation from personal activity trackers is the limited number and selectivity of respondents that own a personal activity tracker or smartwatch. Even though tracker ownership was similar to what was found in other studies (Toepoel, Luiten, and Zandvliet 2021; Kraakman et al. 2023), it is a subsample from the total population. Furthermore, respondents who owned a personal activity tracker were younger, higher educated and more active. Subsequently, an additional age bias occurred in the 1 week data donation (either by copying or uploading), which can possibly be explained by the difficulty of the task. Copying data of 1 day suffers from a slightly different bias pattern; in this case an additional bias occurred among the higher educated. This lack of representativity is a significant limitation that hampers the implementation of data donation from personal activity trackers to generate statistics representing the general population, upon which policies can be based.
Concerning the selective group of respondents, the percentage of people adhering to the PA guidelines calculated based on the donated data was higher than when using the SQUASH, the standard measurement instrument used in the Netherlands. These differences are likely an accumulation of the subsequent stages of the data collection and analysis process: measurement error of the device itself in measuring the required construct, data alterations while transferring the measured data from the tracker to the researchers, and the consequences of the processing steps needed to create the target variable of interest. These stages can be mapped on the errors described on the measurement side of the total survey error framework (TSEF; Groves and Lyberg 2010; validity, measurement error and processing error, respectively), just as the errors that are known to exist for the SQUASH (recall bias, social desirability, and data processing consequences). Quantifying and comparing the error at each step when using either method requires further research using a ground truth measurement method of PA. Since spreadsheet uploads proved challenging for respondents and data entry is prone to many of the same errors as self-report questionnaires, the case for using personal PA trackers over self-report questionnaires is weakened.
Whereas this study cannot conclude whether measurement error is larger for one or the other method, conclusions can be drawn for the representation side of the TSEF. In the statistic based on the donated data, more representation errors occur compared to the statistic based on survey data due to the dependency on device ownership and the selectivity of this group. The sample is further affected by the multitude of available devices, the technical capability of respondents to (understand instructions to) find their requested data and the availability of the measurement of specifically the number of active minutes on the device. This reduces the actual sample that can be used to donate data greatly.
These errors on the representation side of the framework are the main cause underlying the advice against using data from respondents’ personal activity trackers for national surveillance of PA. Moreover, these findings highlight broader concerns relevant to any general population study utilizing self-owned PA trackers. The observed limitations—such as the potential for biased samples due to variability in willingness and ability to donate data—suggest that caution is warranted when using personal activity trackers as the primary data source.
At the time of writing, distributing one type of research-grade device amongst the population to gather objective data on PA seems to be the most viable method. This completely overcomes the dependency on device ownership, and reduces the representativity issues due to a relatively high willingness to participate in such studies (Kapteyn et al. 2018). Additional large scale studies employing research-grade devices will have to examine whether using research-grade devices would indeed yield less measurement error and better statistics, and whether this outweighs the lower representativity of the sample and the costs of the logistics. Until then or in case of a limited research budget, self-report questionnaires are likely still the best way to monitor the PA of the general population.
Acknowledgements
We would like to thank Centerdata’s research assistants Dana Adriaansens and Soldado Koval, as well as panel manager Josette Janssen, for their tremendous help in developing study materials, planning the study, and conducting the fieldwork.
The views expressed in this report are those of the authors and do not necessarily correspond to the policies of Statistics Netherlands.
Lead author contact information
Maaike Kompier
me.kompier@cbs.nl
CBS-weg 11, 6412 EX Heerlen
