Digital Health Enablment (DHE) Study
Draft Report of the Pilot Conducted in July 2025
This Report
This report summarises the pilot of the digital health enablement (DHE) measure with 65 participants (50 online, 15 paper) in July 2025. Its purpose is to evaluate the study methodology and identify areas for improvement before full-scale implementation.
Executive Summary
A pilot survey was conducted in July 2025 to test and refine the survey methodology. The study recruited 65 participants through two channels: 50 participants via the online survey recruitment platform Prolific and an additional convenience sample of 15 friends and family members of the research team who completed paper-copies of the survey. The purpose of the pilot was to evaluate the survey instrument and procedures prior to full-scale implementation.
50 of 52 online participants successfully completed the study. Four attention check questions requiring participants to select specific predetermined answers were embedded throughout the survey and were passed in 100% of cases among online participants, though these questions caused some confusion in paper-based versions, with one participant failing to respond to them.
The sample comprised 54% males (n = 27) and 46% females (n=23). Efforts to over-represent lower socioeconomic backgrounds were successful, with 58% earning below £28,000 annually and 54% having school-level, vocational, or no formal qualifications. Ethnic composition mirrored the UK population: 88% (n = 45) White British/Northern Irish and 12% (n=6) ethnic minorities.
Author: Lee Mercer
Date: 12 August 2025
Version: 0.1
1. Prolific recruitment and completion
Histogram of completion times
Summary
| Metric | Value |
|---|---|
| Overall Completion Rate | 98% |
| Average Completion Time | 6.4 minutes (SD = 2.1) |
| Median Completion Time | 5.9 minutes |
2. Sample demographics
Age
| Statistic | Value |
|---|---|
| Mean | 46.5 |
| Standard Deviation | 14.8 |
| Median | 49.5 |
| Minimum | 21.0 |
| Maximum | 77.0 |
Age groups
| Age Group | Count | Percentage |
|---|---|---|
| 16-24 | 4 | 7.8 |
| 25-34 | 7 | 13.7 |
| 35-44 | 12 | 23.5 |
| 45-54 | 12 | 23.5 |
| 55-64 | 9 | 17.6 |
| 65+ | 6 | 11.8 |
| NA | 1 | 2.0 |
Age distribution
Gender
| Gender | Count | Percentage |
|---|---|---|
| Male | 27 | 54 |
| Female | 23 | 46 |
Education
| Education Level | Count | Percentage |
|---|---|---|
| No Qualifications | 6 | 12 |
| School-level (GCSEs, CSEs, O-Levels, or equivalent) | 14 | 28 |
| Vocational and technical (NVQ Level 1 and 2, BTEC, Apprenticeships, or equivalent) | 7 | 14 |
| Further education (A-Level, NVQ, Scottish Highers, or equivalent) | 6 | 12 |
| Higher education (HNC, HND, Foundational Degree, or equivalent) | 1 | 2 |
| Undergraduate Degree (BA, BSc, BEng, or equivalent) | 10 | 20 |
| Postgraduate and professional (MA, MSc, MBA, PhD, or equivalent) | 6 | 12 |
Ethnicity
| Ethnic Group | Count | Percentage |
|---|---|---|
| English/Welsh/Scottish/Northern Irish/British | 44 | 88 |
| Indian | 2 | 4 |
| Any other White Background | 1 | 2 |
| Chinese | 1 | 2 |
| White and Black African | 1 | 2 |
| White and Black Caribbean | 1 | 2 |
Income
| Income Level | Count | Percentage |
|---|---|---|
| Less than £19,000 a year | 16 | 32 |
| £19,001 to £28,000 | 13 | 26 |
| £28,001 to £38,000 | 10 | 20 |
| £38,000 to £55,000 | 4 | 8 |
| More than £55,000 a year | 7 | 14 |
3. Item-Level Analysis
Items with ‘Critical’ and ‘High’ Concerns
Whilst there are no specific thresholds for skewness or kurtosis in the literature, skewed distributions limit the ability of a measure to discriminate between individuals, and impact upon correlation analysis (DeVellis and Thorpe, 2021). Ceilings or floors of less than 20% are generally acceptable and the skew should ideally be no greater than -2 or +2.
The items listed in the table below demonstrate problematic response distributions, typically characterized by severe skewness (|skew| > 2), floor or ceiling effects (>40% of responses at scale endpoints), and restricted variance that limits their discriminative capacity.
Definitions
- Floor effect - When a large percentage of participants select the minimum response option (e.g., “Strongly Disagree”).
- Ceiling effect - When a large percentage of participants select the maximum response option (e.g., “Strongly Agree”).
- Skewness - A measure of the asymmetry of the response distribution. Values between -1 and +1 are generally acceptable, while values beyond ±2 indicate severe skewness.
- Kurtosis - How much probability mass is concentrated in the tails of a distribution compared to a normal distribution.
While the pilot study’s sample size (n = 50) necessitates cautious interpretation, identifying problematic items at the pilot stage, present an opportunity improve these items items before full-scale implementation.
In total, eleven items (22% of the substantive survey content) met the criteria for high severity or critical classification according to the psychometric thresholds detailed below.
Crtieria
| Issue | Critical | High | Moderate | Mild |
|---|---|---|---|---|
| Skewness | |skew| > 6 | |skew| > 2 | 1 < |skew| ≤ 2 | 1 < |skew| ≤ 1.5 |
| Floor Effect | 100% of responses at minimum | >40% at minimum | 20-40% at minimum | 15-20% at minimum |
| Ceiling Effect | 100% of responses at maximum | >40% at maximum | 20-40% at maximum | 15-20% at maximum |
| Variance (SD) | SD = 0 | SD < 0.70 | 0.70 ≤ SD < 1.0 | 0.90 ≤ SD < 1.0 |
| Kurtosis | |kurt| > 10 | |kurt| > 4 | 3 < |kurt| ≤ 4 | 2 < |kurt| ≤ 3 |
| Response Categories | Only 1-2 categories used | Only 3 categories used | Only 4 categories used | Only 5 categories used |
| Classification Logic | If any critical condition is met | If any high condition is met OR ≥3 moderate conditions | If any moderate condition is met OR ≥2 mild conditions | If any mild condition is met |
Table of items exhibiting ‘critical’ or ‘high’ concerns
| Item | Item Wording | Severity | Mean | SD | Skewness | Kurtosis | Floor % | Ceiling % | Issues |
|---|---|---|---|---|---|---|---|---|---|
| Item_1 | I would understand what a health technology was asking me to… | High | 6.041 | 0.763 | -0.341 | -0.531 | 2.0 | 28.6 | ceiling effect; low variance |
| Item_18 | I would expect health technologies to have measures in place… | High | 6.020 | 1.216 | -1.604 | 2.584 | 4.1 | 42.9 | moderate negative skew; extreme ceiling effect |
| Item_24 | I am confident I could learn how to use a fitness app if I w… | High | 6.327 | 0.774 | -1.139 | 1.137 | 4.1 | 46.9 | moderate negative skew; extreme ceiling effect; low variance |
| Item_25 | I am confident I could use a fitness app to monitor my healt… | High | 6.347 | 0.751 | -0.926 | 0.293 | 2.0 | 49.0 | extreme ceiling effect; low variance |
| Item_26 | I am confident I could use a fitness app regularly if I want… | High | 5.939 | 1.008 | -1.433 | 3.043 | 2.0 | 28.6 | moderate negative skew; ceiling effect; high positive kurtosis |
| Item_27 | I am confident I could use a fitness app without help if I w… | High | 6.327 | 0.658 | -0.434 | -0.814 | 10.2 | 42.9 | extreme ceiling effect; very low variance; limited range usage |
| Item_32 | I wouldn’t use a a fitness app because I find technology fru… | High | 2.000 | 1.429 | 2.267 | 4.882 | 40.8 | 4.1 | severe positive skew; extreme floor effect; high positive kurtosis |
| Item_4 | I would understand information about my health from health t… | High | 5.735 | 0.995 | -1.081 | 2.225 | 2.0 | 20.4 | moderate negative skew; ceiling effect; low variance |
| Item_6 | I can usually download apps. | High | 6.592 | 0.674 | -1.725 | 2.964 | 2.0 | 67.3 | moderate negative skew; extreme ceiling effect; very low variance |
| Item_7 | I can usually use digital technology. | High | 6.163 | 0.986 | -1.725 | 4.450 | 2.0 | 42.9 | moderate negative skew; extreme ceiling effect; low variance; high positive kurtosis |
| Item_9 | I often struggle to use digital technology. | High | 2.041 | 1.224 | 1.927 | 4.373 | 34.7 | 2.0 | moderate positive skew; floor effect; high positive kurtosis |
Revise wording to items?
One option to address problematic items is to modify items to avoid extreme language that pushes responses toward scale endpoints. For example, Item 9 - “I often struggle to use digital technology” could be revised to “I find it easy to adapt when digital technology updates or changes” which would allow more meaningful discrimination between respondents’ capabilities.
Suggestions for wording revisions are set out in the item-level summaries which follow.
4. Individual item analysis
Item 1 - “I would understand what a health technology was asking me to do to improve my health”
| Response | Response Value | Count | Percentage |
|---|---|---|---|
| Neither agree nor disagree | 4 | 1 | 2.0 |
| Somewhat agree | 5 | 10 | 20.4 |
| Agree | 6 | 24 | 49.0 |
| Strongly agree | 7 | 14 | 28.6 |
Suggested Revision
“I would understand detailed health recommendations from technology”
Item 18 - “I would expect health technologies to have measures in place to keep my information safe”
| Response | Response Value | Count | Percentage |
|---|---|---|---|
| Disagree | 2 | 2 | 4.1 |
| Neither agree nor disagree | 4 | 3 | 6.1 |
| Somewhat agree | 5 | 6 | 12.2 |
| Agree | 6 | 17 | 34.7 |
| Strongly agree | 7 | 21 | 42.9 |
Suggested Revision
“I don’t have concerns about the security of my personal data when using health technology”
Item 24 - “I am confident I could learn how to use a fitness app if I wanted to”
| Response | Response Value | Count | Percentage |
|---|---|---|---|
| Neither agree nor disagree | 4 | 2 | 4.1 |
| Somewhat agree | 5 | 3 | 6.1 |
| Agree | 6 | 21 | 42.9 |
| Strongly agree | 7 | 23 | 46.9 |
Suggested Revision
“I am confident I could quickly learn how to use a fitness app if I wanted to”
Item 25 - “I am confident I could use a fitness app to monitor my health if I wanted to”
| Response | Response Value | Count | Percentage |
|---|---|---|---|
| Neither agree nor disagree | 4 | 1 | 2.0 |
| Somewhat agree | 5 | 5 | 10.2 |
| Agree | 6 | 19 | 38.8 |
| Strongly agree | 7 | 24 | 49.0 |
Suggested Revision
“I am confident I could get the most of the health monitoring features in a fitness app if I wanted to”
Item 26 - “I am confident I could use a fitness app regularly if I wanted to”
| Response | Response Value | Count | Percentage |
|---|---|---|---|
| Disagree | 2 | 1 | 2.0 |
| Neither agree nor disagree | 4 | 3 | 6.1 |
| Somewhat agree | 5 | 7 | 14.3 |
| Agree | 6 | 24 | 49.0 |
| Strongly agree | 7 | 14 | 28.6 |
Suggested Revision
“I am confident I could remember to use a fitness app every day if I wanted to”
Item 27 - “I am confident I could use a fitness app without help if I wanted to”
| Response | Response Value | Count | Percentage |
|---|---|---|---|
| Somewhat agree | 5 | 5 | 10.2 |
| Agree | 6 | 23 | 46.9 |
| Strongly agree | 7 | 21 | 42.9 |
Suggested Revision
“I am confident I could master a fitness app without help if I wanted to”
Item 32 - “I wouldn’t use a fitness app because I find technology frustrating”
| Response | Response Value | Count | Percentage |
|---|---|---|---|
| Strongly disagree | 1 | 20 | 40.8 |
| Disagree | 2 | 22 | 44.9 |
| Somewhat disagree | 3 | 3 | 6.1 |
| Somewhat agree | 5 | 1 | 2.0 |
| Agree | 6 | 1 | 2.0 |
| Strongly agree | 7 | 2 | 4.1 |
Suggested Revision
“I wouldn’t want to try a new fitness app because I can find technology frustrating”.
Item 4 - “I would understand information about my health from health technology”
| Response | Response Value | Count | Percentage |
|---|---|---|---|
| Disagree | 2 | 1 | 2.0 |
| Neither agree nor disagree | 4 | 3 | 6.1 |
| Somewhat agree | 5 | 13 | 26.5 |
| Agree | 6 | 22 | 44.9 |
| Strongly agree | 7 | 10 | 20.4 |
Suggested Revision
“I would understand detailed reports about my health from health technology”
Item 6 - “I can usually download apps”
| Response | Response Value | Count | Percentage |
|---|---|---|---|
| Neither agree nor disagree | 4 | 1 | 2.0 |
| Somewhat agree | 5 | 2 | 4.1 |
| Agree | 6 | 13 | 26.5 |
| Strongly agree | 7 | 33 | 67.3 |
Suggested Revision
“I hardly ever run into difficulties when downloading apps”
Item 7 - “I can usually use digital technology”
| Response | Response Value | Count | Percentage |
|---|---|---|---|
| Disagree | 2 | 1 | 2.0 |
| Neither agree nor disagree | 4 | 1 | 2.0 |
| Somewhat agree | 5 | 7 | 14.3 |
| Agree | 6 | 19 | 38.8 |
| Strongly agree | 7 | 21 | 42.9 |
Suggested Revision
Item 7 - “I can use most forms digital technology without difficulty”
Item 9 - “I often struggle to use digital technology”
| Response | Response Value | Count | Percentage |
|---|---|---|---|
| Strongly disagree | 1 | 17 | 34.7 |
| Disagree | 2 | 23 | 46.9 |
| Somewhat disagree | 3 | 4 | 8.2 |
| Neither agree nor disagree | 4 | 2 | 4.1 |
| Somewhat agree | 5 | 2 | 4.1 |
| Strongly agree | 7 | 1 | 2.0 |
Suggested Revision
Item 9 - “I find it easy to adapt when digital technology updates or changes”.
5. Inter-Item Relationships
Inter-item correlation analysis
Total survey items (excluding attention checks): 45 Correlation summary statistics
| Statistic | Value |
|---|---|
| Mean Correlation | 0.129 |
| Median Correlation | 0.184 |
| Standard Deviation | 0.379 |
| Minimum Correlation | -0.858 |
| Maximum Correlation | 0.887 |
| 25th Percentile | -0.146 |
| 75th Percentile | 0.420 |
Problematic correlations
Highly correlated items (r> .85-.90) can create statistical problems of multicolliniarity during analysis, making it difficult to interpret regression analysis/structural equation modelling. Further, they may indicate redundancy and inefficiency in the scale, by measuring the same construct twice. However, this is less of an issue with a long-form draft scale where the intention is to reduce the number of items following analysis.
| Item_1 | Item_2 | Correlation |
|---|---|---|
| Item_29 | Item_36 | 0.887 |
| Item_31 | Item_36 | -0.858 |
| Item_36 | Item_41 | 0.859 |
Correlation distribution
Items within unidimensional scales would be expected to strongly correlated (typically r>.50-.80). However, items within multidimensional scales, where the dimensions are not independent and theoretically related might show weaker correlations (r>.10-.50).
6. Recommendations for Full Study
Survey Modifications
Items requiring revision
- Consider revisions to Items 1, 4, 6-7, 9, 18, 24-27 & 32 to improve discriminant validity
- Remove attention checks from paper-based surveys
Changes to format
- Remove randomisation from paper-based surveys to aid data entry
- Ensure Prolific ID is a mandated field
Data Collection Protocols
Participant screening
- Consider additional focus on participants from ethnic minorities, and younger people on low incomes.
Sample size considerations
Integration with Main Dataset
Feasibility assessment:
- Can pilot data be included in main analysis?
- Not possible to exclude those who have already participated.
- Wording of some items to be revised
- Not possible to exclude those who have already participated.
Appendices
Appendix A: Correlation Matrix
[Include full inter-item correlation matrix]