The Impact of Health Literacy–Sensitive Design and Heart Age in a Cardiovascular Disease Prevention Decision Aid: Randomized Controlled Trial and End-User Testing

Introduction

Background

Prevention of cardiovascular disease (CVD) includes lifestyle interventions and medication for those at highest risk who are most likely to benefit. An absolute risk approach is supported by clinical evidence and endorsed by many national guidelines worldwide [1-5]. The absolute risk of a heart attack or stroke in the next 5-10 years can be assessed using widely available calculators [1]; however, these tools are substantially underused in practice [6-11]. Providing medication to high-risk and not low-risk patients is a cost-effective approach [6]. However, up to 75% of high-risk patients do not receive recommended medication to prevent death and disability from CVD, whereas 25% of low-risk patients take medication they are very unlikely to benefit from [7]. Recent guideline changes have led to calls for a shared decision-making approach to ensure that medication prescription for blood pressure and cholesterol is more in line with patient values [12-14].

Health literacy also plays a role in CVD prevention. Low health literacy is common in many countries, with estimates ranging from 36% to 60% of the population in Australia, Europe, and the United States [15-17]. This is associated with poorer self-management, less access to the health system, increased incidence of chronic diseases, including CVD, and increased mortality [18]. Therefore, it is important to engage this group in communication strategies for CVD prevention. This requires changes to the design of web-based patient resources, as many Australians seek health information on the web [19,20], but fewer than 1% of health information websites meet the recommended readability levels. Grade 8 is recommended to meet the needs of people with varying health literacy [21,22].

Some countries have used web-based CVD risk assessment tools for absolute risk and heart age to engage consumers in CVD prevention, with millions of users worldwide [23-26]. However, our systematic review of 73 web-based CVD risk assessment tools available to consumers found that they were not suitable for people with lower health literacy: their readability level was too high; they frequently used unexplained medical terms; few used best practice risk communication formats such as frequencies in icon arrays; and they rated poorly on actionability (ie, clarity in instructions of what actions or steps to take), which makes it difficult for the average person to know what to do about the risk assessment result [27]. Our review of 25 web-based decision aids (DAs) for CVD prevention found similar issues with understandability and actionability [28], and few included lifestyle changes as an option to reduce risk, with many focusing on medication only.

There are several evidence-based strategies to address the issue of communicating CVD risk to people with lower health literacy, such as:

1. Use literacy-sensitive design to improve the readability of health information and reduce the cognitive load of action plans for behavior change [29-31].
2. Use best practice risk communication formats to explain abstract probabilities (eg, 16%) using icon arrays and more concrete frequencies (eg, 16 out of 100 people like you) [32-35].
3. Use patient DAs to improve understanding and decision-making, including both lifestyle change and medication, as clear actions that patients can take to reduce their CVD risk [29,36,37].

Objectives

This study aims to develop and test a new consumer engagement tool for CVD prevention based on the aforementioned strategies to address the needs of Australians with different levels of health literacy. It builds on our previous development of a general practitioner (GP)-focused risk calculator and DA [38] and evaluation of the national heart age calculator [26].

Methods

Ethics Approval

This study received ethics approval from the Human Research Ethics Committee of the University of Sydney (project number 2019/774).

Stage 1: Develop Consumer Engagement Tool

In stage one, we developed a literacy-sensitive version of our existing GP DA [39], which calculates 5-year risk of a CVD event based on current guidelines [1] and shows the effects of 9 lifestyle, medication, and supplement interventions [38]. This was based on previous reviews and evaluations of 73 CVD risk calculators and 25 CVD prevention DAs, which identified tools for many different CVD models, but none that matched Australian guidelines and best practice communication principles [27,28]. We added heart age to the Australian absolute CVD risk calculation based on published methods from New Zealand, both of which use the 5-year Framingham equation [40]. The literacy-sensitive design included simple language, supporting images, and white space to improve readability and understandability [30]. The text within this DA was evaluated using the Sydney Health Literacy Editor, a tool that automatically applies readability and actionability criteria to the text [41]. On the basis of this feedback, the final tool met the recommended grade 8 level. The literacy-sensitive version also included a novel action plan format developed by our team, which has been shown to reduce unhealthy lifestyle behaviors among people with low health literacy [31]. We added options for physical activity and smoking to the existing tools to reduce unhealthy snacking, drawing on previous literature on effective if-then plans in these areas. If-then plans help people identify an important environment context or trigger in which they find that they often carry out an unwanted behavior and to identify a new behavior that can be substituted for the unwanted behavior. These 2 components are formulated into an if-then statement or plan; for example, If I find myself eating unhealthy snacks when drinking a cup of tea, then I will eat a piece of fruit instead. In this study, we used an if-then format called a volitional help sheet, which prompts the person with predefined if and then statements [42-44].

Stage 2: Randomized Trial to Identify Best Formats for Low Versus High Health Literacy Design

Overview

The randomized trial was based on a 3×2 factorial, between-subject design to test the effect of literacy-sensitive design (literacy-sensitive DA, standard DA, or control: Heart Foundation patient information) and risk format (explaining CVD risk only [as a percentage risk], or CVD risk percentage+heart age) on psychological and behavioral outcomes. See Table 1 and Figure 1 for study design and Figure 2 and Multimedia Appendix 1 for example intervention content. The trial was preregistered at the Australian New Zealand Clinical Trials Registry (ACTRN12620000806965).

‎View this figure

‎View this figure

Recruitment

A national sample was recruited through Qualtrics (Qualtrics Inc), a web-based social research agency, with stratified sampling based on gender and age groups (5-year age groups from 45 to 74 years). Participants completed a CVD risk assessment based on the Australian guidelines and New Zealand approach to calculate heart age [1,40]. If blood pressure or cholesterol were not known, the average by age and gender based on non-diabetic participants in the AusDiab cohort was used (accessed via author JD), and all participants were advised to see a GP for a more accurate risk assessment. Participants with established CVD or those taking CVD prevention medications were excluded. Duplicate IP addresses were replaced, and stratified sampling was relaxed with additional quality checks added if hard-to-reach groups did not reach the quota after 2 weeks.

Measures

Established measures were used for the primary outcome of behavioral intentions (validated theory of planned behavior scale applied to smoking, diet, exercise, and GP visit) [49-51]. Secondary outcomes included self-reported behavior after 4 weeks compared with national guidelines for diet and physical activity [50,51], gist and verbatim knowledge (absolute risk percentage and heart age), emotional response using a validated scale (3 positive emotions, eg, hopeful, and 3 negative emotions, eg, anxious) [52], credibility of the information (that the information is personally relevant) [53], and decision conflict scale (uncertainty in decision-making) [54]. Details are presented in Table 2.

Analysis

A priori sample size calculations determined that 85 participants per randomized group (total n=510) would yield 90% power to detect a moderate effect size of Cohen d=0.5 (a standardized difference; this generic effect size estimate was selected because of the absence of similar trials on which to base calculations) in the primary outcome of intention to change lifestyle or any of the secondary outcomes, assuming a 2-sided Cronbach α of .05. We aimed to recruit an additional 20% more cases to account for potential missing values, totaling 600 participants (100 per group) at follow-up. This sample was inflated for recruitment to 850 participants to account for potential attrition of up to 30% between the intervention and follow-up.

Continuous outcome variables were modeled using linear regression. Dichotomous outcomes were analyzed using modified Poisson regression (using a log-link function with robust error variances). Ordinal logistic regression was used to analyze the ordered categorical outcomes. Count outcomes were modeled using negative binomial regression. All regression models included the DA group (literacy-sensitive DA, standard DA, or basic Heart Foundation patient information) and risk format (CVD risk percentage only or CVD risk percentage+heart age) as categorical variables and controlled for health literacy adequacy (categorical based on the Newest Vital Signs measure [55,56]: low, moderate, or adequate) and absolute risk (percentage). Postintervention and follow-up outcomes were analyzed separately, with follow-up analyses controlling for preintervention values where available. Pairwise comparisons were conducted to test these hypotheses. We also conducted exploratory analyses of potential differences in DA effects between health literacy levels by including a literacy-sensitive-by-DA interaction term and heart age category for heart age groups (younger or same vs older in stratified analyses). Chi-square test for paired proportions by McNemar was used to compare knowledge of heart age versus percentage risk among those who saw both. Analyses were conducted using Stata (version 16.1; StataCorp). No adjustments were made for multiple comparisons.

Hypotheses

1. The two DA formats will be more effective (ie, increase lifestyle change intentions or behavior and knowledge of risk without reducing credibility) than the standard Heart Foundation information.
2. The literacy-sensitive DA will be more effective than the standard DA for everyone (not just people with lower health literacy).
3. Adding heart age to absolute risk will be more effective than absolute risk alone.

Stage 3: Iterative End-User Testing With Varying Health Literacy Levels

As part of the follow-up survey, participants in the trial were invited to opt-in to a think aloud interview to provide further end-user testing and feedback for the literacy-sensitive version of the intervention. From the 27 participants who provided email addresses, 20 (74%) participants were selected to represent a range of ages, genders, risk levels, and health literacy levels. Participants went through the risk calculator in full while saying out loud everything they were thinking; for example, any areas of confusion. Further questions were asked to prompt more discussion or elaboration. Transcripts were thematically coded and discussed after each set of 4-5 interviews, and improvements were made to the intervention before the next set of interviews. We conducted 2 rounds of interviews with people with low health literacy as our key target group (8/20, 40%) and then tested the improved tool with people who had higher health literacy to ensure that it was suitable for these users in another 2 rounds (12/20, 60%).

Results

Stage 1

We used the question format and style of the current national heart age calculator as the basis for the risk factor questions in all groups, as well as the heart age presentation on that tool. The CVD risk results and DA were presented based on (1) our existing GP DA tool [39] (standard DA group), (2) a simplified version of the standard DA with supporting images (literacy-sensitive DA group; Figure 2), and (3) the current risk calculator from the National Vascular Disease Prevention Alliance [45]. See Multimedia Appendix 1 for example intervention content in each group.

Stage 2

Overview

The CONSORT (Consolidated Standards of Reporting Trials) diagram is shown in Figure 3, and the characteristics of all the participant groups in the intervention are shown in Table 3. We conducted a soft launch with 100 participants to check that we had an adequately low health literacy sample and adequate follow-up considering the COVID-19 disruptions in 2020 before proceeding with the full trial with no changes to the preregistered method. We recruited 859 participants for the intervention (including the 100 in the soft launch), with a target of 600 at the 4-week follow-up, for which we recruited 596 participants. The characteristics were similar among the groups for age and gender but some differences were observed for health literacy (relating to education) and absolute risk (relating to smoking and heart age); therefore, these 2 factors were controlled for in the analyses. In terms of dropout, there was no difference in the randomized DA group (P=.71), randomized to heart age (P=.91), health literacy level (P=.69), CVD risk level (P=.56), or heart age result (P=.30) between those who returned for follow-up and those who did not. The outcomes by trial group are shown in Table 4, and the analyses for each of the 3 hypotheses are shown in Tables 5-7.

‎View this figure

Postintervention Differences Among DA Groups

Immediately after the intervention, there were no differences among the 3 DA groups for the primary outcome of lifestyle intentions or secondary outcomes of risk perception, credibility, emotional response, or decisional conflict. For hypothesis 1, the combined DA groups did not differ from the control group for any outcome (Table 5). For hypothesis 2, there was no difference between standard and literacy-sensitive DAs for any outcome (Table 6). There were significant interactions between DA and health literacy for intention to talk to a doctor about medication (P=.02) and emotional responses (positive P=.01; negative P=.006). Participants with lower health literacy who received literacy-sensitive DA had a more negative or less positive emotional response and had stronger intentions to see a doctor about medication compared with the other groups (Table 6).

4-Week Differences Among DA Groups

At follow-up after 4 weeks, there were no significant differences between the control and DA groups for most self-reported behaviors. However, the literacy-sensitive DA group had higher fruit consumption compared with both the control (difference in predicted counts=0.69, 95% CI 0.32-1.06; P<.001) and standard DA groups (difference in predicted counts=0.48, 95% CI 0.11-0.86]; P=.01). The DA groups were more likely to know whether their risk was low, medium, or high than the control group (literacy-sensitive DA: incident rate ratio [IRR]=1.28, 95% CI 1.04-1.58; P=.02 and standard DA: IRR=1.41, 95% CI 1.14-1.74; P=.002). The standard DA group was more likely to know their exact risk percentage result compared with the control group (IRR=3.25, 95% CI 1.31-8.07; P=.01; Table 5). There were significant differences among DA groups by health literacy levels for self-reported calls to the Heart Foundation helpline (P<.001) and verbatim knowledge of CVD percentage risk at follow-up (P<.001). None of the participants with low health literacy reported calling the helpline or remembered their exact CVD risk in the control group. Standard DA increased both outcomes in all health literacy groups, and literacy-sensitive DA increased both outcomes in the low and high health literacy groups but not in the medium group (Table 6).

Postintervention Differences Among Heart Age Groups

Immediately after the intervention, there were no differences between the 2 heart age groups in the primary outcome of lifestyle intentions or secondary outcomes of risk perception or decisional conflict. For hypothesis 3, the heart age group was less likely to have a positive emotional response (mean difference −0.56, 95% CI −0.88 to −0.24; P=.001; Cohen d=0.23), less likely to perceive the message as credible (mean difference −0.20, 95% CI −0.35 to −0.05; P=.01; Cohen d=0.17), and more likely to know whether their risk was low, medium, or high (odds ratio 2.03, 95% CI 1.33-3.08; P=.001), compared with the percentage risk only group (Table 7). When the heart age result was older, there were significant differences indicating less positive (mean difference −0.75, 95% CI −1.19 to −0.31; P=.001; Cohen d=0.31) and more negative (mean difference 0.57, 95% CI 0.12 to 1.02; P=.01; Cohen d=0.23) emotional responses, lower credibility (mean difference −0.29, 95% CI −0.49 to −0.09; P=.005; Cohen d=0.25) and higher perceived risk level (odds ratio 2.11, 95% CI 1.31-3.39; P=.002) when heart age was shown. No such differences were found in those who received the same age or younger results (Table 7).

4-Week Differences Among Heart Age Groups

At the 4-week follow-up, there were no significant differences among the heart age groups in terms of lifestyle behavior change, seeing a doctor for a heart health check, or gist knowledge of risk level (Table 7). Unsurprisingly, being shown heart age led to greater gist knowledge of heart age (IRR 2.90, 95% CI 2.10-3.99; P<.001) and verbatim knowledge of heart age (IRR 18.13, 95% CI 4.36-75.48; P<.001) compared with those who were not shown their heart age, but there was no difference between the heart age and percentage risk only groups for knowledge of percentage risk. Within the heart age group that saw both risk formats, participants were more likely to have verbatim knowledge of their heart age (11%) than their percentage risk (6%, chi-square test for paired proportions by McNemar: χ²₁=6.1; P=.01, difference in proportions 5.4%, 95% CI 0.8%-10.0%). When the heart age result was older, there were significant differences indicating more vigorous exercise (mean difference 0.58, 95% CI 0.09-1.07; P=.02), more vegetable serves (mean difference 0.57, 95% CI 0.05-1.09; P=.032), higher chance of meeting guidelines for exercise (IRR 1.23, 95% CI 1.05-1.45; P=.01) and diet (IRR 1.48, 95% CI 1.00-2.18; P=.048), when heart age was shown. When the heart age result was the same or younger than their current age, there were significant differences, indicating fewer soft drink serves (mean difference −0.34, 95% CI −0.61 to −0.07; P=.012) and a higher chance of calling the Heart Foundation helpline (IRR 12.66, 95% CI 1.76 to 4.03; P<.001), when heart age was shown (Table 7).

Stage 3

Participant interviews were conducted in 4 stages so that any user feedback from the interviews could be discussed among the team (C Bonner, C Batcup, and JA) and then implemented into the calculator for the next interviews in an iterative process. The issues addressed in each round of interviews are shown in Multimedia Appendix 2.

Discussion

Principal Findings

We used both a mixed method development and evaluation process to produce a CVD DA that is effective for improving verbatim and gist knowledge of CVD risk and fruit consumption after 4 weeks. The resulting intervention is a scalable eHealth tool suitable for people with varying levels of health literacy. This consumer tool will supplement a GP version for use within consultations [38,39], providing GPs with a clear action for their patients to follow up when lifestyle change is recommended. This paper provides an example of how to apply literacy-sensitive design principles to evidence-based decision-making and behavior change tools. The results show that literacy-sensitive DAs can support people with lower health literacy in making informed decisions, while still being suitable for the general population.

Comparison With Previous Work

A recent review of DAs for people with lower health literacy [57,58] showed that DAs that use health literacy design strategies lead to improved knowledge, decisional conflict, and decision-making outcomes. Furthermore, DAs that used explicit strategies to reduce cognitive burden showed greater improvements in knowledge for people with low health literacy and from disadvantaged backgrounds [58]. The review highlighted the need for more consideration of health literacy in DA development. This study addresses these findings in the context of CVD prevention for the first time.

We observed several interactions with health literacy, showing the importance of considering this as a covariate when investigating shared decision-making and behavior change outcomes. The literacy-sensitive version of the DA produced more negative emotional responses and greater intention to speak to a doctor about medication options to reduce CVD risk among those with lower health literacy. This may reflect risk and choice awareness in this group if they had not previously considered themselves to have risk factors for heart disease that could be addressed with preventive medication. As this sample was predominantly low-risk, we would not want a DA to lead to greater actual medication uptake in this group; however, speaking with a physician about risk and how to reduce it may be a positive outcome in line with guidelines to assess risk in this age group [1]. We replicated previous DA studies by finding increased knowledge of risk among the DA groups compared with the control group [37]. We also replicated our previous finding that a literacy-sensitive action plan can improve diet outcomes across different levels of health literacy, although this was more marked for people with low health literacy [31,59].

This study also replicated several heart age effects found in reviews of previous research, in that it leads to a more negative emotional response, increased gist and verbatim knowledge of heart age, but not percentage risk, and reduced credibility, but is neutral for lifestyle change overall [60,61]. Our subgroup analyses suggest that more nuanced study designs are required to better understand the effects of heart age. First, among those who were shown their heart age, gist knowledge of percentage risk initially improved, but after 4 weeks, gist and verbatim knowledge were higher for heart age than for percentage risk. Previous studies have shown that people who receive an older heart age may react defensively and focus on other information, such as a low short-term risk level, which in turn may reduce their credibility of the risk result [26,62]. Analyses of people who received an older heart age result suggest that it may be useful as a marketing tool to gain attention and initiate behavior change, but knowledge of heart age did not translate to knowledge of risk. For the intended purpose of a DA to be used in a clinical context, the focus must be on validated risk results to make informed decisions about medication. Therefore, we decided to use the non–heart age version of the literacy-sensitive DA in future research in general practice. However, web-based heart age tools can incorporate DA and action plan elements with no detrimental effects.

Future Directions

Future trials need to be designed to isolate older heart age results and follow-up behavior over time. In considering how to power such trials, researchers will need to consider how the specific heart age tool they use is calibrated for the intended population (eg, approximately 50% older in our sample using the New Zealand method vs approximately 80% in the Australian/United Kingdom Heart Foundation tool [25,26]). The primary outcomes also need to be considered carefully. Most heart age research has been conducted with a primary outcome of immediate lifestyle change intentions, where we found no differences. More research could be done to verify the self-reported behavior change among people receiving older heart age results we observed after 4 weeks, using more objective measures such as pedometers.

The end-user interviews were helpful for improving simple navigation and wording issues in the literacy-sensitive version of the DA, but there were some larger issues that could not be resolved using a web-based tool. Most users did not know their blood pressure or cholesterol results; however, even if they had been assessed recently, they had difficulty understanding where different numbers should be entered. This was particularly difficult for cholesterol results in pathology test reports. Therefore, we will test the final revised tool in clinical practice to address the issue of unknown blood pressure and cholesterol, which reduces the accuracy and limits the display of options in line with the current medication guidelines. This tool will be integrated with additional Heart Foundation resources to improve other lifestyle outcomes.

Strengths and Limitations

A major strength of this study is that we were able to recruit a large, diverse sample in terms of health literacy and risk results. We had sufficient follow-up to run the study per protocol despite the COVID-19 disruptions and observed no difference in dropouts for key variables. A limitation is that the web-based panel sample may not be representative of the general population and may better reflect users of web-based heart age tools than patients presenting to primary care for CVD risk assessment. Furthermore, many participants did not know their blood pressure and cholesterol levels, which may have affected their response to the DA because of a less accurate CVD risk result. However, the use of averages reflects the approach used in currently available consumer tools for CVD risk assessment [26-28]. Different countries also use different CVD risk models or heart age algorithms, which may affect the results given the differences we observed in the older heart age sample. We conducted a large number of analyses on multiple outcomes; however, given the exploratory nature of the study, we did not make adjustments for multiple comparisons. The study was powered by moderate effect sizes and therefore may have lacked the power to detect more subtle differences; however, these findings will be useful for informing sample size calculations for future studies. Finally, we used validated outcomes where possible but behavior changes were self-reported. Future research on heart age should use objective measures over time.

Conclusions

This study shows the value of combining health-literacy–sensitive design with best practice risk communication and behavior change tools. Although aimed at addressing the needs of people with lower health literacy, this approach improved knowledge of CVD risk, heart age, and behavior in a sample with varying health literacy levels. The role of heart age remains somewhat unclear, with both advantages and disadvantages; however, there is no clear evidence of an effect on lifestyle change intentions or behavior overall. Further research should investigate implementation pathways for integrating such consumer tools with clinical practice and distinguish between older and younger heart age results.