A potential new data source for assessing the impacts of health reform: Evaluating the Gallup-Healthways Well-Being Index

in particular types of coverage.


INTRODUCTION
The Gallup-Healthways Well-Being Index is part of the Gallup Corporation's daily tracking poll, featuring questions on health insurance, health status, and access to care. The Well-Being Index (WBI), created in 2008, represents a potentially valuable new data source for health and health care research. It has been used for policy analysis on the expansion of insurance coverage to young adults under the Affordable Care Act (ACA), 1 and state-level variation in uninsurance rates. 2 Survey data in general is used extensively in health services research, particularly for studies of large coverage changes and insurance participation rates. [3][4][5][6] With major insurance expansions under the ACA set to begin in 2014 7 and ongoing major changes in health care delivery, the availability of rapid and accurate estimates on coverage and access to care will be critical. While administrative data can often provide rapid and accurate estimates of how many individuals have enrolled in or utilized a particular program, survey data provides an important cross-sectional view of overall coverage rates in the population, across the range of government programs and private plans. While the Gallup-Healthways WBI may be capable of fulfilling an important research role in the coming years, its comparability to wellestablished national surveys is unknown and is a pre-requisite to its broader use for research.
The Gallup-Healthways WBI is a daily telephone interview conducted throughout the year, interviewing 500 adults each day from all 50 states and the District of Columbia (1000 per day before 2013). Interviews are conducted with respondents on listed landline telephones and random-digit-dial cellular phones, with interviews in Spanish as needed. Gallup recently increased the proportion of cell phone participants based on shifts in communication behavior. 8 Samples are weighted to targets for region, age, gender, race, ethnicity, education, number of adults, and telephone status percent based on the Current Population Survey (CPS). Data can be summarized at the state, congressional district, and Metropolitan Statistical Area (MSA) levels.
Additional methodological detail is provided in the Appendix.
The objectives of this study are to describe the Gallup-Healthways WBI and compare its estimates of several health policy measures to other established national surveys.

METHODS 2.1 Data
We use data from the 2008-2011 Gallup-Healthways WBI. We compared the WBI to the We note that other national surveys, such as the National Health and Nutrition Examination Survey (NHANES), also provide estimates of health insurance coverage and access to care. However, we have limited our comparisons to the largest national surveys, with a particular emphasis on those surveys that can produce state-level estimates or that have been used most frequently to analyze major changes in insurance coverage. [9][10][11][12] For each survey, we used data from 2008 through the most recently available year at the time of our analysis, which differed by survey (2011 for ACS, CPS, BRFSS, and NHIS; 2009 for MEPS). Gallup-Healthways WBI data were obtained under contract with the Gallup Organization. The remaining data sources are publicly available at no cost.

Statistical Analysis
Our analytical approach was largely descriptive. We compared the following demographic characteristics of the Gallup-Healthways WBI adult sample to recent estimates (calendar year 2010) from the CPS, ACS, and NHIS: age, race/ethnicity, children in the household, education, and income. For each outcome, we did bivariate comparisons between Gallup-Healthways WBI estimates and the government surveys using chi square tests for binary outcomes and t-tests for continuous outcomes. For context, we also conducted bivariate comparisons between estimates from the government surveys. We categorized each outcome as similarly as possible to its counterpart across surveys. For instance, household income in the Gallup-Healthways WBI is reported in discrete categories, so in comparing to other surveys we used the same income cutoffs. We limited the sample in all analyses to adults 18 and over, to be consistent with the Gallup-Healthways WBI.
We compared the following health insurance outcomes in Gallup-Healthways WBI to the CPS, ACS, and NHIS: any health insurance by year (2008)(2009)(2010)(2011)  Uninsured. For elderly adults, Medicare was first in the hierarchy, with the remaining order unchanged. We also tested an alternative hierarchy, in which ESI was placed before Medicare for adults over age 65. For coverage rates by state and over time, we used Pearson correlation coefficients to describe the relationship between the Gallup-Healthways WBI estimates and those in other data sets.
We compared access measures and health status using the NHIS, MEPS, and BRFSS.
Outcomes were having a personal doctor / usual source of care, cost-related barriers to care, selfreported health, and history of specific disease diagnoses.
Precise wording for the Gallup-Healthways WBI items are presented in the Appendix.
All analyses were conducted using STATA 12.0, accounting for the specific survey design of each data source and using appropriate weights. In the Gallup-Healthways WBI, this included using separate weights, provided by Gallup-Healthways, for state and national estimates; national weights are available on a daily basis, but state-specific weights are only calculated by Gallup-Healthways every 6 months. Table 1 presents key features of each survey. Notable strengths of the Gallup-Healthways WBI are that it covers more health-related domains than Census surveys; it has a large sample (355,000) compared to the most detailed health surveys (NHIS and MEPS, which survey approximately 100,000 and 35,000 individuals, respectively); it supports state-level analyses; and it has a very rapid turnaround time with data available for analysis within one week of collection (though state-level weights are only calculated twice annually). The survey's limitations for research purposes include a lack of information on children, lack of historical data prior to 2008, more limited income data, and availability through direct purchase from Gallup-Healthways. Additionally, the Gallup-Healthways WBI asks respondents to report detailed insurance and health-related information only about themselves, unlike most national surveys, which also collect information about other members of the household.

RESULTS
The Gallup-Healthways WBI describes its purpose as to provide "statistics on the state of well-being in the United States." But as a commercial product designed to provide real-time data to its customers, it differs in important ways from the government surveys that are designed primarily for public research efforts. For instance, Gallup-Healthways WBI's frequent methodological and question changes introduce a level of uncertainty not generally encountered in government surveys, and the reasons for methodological changes are not always made public.
In this vein, the daily WBI sample was cut in half as of January 2013 to enable Gallup to move resources towards expanding their international polling; the net impact was a significant reduction in statistical power and precision compared to the first four years of the survey. While government surveys also change over time -in particular some government surveys are introducing question changes to better detect the coverage and access effects of the Affordable Care Act -these changes tend to be approached with caution and attention to minimizing breaks in trend. In addition to WBI's unpredictability, the most concerning methodological limitation of the survey is its response rate of 11%. While this rate is similar to that of other telephone surveys, it is far below those of the government surveys (which range from 50%-98%). Table 2 compares demographics from the surveys. Essentially all the demographic findings differed statistically between the WBI and other surveys, in some cases even for differences that were less than 0.5 percentage points, due the extremely large samples for pooled analyses (ranging from approximately 400,000 for the Gallup-Healthways WBI vs. MEPS comparisons, to nearly 3.5 million for Gallup-Healthways WBI comparisons to the ACS). For the same reason, the government surveys have statistically significant differences from one another on numerous outcomes, including income, education, insurance coverage types, health status, and access to care. Given these concerns, our discussion below focuses on the magnitude of the differences in estimates that may be relevant for policy purposes, rather than statistical significance.
As shown in Table 2, the Gallup-Healthway WBI sample had fewer adults ages 18-34 (25.3% total, versus 30.8% in CPS, 30.5% in ACS, and 30.8% in NHIS) and slightly more elderly adults (10.8% between the ages of 65-74 and 8.3% who are 75 or older, compared to 9. In terms of income, 22% of the Gallup-Healthways WBI survey participants did not respond to the income question, although CPS and ACS have imputed income in their public use files, and we used the NHIS imputed income files, creating the false appearance that no adults have unknown incomes in those surveys when in fact the imputation rates for those surveys is quite similar to that in the WBI. For example, between 24-33% of adults in the NHIS had unknown incomes in survey years 2007 to 2010, and 22.4% of ACS respondents had some or all income imputed in 2010. To test for any underlying pattern in the missing data for income in the Gallup-Healthways WBI, we conducted an imputation for missing income values using multivariate regression. First, we translated the Gallup-Healthways WBI categories into a linear income variable by taking the midpoint of each income bucket. Then, we modeled income as a function of age, race/ethnicity, sex, state of residence, family size, marital status, self-reported health status, employment, and health insurance. Overall, those with missing income data had an imputed average income of $51,000, compared to $55,000 for those reporting their household income. Imputed income values also were more clustered in the middle of the income distribution, with fewer households having income under $24,000, and fewer having income higher than $90,000. Putting these imputed values together with the observed values (Appendix Table 1), the overall impact is that the Gallup-Healthways WBI has a similar share of lowincome adults but more adults in the middle income range ($24,000-$60,000) and fewer higherincome adults (above $90,000) than the government surveys.  Panel B reports estimates using a health insurance hierarchy to assign each person a primary form of coverage. This produces somewhat more comparable estimates between Gallup-Healthways WBI and other surveys, though large differences persist. Among non-elderly adults, Medicaid rates in Gallup-Healthways WBI remained roughly half of those in the other surveys, though the Gallup-Healthways WBI Medicare rate exceeded other surveys. The "Other Sources" category in Gallup-Healthways WBI remained nearly twice as large as the non-group share in the CPS, ACS, and NHIS. Among the elderly, the insurance hierarchy led to more similar results among the surveys for non-Medicare forms of coverage, but overall, Medicare rates in Gallup-Healthways WBI remain much lower than in other surveys. 3.7% of non-elderly adults in Gallup-Healthways WBI reported having insurance but did not know or refused to provide their coverage type. Though Table 3 appears to indicate no missing values for health insurance for the CPS and ACS, insurance values are imputed for these surveys. In 2010, between 5.2% and 8.5% of health insurance type values were imputed in the ACS, depending on insurance type.

Key Findings
In this exploratory analysis of the Gallup-Healthways WBI, we found that the survey provides reasonably similar data when compared to established national surveys on several important health-related measures, including population demographics, the overall uninsured rate at the national and state levels, trends in coverage over time, access to care, and self-reported health status, though some important differences are evident and would need to be acknowledged in any future research using the Gallup-Healthways WBI: the Gallup sample is slightly older, has fewer minorities, and is more highly educated than samples in other national surveys. In all of these results, we focus on the absolute magnitude of differences in estimates, as we feel that statistical significance has little relevance in very large samples such as these -evident in the fact that the gold-standard government surveys themselves revealed numerous statistically significant differences from one another in our analyses.
Several key features of the Gallup-Healthways WBI make it appealing for research: it has a large enough sample to support state-level estimates on an annual basis for nearly all 50 states (though with a significant loss of precision due to the 50% reduction in health-related sample size in 2013); it has a very rapid turnaround time, enabling data analysis to occur literally within days of data collection; and it has questions across multiple domains of health care. The CPS and ACS have limited or no information on health care utilization, while MEPS and NHIS do provide considerable data on this measure, but with smaller samples and slower turnaround times than Gallup-Healthways WBI. These strengths of the Gallup-Healthways WBI may help fill some of the data gaps that others have identified in the current national survey options for health services research. 14 The demographic differences in the Gallup-Healthways WBI discussed above may particularly have an effect on measures of insurance type, as Gallup-Healthways may not be reaching younger, minority, lower education populations as effectively. Some of these differences may be an artifact of Gallup-Healthways WBI's weighting to telephone households, not the entire adult population. But overall differences were generally small, on the order of several percentage points. Also, the Gallup-Healthways WBI questions on race and ethnicity differed from other surveys prior to 2011. Gallup-Healthways WBI did not begin recording separate results for American Indians / Alaska Natives, Native Hawaiians, or Pacific Islanders until April 2011, meaning that these groups are included in the "Other" category prior to that year. Also beginning in 2011, respondents were allowed to select more than one race, whereas previously they had to choose just one. Finally, Gallup-Healthways WBI is conducted only in English and Spanish, which may explain the lower prevalence of Asians.
Gallup-Healthways WBI's income data is also collected differently from other surveys.
Instead of collecting specific dollar values for different types of income for each person, and then summing across household members, Gallup-Healthways WBI simply asks the respondent to describe "monthly household income" and records the response in 10 discrete categories. This precludes making precise estimates of a family's poverty status and/or income related to key program eligibility thresholds, such as the ACA's Medicaid eligibility expansion up to 138% of the Federal Poverty Level. In this regard, Gallup-Healthways WBI data provide limited income information analogous to that collected in the BRFSS, which is one disadvantage compared to the Census surveys, NHIS, and MEPS. Nonetheless, our analyses suggest that the Gallup-Healthways WBI captures a similar share of low-income households as other surveys, and in fact, the main income differences were primarily the lower prevalence of high-income households in Gallup-Healthways WBI and a higher prevalence of respondents refusing to provide income data. After income imputation, the Gallup-Healthways WBI has a similar share of low-income adults but more adults in the middle income range ($24,000-$60,000) and fewer higher-income adults (above $90,000) than the government surveys. There is some evidence that omnibus income questions, such as those used by Gallup-Healthways, produce lower estimates than the more detailed series of questions in the CPS. 15 In terms of health insurance, results were mixed. The government surveys differ in how questions about health insurance are asked, but despite that, produce similar estimates of the uninsured rate -other than the BRFSS. Gallup-Healthways WBI reports an uninsured rate

Limitations
Like the Gallup-Healthways WBI, our analysis has strengths and limitations. In the interest of introducing this dataset to the research community, we analyzed a wide range of outcomes at a broad descriptive level. An alternative approach would be to take a small number of outcomes (such as insurance coverage type) and conduct a more intensive exploration, examining patterns for various subgroups and comparing trends over time within those groups.
Future research along those lines for key policy outcomes would be worthwhile.
Another issue for future exploration is whether the differences we observed between Gallup-Healthways WBI and other sources could be mitigated with various statistical techniques.
Options would include using imputation for missing values, and potential re-weighting to more closely match national estimates. However, the latter approach is only possible using older data from other sources since new data from Gallup-Healthways WBI will be available months before the other surveys release their findings. Given that one of the main advantages of the Gallup-Healthways WBI data is its rapid release, waiting for corroborating results from other surveys may not be practical.

Conclusion
The Gallup-Healthways Well-Being Index is a valuable but imperfect new data option for health policy research. In the areas of population demographics, overall rates of health insurance coverage, access to care, and self-reported health status, Gallup-Healthways WBI data are reasonably similar to well-established and validated national surveys. Estimates of type of insurance are less comparable, particularly for public coverage, which could lead to improper estimation of effects of particular policies or trends on coverage type. Gallup-Healthways WBI data seem particularly well-suited for real-time analyses of certain changes in health care trends that do not require distinguishing between different types of insurance coverage, such as whether the ACA had reduced the number of uninsured adults in the U.S., similar impacts of state expansions, and the impact of these changes on access to care. However, the lack of historical data or information on children, the lower response rate compared to in-person interview surveys, and its availability only under direct purchase from Gallup-Healthways WBI are important limitations to its use. Overall, the Gallup-Healthways WBI can fill important needs as a complement to but not a replacement for the well-established and highly rigorous government surveys, which likely will remain the workhorses for health services research in the years to come.   For the NHIS, we included "other public coverage" and "other government coverage" in Medicaid and CHIP, and "all private insurance not associated with employment or a union" and "other private coverage" in our "other/nongroup" category. In analysis of the CPS, we describe the data as based on the year to which the questions were in reference. While some prior research indicates that CPS insurance estimates more closely resemble point-in-time estimates than 'full prior year' estimates (Swartz 1986), other research indicates that respondents do understand that the questions specifically refer to the previous year (Klerman, Davern et al. 2009), and this is likely also the case for income. § For adults 18-64, the hierarchy was: 1) ESI, 2) Medicaid/CHIP, 3) Medicare, 4) Military coverage, 5) Other or Non-Group Coverage, 6) Uninsured. For elderly adults, Medicare was first in the hierarchy, with the remaining order unchanged. Using chi square tests for categorical variables. † This item in the MEPS measured the % of adults reporting that they "felt sad or depressed at least several days in the past 2 weeks," and over the past 30 days in the BRFSS. All other questions took the form, "Have you ever been diagnosed with…"