Prospective evaluation of a complex public health intervention: lessons from an initial and follow-up cross-sectional survey of the tuberculosis strain typing service in England

Abstract

Background: The national tuberculosis strain typing service (TB-STS) was introduced in England in 2010. The TB-STS involves MIRU-VNTR typing of isolates from all TB patients for the prospective identification, reporting and investigation of TB strain typing clusters. As part of a mixed-method evaluation, we report on a repeated cross-sectional survey to illustrate the challenges surrounding the evaluation of a complex national public health intervention. Methods: An online initial and follow-up questionnaire survey assessed the knowledge, attitudes and practices of public health staff, physicians and nurses working in TB control in November 2010 and March 2012. It included questions on the implementation, experience and uptake of the TB-STS. Participants that responded to both surveys were included in the analysis. Results: 248 participants responded to the initial survey and 137 of these responded to the follow-up survey (56% retention). Knowledge: A significant increase in knowledge was observed, including a rise in the proportion of respondents who had received training (28.6% to 67.9%, p = 0.003), and the self-rated knowledge of how to use strain typing had improved (‘no knowledge’ decreased from 43.2% to 27.4%). Attitudes: The majority of respondents found strain typing useful; the proportion that reported strain typing to be useful was similar across the two surveys (95.7% to 94.7%, p = 0.67). Practices: There were significant increases between the initial and follow-up surveys in the number of respondents who reported using strain typing (57.0% to 80.5%, p < 0.001) and the proportion of time health protection staff spent on investigating TB (2.74% to 7.08%, p = 0.04). Conclusions: Evaluation of a complex public health intervention is challenging. In this example, the immediate national roll-out of the TB-STS meant that a controlled survey design was not possible. This study informs the future development of the TB-STS by identifying the need for training to reach wider professional groups, and argues for its continuation based on service users’ perception that it is useful. By highlighting the importance of a well-defined sampling frame, collecting baseline information, and including all stakeholders, it provides lessons for the implementation of similar services in other countries and future evaluations of public health interventions. Electronic supplementary material The online version of this article (doi:10.1186/1471-2458-14-1023) contains supplementary material, which is available to authorized users.