More photos »

Quality-of-care indicators and analysis: Scaling up beyond the numbers

Currently, pronouncements on the progress and success in ART implementation have largely been confined to total numbers of patients started on therapy, but the benefits of ART are best realized when patients remain under long-term and uninterrupted treatment. MSF has integrated treatment literacy; adherence counseling; patient and outcome tracking; stockout prevention; decentralization and task-shifting; free health care; and treatment simplification to enhance ART programs and thus the likelihood of patients remaining on long-term ART.

Between 2002 and the end of 2007, the number of people accessing ART is reported to have increased 10-fold, to 3 million individuals in middle- and low-income countries, representing about 30% of those medically in need. In 2006, participating countries in the United Nations General Assembly Special Session (UNGASS) on HIV/AIDS agreed to the principle of “Universal Access” to ART, where by 2010 such therapy is to be available to at least 80% of those in need, representing a greater than 3-fold increase above current coverage.

The budget for national ART programs in these countries has come primarily from external funding sources such as the Global Fund, PEPFAR, and World Bank, though each national program has developed its own strategies for implementation. Although most national programs have many features in common (eg, use of a public health approach, indicators of when to start ART and with which drugs, decentralization of treatment sites to the periphery, task-shifting to non-physician cadres), sufficient differences exist in approach and country capacity in achieving stated program goals that measures of program effectiveness will vary significantly.

Funding entities (such as the Global Fund through “performance-based” funding, and PEPFAR) and international health agencies (such as WHO and UNAIDS) have provided guidance to implementing agencies, including national programs. To varying degrees, they require reporting on certain pertinent program “quality” indicators. Countries are also encouraged to fund and develop their monitoring and evaluation (M&E) activities. Nevertheless, a deficit of data exists regarding the progress in scaling up beyond the total numbers of patients having accessed treatment.

Clearly systems providing HIV care require the capacity to identify people who have HIV infection, clinical and/or biological markers indicating the need for ART, and when to start them on treatment. In recent years, countries have become increasingly reliable in reporting the total number of patients started on treatment, as well as the proportion of those in need who are receiving it. However, with few exceptions, no comprehensive country data are available on how well national programs are faring in patient retention on ART.

Keeping patients under ART places broader and perhaps more exacting demands on nascent national HIV care programs, compared with the already difficult task of identifying individual patients needing ART. Thus, incorporating quality indicators into ART programs would provide measurable parameters for the effectiveness of these projects and help optimize health outcomes. Simplified, systematic, and standardized means of assessing ART programs using core indicators beyond total number of patients started on therapy should be agreed upon, including survival, retention, frequency and extent of treatment interruption (particularly as related to pharmacy stockouts), adherence, and some measure of adequate human resources (eg, number of full-time equivalent ART provider-days per 1,000 ART patients in a given catchment area). These indicators can drive methodical efforts to improve the quality of care in donor-funded scale-up of ART programs.

In implementing HIV care and ART delivery, MSF directly experiences the difficulties in providing chronic treatment to often very sick patients, and strives, not always with immediate success, to adjust strategies over time to provide quality care. In referral areas to our programs where national efforts are ongoing to scale-up and decentralize ART services, MSF has also witnessed the problematic issues affecting continuity and quality of care in some donor-supported clinics. In some cases, MSF is asked to step in to avoid treatment interruptions or program slowdown. For example, in Uganda at the Arua Regional Referral Hospital, MSF currently treats >4,500 patients with ART in a program started in 2002. MSF began supporting peripheral health centers accredited by the national program to provide ART, so as to help increase access to treatment within the region and to ease overcrowding in the Arua clinic, which currently cares for >7,000 HIV-positive patients. The MSF team was frequently required to refill ministry of health (MOH) ARV stockouts due to health center ARV orders not placed or done so too late, inadequate health center pharmacy management, delays in national pharmacy ARV delivery, national pharmacy stockouts, and logistical difficulties in transport (eg, no truck, no fuel).

MSF staff also needed to coordinate drug order transmissions to the central supply managers, give direct patient care in place of absent or overworked health staff, provide supplemental training, and transport patient blood samples to the regional hospital for CD4 testing to avoid patients having to bear the cost and time of traveling to the regional hospital laboratory. In addition, only one clinic had adequately collected data to measure patient retention, with mortality and loss to follow-up at 12 months of 19% and 8%, respectively. The combined proportion of patient loss (27%) for this 12-month period (April 2006-April 2007) was similar to the 30% growth of the overall cohort in the subsequent 12 months (May 2007-May 2008).

In two MSF HIV clinics in Mozambique (Maputo and Lichinga), >5,000 patients are on ART. MSF is in the process of referring these patients from central sites to regional MOH clinics, which receive funding from the Global Fund and PEPFAR. MSF’s decentralization teams provide supervision of MOH staff at these clinics, but several barriers to uninterrupted and quality care are present. Large gaps in HRH cause delays in putting severely ill patients rapidly on treatment, and waiting times discourage regular clinic visits. Recurrent and prolonged drug stockouts are observed for ARVs and OI drugs, where MSF is often asked to supply buffer stocks of drugs. This example shows how such clinics are working hard to provide good care but continue to require further support and resources. Monitoring of certain indicators of patient treatment, retention, and outcomes is thus crucial, both to ensure quality care and to intervene when an indicator worsens.

MSF’s experience illustrates the emerging challenge of quality and continuity in the provision of care for the 3 million people started on ART in lower- and middle-income contexts. Increases in the numbers of patients on ART and clinics providing HIV/AIDS care may hide important programmatic and support deficits like the ones described here. This highlights the importance of balancing the strategies of scale-up/decentralization and the reality in the field.

While attempts to bring together a unified set of indicators have been made (by UNAIDS Monitoring and Evaluation Reference Group, for example), for reporting to the UN, WHO, Global Fund, and PEPFAR, most individual country M&E systems remain underdeveloped and under-resourced. They often do not have input from all local implementing agencies and lack data quality assurance tools and oversight. Governments, multilateral agencies, and funders should make a concerted effort to define and incorporate a simplified framework of ART program quality indicators that are easy to collect for systematic analysis. This analysis should then be used to address any program deficiencies in strategy or resourcing, in a timely fashion to better ensure that scale-up means not only starting patients on ART but keeping them on it.

ART program quality issues and examples of indicators (some are already in use while others have been proposed but not yet widely adopted):

• Patient access
• Estimates of ART coverage: proportion actively on ART (not only “ever started”) of those in need (by age group and gender)
• Retention in pre-ART care: proportions of mortality and loss to follow-up over time (eg, 12, 24, 36 months) by year of HIV program entry
• Program effectiveness
• Proportion/number of patients continuing to receive ART by age group over time (eg, 12, 24, 36 months) by year of ART initiation
• Program attrition rates (mortality, loss of follow-up) using standardized outcome definitions (eg, such as those used by DOTS TB programs)
• Program support
• Proportion of clinics with pharmacy stockouts of ARVs and key OI drugs >1 week per annum
• Adequacy of available ART care providers and clinic consultation days (such as number of ART health providers per 1,000 patients on ART and/or mean number of patient consultations per provider per day)