Prevention of Mother-to-Child Transmission of HIV data completeness and accuracy assessment in health facilities of the Nkangala District

How to cite this article: Mlambo, M.G., Peltzer, K. & Koivu, A., 2014, ‘Prevention of Mother-to-Child Transmission of HIV data completeness and accuracy assessment in health facilities of the Nkangala District’, Health SA Gesondheid 19(1), Art. #774, 8 pages. http:// dx.doi.org/10.4102/hsag. v19i1.774 Background: Even though significant progress has been made in the roll-out and quality of the prevention of mother-to-child transmission of HIV (PMTCT) services in South Africa, the quality of patient data recording remains a challenge.


Introduction
This study aimed to assess prevention of mother-to-child transmission of HIV (PMTCT) data completeness and data accuracy at the primary healthcare level to district level in order to assist with the improvement of the PMTCT data recording in the Nkangala district of Mpumalanga Province, South Africa.This will be done by assessing the completeness and accuracy of data values recorded in the seven selected national PMTCT data elements.We assessed antenatal, postnatal, maternity and infant PMTCT data elements.

Background and rationale
The data coming from African countries do not reflect accurately the issues affecting this region (Ndira, Rosenberger & Wetter 2008:489; World Health Organization, Regional Office for Africa [WHO/AFRO] 2002).According to WHO/AFRO, the data provided by routine health systems in the African Region are generally not always available for most countries; and even when they are, they are not always comprehensive, complete or up to date (Ndira et al. 2008:489;WHO/AFRO 2002).The information flows around the prevention of mother-to-child transmission (PMTCT) of HIV in South Africa make no exception.In addition to the work done around the data quality issues in the PMTCT area (Mate et al. 2009;Mphatswe et al. 2012:176), studies from South Africa (Jacucci 2005) have raised questions about data quality regarding the South African District Health Information System (DHIS).
The data quality issues should be considered in the context of the South African health system.South Africa has a history of disease-or condition-specific approach, though the health system can be considered comprehensive (Lush et al. 1999:771), with integrated tuberculosis (TB) and HIV programmes, including PMTCT of HIV (Uwimana et al. 2012:658).However, it has been argued (Uwimana et al. 2012:658) that these programmes are not yet integrated fully into the health system functions as outlined in the WHO framework on health systems (WHO 2000) and other conceptual frameworks for integration analysis (Atun et al. 2010:104;Cairncross, Peries & Cutts 1997:20).Hence, the varying quality in collection, reporting and use of data should be discussed with reference to a limited integration of largely vertically-delivered programmes to other health system functions.
Research done in other resource-limited settings has indicated that the quality of data reporting is inconsistent (Freund & Kalumba 1985:137;Kumar 1993:305;Mavimbe, Braa & Bjune 2005:108).Weaknesses that contribute to compromised data quality, as well as suboptimal functioning of the health information system, are shown to stem from data incompleteness and untimely reporting (Chaulagai et al. 2005:375;Kimaro & Twaakyondo 2005:189), poor documentation of services provided within the facilities and overly-onerous reporting requirements (Brown 2007;Makombe et al. 2008:310;Mate et al. 2009).Furthermore, as evidenced by earlier research, the larger the number of data elements to be reported upon, the poorer the quality of the data (Stoops, Williamson & Braa 2003).
Numerous studies have shown that there is a need for improved information systems for monitoring healthcare services that will assist with better service provision (AbouZahr & Boerma 2005:578;Chan et al. 2010;Nash et al. 2009:S58;Rommelmann et al. 2005:569;Shaw 2005:632).A patient register-based system has been utilised to a large extent by South African primary healthcare facilities in order to record patient data and it continues to experience challenges which point to many inaccuracies.Such has been the case in resource-limited settings whereby indicators for monitoring and evaluation are often hand-tallied from paperbased sources and are reported to people at different levels of the programmatic chain (Joint United Nations Programme on HIV/AIDS [UNAIDS] 2008;WHO 2005).Similar to other countries (AbouZahr & Boerma 2005:578), where routine health information systems have been introduced as a way of improving healthcare system performance, South Africa has also introduced a health information system (HIS) which has brought about some improvements in patient data quality sheets (Garrib et al. 2008:549) and data recorded on the DHIS.

Aim of the study
This study aimed to assess the PMTCT data completeness and data accuracy of the primary healthcare level to district level in order to assist with the improvement of the PMTCT data recording in the Nkangala district of Mpumalanga Province, South Africa.This was done by assessing completeness and accuracy of data values recorded in the seven selected national PMTCT data elements.We assessed antenatal, postnatal, maternity and infant PMTCT data elements.

Problem statement
Although improvements have been made in the coverage and quality of PMTCT services in South Africa, the quality of patient data recording remains a challenge, as evidenced by recent research (Mate et al. 2009;Mphatswe et al. 2012:176).The DHIS was introduced as a routine system to track health service delivery in the public health sector of South Africa (Heywood & Rohde 2001) and to support the collection of data that have been used to calculate the indicators by site, subdistrict, district and province.However, several challenges remain which contribute to inconsistent data quality and limited utilisation of available data and information and data flow bottlenecks (Department of Health 2011).On a more detailed level, the challenges also include data recording discrepancies at facility level on case registers and data being recorded on monthly summary.
Therefore, collecting accurate information on the PMTCT data elements would not only strengthen the existing PMTCT services, but would also broaden areas of further intervention as part of comprehensive patient care.To make evidencebased decisions, high-quality data are required at all levels of the healthcare system.At the facility level, the information is used for care delivery and the details about prior events should be recognised and incorporated into current care decisions (Reid & Wagner 2008:987).Furthermore, data collected at facility level can be used to make decisions and improve the quality of healthcare services (Garrib et al. 2008:551-552).Indeed, data analysis is required for advocating, designing, planning and evaluating public health actions (Sánchez, Lloyd-Smith & Getz 2010;WHO 2004) and to inform health policy and resource allocation (WHO 2003).The global interests in the monitoring of development, as illustrated by the Millennium Development Goals (MDGs), generate pressure for quality and timely data in order to demonstrate country progress (Boerma & Stansfield 2007:779).

Study setting
The study was conducted in the Nkangala district (one of the three districts of the Mpumalanga Province) in order to provide technical support and to strengthen PMTCT service implementation at all the subdistricts that fall under its jurisdiction.A similar assessment was conducted in another district (Gert Sibande district) and it also pointed to serious problems with regard to data recording.The study complemented the existing national PMTCT programme and heeds the international call to make women and children a priority in the fight against the epidemic.The PMTCT cascade in the Nkangala district includes antenatal uptake and HIV testing during pregnancy, provision of ART to pregnant women with HIV, safe childbirth practices, safe infant feeding, infant HIV testing and postnatal-care ARV adherence.

Design
This is a retrospective record review study of PMTCT data elements in Nkangala health facilities.The study was conducted between February 2010 and April 2010 and PMTCT data collected on indicators was for the period of August 2009 and January 2010.As part of the PMTCT record review, data for a period of six months (August 2009 -January 2010) were collected from the case registers in the healthcare facilities, using a tool that was developed with data elements for the PMTCT programme.A total of seven PMTCT data elements on antenatal and postnatal care, maternity and infant indicators were assessed.

Sampling criteria
For the baseline assessment, all 80 PMTCT sites in one health district (Nkangala) were identified.Of these 80 sites, four could not be assessed because the PMTCT site manager was unavailable, even after several visits were made to those healthcare facilities.An additional four sites did not agree to provide us with access to assess data recorded in case registers and monthly summary sheets.No facility was excluded from the facility assessments in Nkangala District.Thus, the final sample comprised 72 facilities (53 clinics, 12 community health centers and seven regional and/or district hospitals).

Data collection methods
With regard to the existing data management system, patient data in the Nkangala district healthcare facilities are primarily collected using a paper-based record system.
There are a number of registers that are used to collect patient PMTCT-related information, including the PMTCT register, tally sheet register, drug register, PCR register, HIV testing register and CD4 count register.The data flow from the facility to district level involves the collation of patient data from various registers and transferring it to the case registers.Data from case registers are then summarised on the monthly summary sheet, which is then sent to DHIS.Patient data are handwritten and the information is captured on the computer at district level.
In addition, separate books were used to record useful patient information for the clinic which was not necessarily submitted to the district for recording on the DHIS system, since this information was not regarded as providing indicators for PMTCT.Patient records were also not stored at a central site, which means that in preparation for data to be submitted to the district, the site manager had to take all the different registers from different points in order to record data for that particular month.In most of the healthcare facilities, the primary healthcare (PHC) site manager is the one responsible for collating all the information from different registers and sending it to the DHIS, with the exception of few facilities where there are clerks dedicated to do the data recording.For the current study, we asked a professional nurse in charge of the PMTCT programme at each site to provide the researchers with all the registers that had PMTCT data and monthly summary sheets.As part of the staff awareness regarding the effectiveness of the PMTCT programme, most of the healthcare facilities had monthly statistics for the PMTCT indicators displayed (using graphs) on the healthcare facility notice boards.A total of five Human Sciences Research Council (HSRC) researchers reviewed the case registers in each of the 72 facilities.

Assessment of data completeness
Data completeness was defined as the extent to which observed records were filled in with data values (Simba & Mwangu 2005).The data completeness process involved checking the proportion of data values recorded on the seven PMTCT data elements to see if it was reflecting on both the PMTCT registers (case registers and DHIS).A data element was considered complete if a value was recorded on both the case registers the DHIS for each of the six months of the study period.Although there are three levels of data recording (case registers, monthly summary sheet, DHIS) the acceptable limit for data completeness for this study was based on having recorded data values in both case registers and DHIS.
To assess the completeness of the data, we reviewed the seven PMTCT data elements, namely:

Antenatal and postnatal indicators:
1. antenatal care (ANC) clients pre-test counselled for HIV at first visit 2. ANC clients tested positive for HIV at first test (new) 3. HIV-positive ANC clients tested for CD4 count 4. ANC clients initiated on Azidothymidine (AZT) at 28 weeks or later.

Infant indicators:
6. babies given AZT 7. polymerase chain reaction (PCR) test done on babies born to HIV-positive mothers at six weeks or later,that were reported routinely to the DHIS over a six-month period (August 2009 -January 2010).
In order to ascertain the full completeness level, the selected six months were based on the fact that data were already submitted to the DHIS, so they would not be missing from the case registers.Therefore, this means that if data on the case registers were already submitted to the DHIS using monthly summary sheets, then they are complete.We excluded the following indicators from the analysis because of unavailability of data values recorded on them at the time of the study: NVP given to HIV-positive women at the ANC; babies given NVP, HIV-positive mother; PCR test results given.A data element was considered complete if a value was recorded in the case registers and the DHIS for each of the six months of the study period.

Assessment of data accuracy
Data accuracy was defined as the extent of variation between actual and recorded figures in the PMTCT registers (Simba & Mwangu 2005 ).Assessment of data accuracy involved comparing data values recorded in the DHIS with data values obtained from the individual facility case registers (Mphatswe et al. 2012:176).Using the data elements, patient counts were done by the researcher and a professional nurse for the months that were being assessed (August 2009 -January 2010).We considered a data element to be accurate if the value of the data recorded in the DHIS system was within 10% of the value recorded in the case registers.The 10% accuracy cut-off point was based on previous studies regarding data accuracy levels (Garrib et al. 2008:549;Mate et al. 2009), although for the current study the 10% was calculated by comparing values in the case registers and the DHIS instead of focusing on DHIS data values only.In cases where data were missing from either the case registers or DHIS, we excluded these from the accuracy analysis of the data (Mate et al. 2009).

Data analysis
Data obtained from the primary healthcare facilities and the DHIS were entered using the Software Package for the Social Sciences (SPSS) program, version 20 (released 2011, IBM Corp, Armonk, NY).Descriptive statistics were used to calculate the frequencies for the data recorded on each of the seven PMTCT data elements assessed.Data were analysed according to data completeness and data accuracy.Data analysis was stratified by type of facility for the ANC and maternity indicators.For instance, when analysing data on ANC indicators, we excluded the hospitals and, when analysing maternity indicators, we excluded the clinics from the analysis.The degree of accuracy between data reported in the DHIS and data obtained from individual facility registers was based on a 10% threshold.

Ethical considerations
Ethics approval was obtained from the HSRC ethics committee (Protocol REC 5/11/06/08) and the Mpumalanga Department of Health approved the study.Prior to facility visits, appointments were made with the district coordinator, who in turn liaised with the HIV/AIDS, STIs and TB (HAST) coordinators for all six subdistricts about the HSRC visit to the facilities.The HAST coordinators informed the PHC site managers of the various clinics about the PMTCT baseline assessment.Before conducting the study, the site managers or PMTCT nurses were given background information about the study for purposes of enhancing their understanding about the study and they were also requested to provide informed consent.The district information manager was also asked to provide DHIS data for the period of August 2009 -January 2010.The PMTCT record review took a minimum of two to three hours per site, depending on the ease of access to the records.The nature of the study was emphasised, making sure that the site managers understood that the study was not about their personal evaluation, but was rather an assessment of the PMTCT programme at large.

Data completeness
We assessed the seven PMTCT indicators and found that data completeness varied from one indicator to another.The results show that the data were only complete less than 25% of the time for most of the ANC indicators (with percentages of completeness ranging between 0.5% -44%) and for the hospitals, data were completed only 11% of the time.The data incompleteness was due to missing values in either the case registers or the DHIS.Of the seven data elements, the bestreported data elements, showing maximum completeness, were: 'ANC clients tested positive for HIV at first test (new)' (n = 171; 44%) and 'ANC clients pre-test counselled for HIV at first visit' (n = 158; 40.5%).The data elements with very minimal completeness were: 'babies given AZT' (n = 2; 0.5%) and 'PCR done on babies born to HIV-positive mothers at six weeks or later' (n = 11; 2.8%).Therefore, data were only completed between 0.5% and 44%, as is reflected in Figure1.

Data completeness by month
The  1).

Missing data
The results show that data were missing from case registers, monthly summary sheets and the DHIS between 30% and 99% of the time (Figure 2).The data elements with the highest missing data were: 'babies given AZT' (n = 389; 99%)', 'PCR done on babies born to HIV-positive mothers at six weeks or later' (n = 379; 97.1%) and 'ANC clients initiated on AZT at 28 weeks or later' (n = 329; 84.3%).

Data accuracy and inaccuracy
The results show that there were data elements which had accurate data values falling within 10% accuracy.The data accuracy in this case ranged from 0.5% to 44%.There were also data elements that showed values with > 10% deviation and these ranged from 0.5% to 17% (Figure 3).

Discussion
The results revealed that patient data recording in the Nkangala district remains a challenge because of the lack of accurate data recording by the healthcare professionals.
We found that the inaccurate data recording stems from the lack of a double-checking system for the data recorded on the monthly summary sheets before they are submitted to the district to be recorded in the DHIS.There are a number of registers that are used to collect patient PMTCT-related information including the PMTCT register, tally sheet register, drug register, PCR register, HIV testing register and CD4 count register.All of these registers cover the information that is recorded on the case registers and that is summarised on the monthly summary sheet which is then sent to DHIS.The results indicate that because of many PMTCT registers, there is non-verification of data that are being transferred from one register to another.When assessing data completeness and accuracy for the seven PMTCT data elements, we found that the best reported data element, showing maximum completeness and accuracy, was 'ANC clients tested positive for HIV at first test (new)'.Furthermore, the results show that, for the hospitals in the study, there was minimal completion of data.Our findings regarding the incompleteness and inaccuracy of the PMTCT data are in accordance to other studies conducted in South Africa (Mate et al. 2009;Mphatswe et al. 2012:176).This is worrying considering that the unreliability of data compromises the efforts to improve the healthcare systems that provide PMTCT interventions (Mate et al. 2009).The   current study concurs with findings from other studies (Garrib et al. 2008:549) which found that data validation to ensure data completeness and accuracy was limited.
We also found that the two data elements with minimal completeness were on infant indicators, namely, 'babies given AZT' and 'PCR test done on babies born to HIV-positive mothers at six weeks or later'.The lower data recording of values for the two data elements could be as a result of no service provision during the assessment period (August 2009 -January 2010).Most of the antenatal facilities were not providing AZT or even doing infant PCR testing.If they were providing that service, there was very little recording for the two data elements during our assessment period.However, it is worth noting that most of the healthcare facilities of the Mpumalanga Province are now giving AZT to the infants and they do infant PCR testing at six weeks.The study conducted by Sherman and Lilian (2011) showed improved PCR testing rates from 18.4% in 2008 to 39.3% in 2010.When assessing data completeness by month, the two data elements with the highest completeness were 'ANC clients tested positive for HIV at first test (new)' and 'ANC clients pre-test counselled for HIV at first visit', which do not show any significant difference when all six months' data are combined.
In this study we found that data recording inaccuracy was common at all levels (PHC to district).As with the reason for data incompleteness, data inaccuracy was also as a result of the over-recording of data values in the DHIS (by over 10% when compared with the data values recorded in the case registers and monthly summary sheets).We found that the over-recording of the values on data elements were mainly in the DHIS.Other studies also suggest that even though data are recorded in the DHIS, the quality is not good (Mphatswe et al. 2012:176).In order for patient data to be useful, they must be valid and accurate (WHO 2003).The results for this study show that patient data are clearly not accurate since DHIS data are collected from many sources and thus may not always be reliable.Another worrying factor is that we found that data were missing from case records, monthly spreadsheets and the DHIS, with the highest missing data element being on 'babies given AZT'.
In a study conducted in KwaZulu Natal, South Africa, of the total data values that had been collected, they found a mean of 2.5% missing data in the DHIS and healthcare facilities had not provided any explanation for the omission (Garrib et al. 2008:549).This suggests that patient data recording is a weakness which requires immediate intervention in order for the PMTCT service provision to improve significantly.We concur with the findings of other studies which point out that regular audit of health data could improve data recording significantly.A study conducted in South Africa found significant improvements in all key PMTCT indicators after PMTCT intervention, which included conducting a roadshow to orient staff on PMTCT protocol and monthly support visits by a senior experienced professional nurse in order to assess routine programme indicators (Doherty et al. 2009:406).
There is, therefore, a need to perform interventions aimed at improved data recording quality.Other studies have demonstrated that such interventions work in underresourced countries (Mphatswe et al. 2012:176).After an intervention that included training on data collection, monthly reviews of data and data audits at individual facilities, a study from South Africa found that data completeness in the studied antenatal care clinics improved from 25% (before the intervention) to 63% (after the intervention) and in the study hospitals, from 28% to 86%.The overall data accuracy improved from 35% to 65%.
Reviewing data quality and completeness could be done on a monthly basis during monthly district programme review meetings (Mphatswe et al. 2012:176).Other studies in underresourced countries also reported on interventions that could lead to the improvement of data quality and completeness.For instance, in Kenya, the onsite assessments and feedback on data completeness and validity were used to improve the quality of routine health information.As a result, the number of HIV tests recorded, as well as the number of HIV-positive patients, doubled (Otwombe et al. 2007:315).In some settings, data improvements were linked to clinic visits by supervisors  % of data recorded on DHS with <10% value (Huaman et al. 2009:16).Several studies (Berwick 2004(Berwick :1124 indicators is required at multiple levels of the healthcare system (site, regional and national levels) (Nash et al. 2009:S58).The WHO recommends the development of ongoing quality assessment plans which involve identifying important aspects of data collection (accuracy, validity, reliability, completeness and timeliness) and obtaining a threshold to determine acceptable error rate (WHO 2003).
It is not sufficient to focus only on interventions that improve data quality.In many countries there remains an inadequate appreciation and use of available information in planning and management of health services (Chaulagai et al. 2005:375).In other words, if the use of the collected PMTCT data elements is limited, it undermines any effort to improve the different aspects of data quality.Moreover, further research may be beneficial with regard to determining whether the local health professionals perceive the current data recording system adequate or to investigate if any perceived deficiencies in the data recording system may contribute to the atmosphere which, in turn, contributes to suboptimal data entry into the DHIS.Kawonga, Blaauw and Fonn (2012) argue that the key drivers of the silo approach are specific reporting requirements linked to ear-marked funding, politically-driven imperatives and mistrust of DHIS capacity.They conclude that the parallel systems that bypass the DHIS represent a missed opportunity to strengthen system-wide M&E capacity.Finally, they recommend the integration of HIV M&E into the health system.M&E functions such as these would mobilise earmarked HIV funding toward improving DHIS capacity so as to produce quality and timely HIV indicators (Kawonga et al. 2012).Hence, it should be explored to what extent the possibly parallel data requirements of the PMTCT programme or by the non-governmental organisations working in PMTCT may contribute to the data quality issues in the DHIS.Although we found a lot of data inaccuracy and incompleteness in our study, we did not provide any interventions because of funding limitations.The study recommendations were made available to the district.

Limitations of the study
Study limitations include that the study was conducted in only one district of the Mpumalanga Province, thus findings cannot be generalised to every district in South Africa.It cannot, therefore, be assumed that similar issues with data incompleteness and inaccuracy are present in other districts.

Recommendations
In this study, we found that patient data recording varies between different PMTCT data elements whereby antenatal indicators seem to be more recorded than maternity and infant indicators.There is a need for ongoing training on data recording at all levels, which support on DHIS is critical for it to function properly.

Conclusion
To maintain data quality, healthcare data must be appropriate, organised, timely, available, accurate and complete (Abdelhak et al. 1996).Data entry checks are also critical for accurate data reporting (WHO 2003).For this reason, having dedicated personnel in each facility would improve data recording quality significantly.

FIGURE 1 :
FIGURE 1: Percentage of prevention of mother-to-child transmission of HIV data completeness.
PCR test done on babies born to HIV-positive mothers at 6 weeks or later ANC clients initiated on AZT at 28 weeks or later HIV-positive ANC clients tested for CD4 count ANC clients pre-test counselled for HIV at 1st visit ANC clients tested positive for HIV at 1st test (new) of mother-to-child transmission of HIV; ANC, Antenatal Care; PCR, Polymerase chain reaction; AZT, Azidothymidine; CD4 Cluster of differentiation (CD4).

FIGURE 2 :
FIGURE 2: Percentage of missing data values on case registers, monthly summary sheet and District Health Information System.

FIGURE 3 :%
FIGURE 3: Percentage of data accuracy and inaccuracy by data element.

test counselled for HIV at 1st visit ANC clients tested positive for HIV at 1st test (new) HIV-positive ANC clients tested for CD4 count ANC clients initiated on AZT at 28 weeks or later Women receiving NVP in labour Babies given AZT PCR test done on babies born to HIV- positive mothers at 6 weeks or later
ANC, Antenatal Care; PCR, Polymerase chain reaction; AZT, Azidothymidine; CD4, Cluster of differentiation; NVP, Nevirapine.
; Center for Applied Research on Population and Development 2003) advocated for quality improvement interventions using a data-driven approach to ensure sustainable improvements in health programmes.Other studies indicate that in order to ensure acceptable completeness and accuracy of patient data, ongoing review of monitoring and evaluation (M&E)