Perceived quality of primary healthcare post-National Health Insurance pilot implementation

Background Contracting private medical practitioners for the National Health Insurance pilot project in 2012 by the National Department of Health in South Africa was envisaged to reduce workload at referral district hospitals by reducing self-referral by clients as the perceived quality of care at the primary healthcare level improves. Aim To describe the effect of contracting private medical practitioners at primary healthcare facilities on the self-referral rate of clients at district hospitals as a proxy for perceived quality of care in a National Health Insurance pilot district. Setting The study was set in Tshwane National Health Insurance pilot district compared to Ekurhuleni district. Methods We compared findings before and after implementing the National Health Insurance private medical practitioners contracting between a pilot and a non-pilot district. A quasi-experimental ecological study design was used to compare district hospital outpatient department indicators of clients follow-up, self-referral, self-referral rate and referred in the two districts from June 2012 to May 2014 using single and controlled interrupted time-series analyses. Results Controlled interrupted time series analysis found decreases in self-referral rate (−1.8 [−2.2, −1.1] [p < 0.0001]) and the initial trend of headcounts of self-referral (−516 [−969, −66] [p = 0.0260]), but an increase in headcounts of referred clients (1293 [77, 2508] [p = 0.0376]) in the pilot compared with the non-pilot district. Conclusion We concluded that the implementation of contracting private medical practitioners in primary healthcare facilities might have resulted in an improved perceived quality of care at primary health care facilities. However, the higher number of outpatient department headcounts for follow-up and the increase in referred cases in the pilot district would need to be investigated.


Introduction
Understanding the effects of the perceived quality of primary healthcare (PHC) is critical to the attainment of universal health coverage (UHC) globally (World Health Organization 2019). In 2019, the United Nations (UN) reaffirmed UHC with strong health systems based on the values of PHC for the provision of quality healthcare as a goal for member countries to attain by 2030 (United Nations 2019). Poor quality of health service has been recognised as a significant problem in developing countries (Chou 2019). In South Africa, structural challenges with the public PHC system have contributed to a perception of poor quality of services, which include a shortage of staff, poor staff attitudes, long waiting times, low standards of cleanliness, drug stock-outs, poor infection control and inadequate security for both staff and clients and the nonavailability of integrated patient-level health information system for data collection and reporting (National Department of Health, South Africa [NDoH] 2015). All this is happening at the time when the country is experiencing a convergence of epidemiological challenges as a result of human immunodeficiency virus or acquired immunodeficiency syndrome (HIV or AIDS) and antiretroviral treatment, non-communicable diseases (NCDs) such as hypertension, diabetes mellitus, asthma, cancers, cardiac diseases, mental health problems and other lifestylerelated diseases and complex social and behavioural changes and injuries (NDoH South Africa 2015). The impact of these factors has been overutilisation of outpatient departments (OPD) of district hospitals as clients bypass PHC facilities to receive care at a higher level because of shortage or non-existence of medical doctors at the PHC level (Mojaki et al. 2011(Mojaki et al. :2011.
Background: Contracting private medical practitioners for the National Health Insurance pilot project in 2012 by the National Department of Health in South Africa was envisaged to reduce workload at referral district hospitals by reducing self-referral by clients as the perceived quality of care at the primary healthcare level improves.

Aim:
To describe the effect of contracting private medical practitioners at primary healthcare facilities on the self-referral rate of clients at district hospitals as a proxy for perceived quality of care in a National Health Insurance pilot district.
The National Health Insurance (NHI) pilot project implemented in Tshwane and 10 other districts in South Africa from 2012 as part of UHC, focusing on various health system strengthening interventions at the PHC level (Naidoo 2012:149-150), was envisaged to address this by contracting private medical practitioners (MPs) to provide services in PHC clinics (Moosa 2014:155-156).
There are several mechanisms through which MPs were anticipated to improve the quality of PHC services. It was envisaged that MPs would, apart from seeing PHC clients, strengthen clinical governance, reduce waiting times, do infection control inspections, drug stock checks, file reviews, nurse mentoring and training and conduct morbidity and mortality reviews (The Republic of South Africa, Department of Planning Monitoring & Evaluation 2017). The outcome of this would be to not only take-on and manage complicated cases at the PHC level beyond the capacity of nurse practitioners (NPs) but also minimise referrals to higher levels. Thus, it was hypothesised that the presence of privately contracted MPs in the PHC facilities would lead to improvement of care at the PHC level, resulting in a reduction in workload at the higher levels of care (Genesis Analytics 2019). The mechanism of this would be through MPs reducing secondary care activity by various means, including restricting referral criteria, supporting community-based care models, investing in preventative healthcare and promoting services to prevent readmissions (Department of Health 2010).
Others have hypothesised that perceived improved quality of care at PHC facilities would lead to high volumes of clients and that MPs would have an increased workload, to which they would respond by increasing referrals to the next level of care (Groenewegen & Hutten 1991:1111-1119. However, studies have shown that the increase in the utilisation of PHC services in NHI pilot districts cannot be attributed only to the contracting of private MPs (Mukudu et al. 2020). This raises questions about the appropriate measure of the quality of care in view of simultaneous implementation of other aspects of PHC re-engineering and healthcare reforms such as ward-based outreach teams (WBOTs), district clinical specialist teams (DCSTs), centralised chronic medicine dispensing and distribution (CCMDD), Ideal Clinic Realisation and Integrated School Health Programme (ISHP) (Naledi, Barron & Schneider 2011:17-28).
Definitions of quality of healthcare are varied and contested (Campbell, Roland & Buetow 2000:1611-1625, just as quality measures. However, one generally accepted definition is that it is the 'degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge' (Catumbela et al. 2013:236). By this definition, individuals' and populations' perceptions of quality are likely to differ, making quality measurement a contested issue because of the different approaches and models used for assessment. One essential requirement for a better quality of care is universal access. This, notwithstanding, Donabedian (2005) has stated that quality of healthcare can be measured from three components: structure, process and outcomes, which refers to the state of health or events that follow care and that may be affected by healthcare. However, the outcome of care, such as perceived quality, may not only be a component but a consequence of healthcare (Campbell et al. 2000(Campbell et al. :1611(Campbell et al. -1625. Quality of care also has an impact on the implementation fidelity of an intervention and affects the outcome (Carroll et al. 2007:1-9).
Health indicators are used as a measure of outcome by the District Health Information System (DHIS) of the NDoH. One of the indicators routinely collected at hospital OPD in the DHIS is the headcounts and rates of clients not-referred (selfreferral) (NDoH 2011). Several studies in the United States of America (Liu et al. 2008:124-130) and the South African provinces of KwaZulu-Natal (Pillay & Mahomed 2019:33), Limpopo (Visser et al. 2015:333-336) and Western Cape (Becker et al. 2012:800-801), and in the district of Tshwane (Masango-Makgobela, Govender & Ndimande 2020:1-7), have shown a directly proportional relationship between selfreferrals to higher levels of care and a perception of poor quality of services at lower levels of healthcare (PHC). Other related indicators at OPD are headcounts of clients referred and follow-up (NDoH 2011).
Measuring perceived quality in healthcare is important as it differentiates effective coverage from crude intervention coverage (Ng et al. 2014:9). The measurement of the perceived quality of care of a health service will help to inform the implementation of NHI in many areas, such as benchmarking and measurement of programme implementation. The perceived quality of a UHC is crucial because it bridges the gap between health system building blocks and the desired health outcomes (World Health Organization 2010). It will also help to prioritise patients' interest and determine the perceived appropriateness of the care they receive. These data will be imperative in informing legislation and policies to govern clients' referral from PHC to higher levels of care, suggest possible structural changes to the district health system approach and prioritise resources and capacity building. We thus set out to determine the effect of private MPs contracting as part of the NHI pilot project on the headcounts and rates of clients not referred (self-referral) at district hospitals as a proxy for perceived quality of PHC services (Visser et al. 2015:333-336). We used controlled interrupted time series analysis (CITSA) in comparing headcounts and rates of self-referral before and after implementation of the programme. We hypothesise that private MPs' placement in PHC facilities would improve the perceived quality of care, leading to a reduction in onward self-referrals. To the best of our knowledge, this is the first study to determine the perceived quality of care at the PHC level using secondary data on effect at higher levels of care.

Study design
We used a quasi-experimental study design by CITSA using DHIS monthly reports (Anaby et al. 2014:457-470). This design is recommended for evaluating a populationlevel intervention because it considers pre-existing secular trends and that there is a clear differentiation of the pre-intervention and post-intervention periods (Bernal, Cummins & Gasparrini 2017:348-355).

Study setting
This study was carried out in district hospitals of the Tshwane NHI pilot district, Gauteng province, South Africa, which has the largest and most diverse population in terms of socio-economic status of all the NHI pilot districts (NHI Pilot District Profiles|Health -e 2019a). The district has a population of more than 2 921 488 people receiving PHC services from 68 facilities (facility to population ratio of 1:36 980) (Health Systems Trust 2019). We compared the findings with those in a non-NHI district, Ekurhuleni, which has a higher population of 3 178 470 people with 90 PHC facilities (facility to population ratio of 1:35 316) (Metropolitan Municipality|Statistics South Africa 2019a). The selection of Ekurhuleni as a comparison district was based on the proximity of the two districts and similarities in demographic profiles (Metropolitan Municipality|Statistics South Africa 2019b). Being under the same provincial governance, a uniformity of implementation of health programmes was assumed.

Variables and outcome measurements
We measured and compared the selected OPD indicators as they met the criteria of reflecting intended and unintended outcomes and the ability to change relatively quickly after implementing an intervention or after a clearly defined lag (Bernal et al. 2017:348-355). It was anticipated that the improvement of healthcare services in PHC facilities would offset workload in OPD clinics based at district hospitals (Nanyonjo et al. 2015:1-10). The primary outcome variable was the rate of new OPD clients not referred by a PHC facility accessing services at a district hospital OPD (self-referral rate) and the secondary outcome was the headcounts of these self-referred clients because of difference in population size between the two districts. 'Self-referral' in the study was defined as the sum of new clients attending a general or specialist outpatient clinic without a referral letter from a PHC facility or a doctor. The measurement was used to monitor utilisation trends of PHC clients' bypassing clinics and community health centres (CHCs) and the effect of PHC re-engineering on OPD utilisation. The self-referral rate was calculated using self-referred clients as a proportion of the total OPD new clients. The target for both is lower numbers, as an indication of clients entering the health system at the appropriate level of care (South African NDoH 2020). Other variables collected were 'follow-up', defined as the sum of clients attending a general or specialist outpatient clinic for follow-up care, and 'referred', defined as the sum of new clients attending a general or specialist outpatient clinic with a referral letter from a PHC facility or a doctor. The complete list of variables, definitions, use, impact model and mechanism of the impact model are shown in Table 1.

Data quality and management
District Health Information System monthly reports from June 2012 to May 2014 for OPD headcounts of 'follow-up', 'referred', 'self-referral' and self-referral rate in Tshwane and Ekurhuleni districts were collated. Each district hospital collected data on the clients attending outpatient services and sent these to Health Information Management (HIM) officers who create electronic formats (Excel). The data in the DHIS were complete as the values for the elements were reported for every month for the period of the study.

Time limits and selected data points
The unit measure for selected data points was months as per DHIS reporting. A total of 24 time periods with 12 before and 12 after implementation of MPs contracting were selected. The minimum time period required for interrupted time series analysis (ITSA) is 10 before and 10 after implementation of a programme to have at least 80% power. The selection can detect a change level of at least five standard deviations of the pre-data if the autocorrelation is > 0.4 (Kontopantelis et al. 2015:1-4

Statistical analysis
We compared population characteristics from 2010 to 2014 between Tshwane and Ekurhuleni districts using data from Statistics South Africa (Metropolitan Municipality|Statistics South Africa 2019b). The District Health Information System data for OPD indicators were then collated and a comparison of the means (average) of pre-and post-NHI indicators for the two districts was made. On further analytical comparison, we first did a single group ITSA of selected OPD secondary data elements from the DHIS (Department of Health 2011) to project the pre-intervention trend for the post-intervention measurement as a counterfactual for each district separately. We estimated regression coefficients by the ordinary least squares method, producing Newey-West standard errors. Regression diagnostics were tested using the Cumby-Huisinga test on the error distribution to test for residual autocorrelation, which was adjusted for by Prais-Winsten regression. This was carried out to determine whether there was a statistically significant difference between the pre-and post-intervention slopes outside the intervention period in both districts. The series of monthly counts were assumed to follow a Gaussian distribution, which is not affected by overdispersion (Bernal et al. 2017:348-355). Seasonality was accounted for using dummy variables to define each month of the year as adjustment factors in the analysis.
The formula used for this ITSA was as follows: where: • Y t is aggregate OPD headcounts measured at t equally spaced time points. • T t is the time elapsed since the beginning of the intervention. • X t (indicator) represents the intervention. • X t T t is an interaction term between OPD headcounts. • e t is the random error.
• β 0 is the intercept of the line on the vertical axis (initial level of the OPD headcounts). • β 1 is the slope or trajectory of OPD headcounts before the intervention. • β 2 is the change in the level of OPD headcounts immediately after the introduction of the intervention. • β 3 is the difference between the pre-and post-intervention slopes. Therefore, significant values of p at β 2 indicate an immediate intervention effect and significant values of p at β 3 indicate an intervention effect over time.
We then used the Prais-Winsten Auto-Regression (AR)(1) model of regression to perform a multiple group or CITSA. This allowed us to control for both the pre-intervention trends in the outcomes and for potential confounding events that would have affected both the control and the study groups. We modelled the association as a slope change rather than as an immediate level change (sudden increase in headcounts or rates) because the selected outcomes were likely to change gradually as MPs were contracted. Multiple group ITSA was performed according to Linden's description (Linden 2015). We estimated differences in coefficients according to the following formula: where S is a dummy variable for assignment to treatment or control.
Stata statistical software package, version 16 (StataCorp, TX, USA) was used to perform all the statistical analyses.  Table 2 shows that from 2012 to 2014, the population growth rate and the percentage with medical insurance were slightly higher in the pilot district than in the non-pilot district.

Results
A pre-to post-NHI comparison of all measured OPD indicators ( Table 3) showed increases that were more pronounced in Ekurhuleni district than in Tshwane district, except for the OPD headcounts not referred, which was reduced. Paired t-test found a statistical difference for all variables in the two districts except for 'referred' in the non-pilot district before and after NHI MPs contracting.
In comparing the two districts using the independent t-test assuming unequal variances, there were statistical differences in 'referred', 'not referred' and 'follow-up' except for the selfreferral rate.

Discussion
Our findings were based on ITSA, which compared pre-to post-intervention level and trends of OPD headcounts in a pilot and a non-pilot district separately and CITSA that compared OPD headcounts for follow-up, not referred new, not referred rate and referred new and trends between the two districts before and after NHI's MPs contracting. Analysis of our findings using ITSA shows that after the NHI's MPs contracting implementation, there was an increase in the trend for self-referral in both districts and for referred in the pilot district. However, there were decreases in both headcounts/rate and the trends in the pilot district and non-pilot district for self-referral rate. In comparing using CITSA, there were statistically significant differences in level of OPD headcounts follow-up and trends of OPD headcounts not referred new, referred new and self-referral rate between the two districts.
In comparing before and after MPs contracting using ITSA, there were statistically significant monthly increases in 'selfreferral' of about 3400 clients in Tshwane district. Similarly, in Ekurhuleni district the monthly increase was about 5400. This confirms previous findings of bypassing of PHC services, potentially leading to overburdened district hospitals and underutilisation of PHC services because of self-referral, which can lead to wastage and poor allocation of resources (Masango-Makgobela et al. 2020:1-7). The differential findings between the two districts were supported by findings of monthly decreases for both self-referral rate and trend in the post-intervention period of 1.7 and 1.5 in the rate, respectively, in the pilot, but not in the non-pilot district. The findings were also confirmed by the CITSA findings of a difference in the monthly decrease in the post-intervention trend for the selfreferral rate of 1.8 and a monthly increase of 1293 in the postintervention trend for 'referred' in the pilot district compared with the non-pilot district. These CITSA findings of a reduction in self-referral and self-referral rate could be explained by the implementation of MPs contracting in the pilot district and are in line with findings in countries that have implemented some form of UHC with strong PHC, such as the United Kingdom, and contrast to those without, such as the United States of America (Forrest 2003:692-695). The mechanism of action could be improved perception of the quality of healthcare at the PHC level because of the presence of NHI contracted MPs, which has resulted in some clients not seeking primary care in hospitals as per the hypothesis of our study. Conversely, the increase in OPD headcounts referred new could be explained by MP's referring clients to a higher level of care in response to increased workload at PHC level. There was an absence of an immediate increase but rather a gradual one post-intervention as evidenced by an absence of a level change in 'self-referral rate' but an increase in its post-intervention trend only (Groenewegen & Hutten 1991:1111-1119, or an identified need for a higher level care by MPs in clients historically treated by NPs.  was implemented by providing insurance for citizens to access care at public and private providers. Thus, in a study carried out in 2012 (7 years after initiation of NHIS), the UHC programme had led to increased utilisation of both outpatient and inpatient services. This study did not disaggregate PHC to OPD services even if both clinics and district hospitals were included in the study (Dalinjong et al. 2017:1-10). Similarly, in Rwanda, even if the Mutual Health Insurance (MHI) programme was scaled up in 2005, by 2007, an increase in the utilisation of health services was already noted, which coincided with a stabilisation and reduction in the HIV disease burden in the country (Kayirangwa et al. 2006:27-31).
The statistically significant CITSA difference in OPD followup headcounts between the two districts of more than 24 000 shows that despite having a lower population and higher medical insurance coverage, there is higher burden of disease on the district hospitals in the pilot district compared with the non-pilot district. This could be explained by a non-optimal down-referral mechanism from the district hospital to the PHC facilities in the pilot district (Hugo et al. 2020:e1-e10).
The findings of this study and those found in Singapore that also had private medical providers contracting as part of UHC interventions (Tan, Lee & Tan 2019:18-23) illustrate important lessons for the NHI in South Africa. There is a need to develop the capacity of PHC facilities and create organisational structures that would enhance coordination of activities between them and district hospitals by improving down-referral and supportive supervision of the PHC team by clinical specialists. This finding is crucial in addressing the immediate significant MP shortages and capacity at the PHC level and realising the broader vision of a single integrated system as envisaged by NHI.
There are several limitations to the interpretation of our findings. Firstly, the use of secondary data such as DHIS, which were not intended for the specified research question, may be incomplete and the interpretation limited. For example, we are not able to conclusively determine that the high number of follow-up headcounts in the pilot district is because of non-optimal down-referrals as there is no data collected for this indicator both at the PHC and OPD levels. However, the pre-selected data elements used in this study were complete in that they were reported for every month of the period under consideration. Secondly, the proximity of the two districts being compared could have led to population contamination, substitution and migration effects. However, this effect could also be considered negligible as the population growth in both districts was similar.  http://www.hsag.co.za Open Access

Conclusion
Our findings suggest that the implementation of private MPs contracting as part of the NHI pilot project in PHC facilities in the Tshwane district may have resulted in the decrease in self-referrals at district hospitals. Thus, it could have resulted from an improvement in the perceived quality of healthcare at the PHC level. Despite a general reduction in the selected OPD headcounts, the follow-up at the district hospital level was high in the Tshwane district, suggesting an increased workload for staff at district hospitals after private MPs were contracted to work at PHC facilities.

Recommendations
We recommend that with the implementation of NHI, the down-referral systems from district hospitals to PHC facilities and their capacity in managing conditions dictated by the epidemiological transition should be improved.