Original Research

Developing and validating an appendectomy algorithm for use in a South African database

Johnelize Louw, Peter Nyasulu, Rene English, Kathryn Chu

Received: 01 July 2025; Accepted: 19 Sept. 2025; Published: 26 Nov. 2025

Copyright: © 2025. The Author(s). Licensee: AOSIS.
This work is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license (https://creativecommons.org/licenses/by/4.0/).

Abstract

Background: Identifying surgical patients through administrative and clinical data can inform the quality and demand for surgical care. In South Africa, a database exists that comprises data from the public health sector. However, algorithms are lacking to identify surgical procedures like appendectomy in these systems in our setting.

Aim: To develop and validate an appendectomy algorithm for use in a South African database.

Setting: Data from public hospitals in South Africa were the reference standard and comprised appendectomy and other general surgery procedure controls. The index test was the appendectomy algorithm developed and validated using the provincial database in the country.

Methods: A diagnostic test accuracy study was done. The algorithm was developed using four phases: exploration and selection, development, refinement and validation. Data analyses were performed using STATA version 18.

Results: The final algorithm comprised two procedures and nine diagnostic codes and reached a sensitivity of 91.3% and a specificity of 96%.

Conclusion: Our study is the first to validate an appendectomy algorithm in a low-and middle-income country setting. While not the first globally, it addresses a critical gap in the literature by demonstrating that robust, high-specificity algorithms can be developed in resource-constrained settings. Future research should focus on applying the algorithm to evaluate median delays in accessing care within the public health system.

Contribution: This study demonstrates that surgical procedure algorithms can be developed and validated with sufficient sensitivity and specificity using diagnostic and procedure codes for application in a low- and middle-income country setting.

Keywords: appendectomy; appendicitis; algorithm; health information system; procedure codes; diagnostic codes; healthcare.

Introduction

Nearly one-third of the global disease burden is attributable to surgical conditions. Globally, inadequate access to safe, timely and affordable surgical care contributes to four times more deaths than those from HIV or AIDS.^1,2 Despite being one of the most cost-effective health interventions, surgery remains under-researched and is often referred to as the ‘neglected stepchild’ of global health.³ In low- and middle-income countries (LMICs), data on the types and number of surgical procedures are scarce because of the absence of dedicated operative registries and comprehensive clinical or administrative databases.^4,5,6 Such systems are essential for research, quality improvement and resource planning and identifying specific patient groups within them requires validated algorithms. While algorithms have been developed for various health events and conditions,^{7,8,9,10,11,12,13,14,15,16,17} fewer exist for surgical procedures such as appendectomy.^5,18

Accurate coding of surgical conditions and procedures is essential for functions such as billing, epidemiological surveillance, and health system planning.¹⁹ Yet there are several challenges associated with coding, such as misdiagnoses, incomplete records, inconsistent discharge summaries, and financial incentives that could influence coding practices.^{4,20,21,22,23,24,25,26} Assigning codes is further complicated by the multiple possible codes that may apply to a single condition or procedure, depending on severity or treatment type.

In the Western Cape, South Africa, a provincial database was developed that consolidates data from various sources into one of the country’s most comprehensive provincial repositories. Algorithms have successfully been developed to identify patients with chronic health conditions such as HIV, hypertension, and diabetes mellitus in this health information system,^27,28,29,30 often using laboratory results and prescribed medications as key indicators. However, surgical conditions and procedures lack a distinct, longitudinal clinical signature, making them more challenging to identify.

Internationally, the study by Kleif et al. validated having a diagnosis of appendicitis and a registered appendix removal procedure code in a high-income country (HIC) with a national patient registry.⁵ Their findings showed that combining diagnostic and procedure codes improved the accuracy of identifying cases. Until now, this type of algorithm has not been validated in an LMIC context, where coding completeness, accuracy and data integration may be more variable.

Appendicitis is one of the most common time-sensitive surgical conditions globally and has a global incidence of 229.9 per 100 000 and a lifetime risk of 7%.^31,32 The standard treatment is appendectomy, which is the surgical removal of the appendix. This can be performed laparoscopically or through open surgery, with the choice influenced by disease severity, patient factors, surgeon skill and available resources. In uncomplicated acute appendicitis, conservative management with antibiotics may be considered as an alternative to surgery,³³ although the failure rate is still higher than appendectomy. Given the large number of complicated cases and the challenges in outpatient follow-up, this approach is less common in LMICs.

This study addresses a critical gap by developing and validating an appendectomy algorithm for use in an LMIC.

Research methods and design

Study design

A diagnostic test accuracy study was conducted using routinely collected patient data within the Western Cape public health system.

Development of the algorithm

Algorithm development and validation followed four phases: exploration and selection, algorithm development, refinement and validation (Figure 1). Secondary data were retrieved from an existing provincial database for 01 January 2019 – 31 July 2020 and divided into four datasets, each corresponding to a unique phase.

FIGURE 1: A summary of the study procedure that comprises four phases.

Data source and sampling

Provincial database

The index test, the appendectomy algorithm, was developed using data from the provincial database, the Provincial Health Data Centre (PHDC), and exist in the Western Cape. The PHDC was established to enhance service delivery in the Western Cape and is a central repository that stores information related to public health care facilities, patients and clinical data. Clinicom™, one of the primary sources of data for the PHDC, is based on the Cache™ hierarchical database and is searched daily for new patient entries, admission and discharge dates, procedure and diagnostic codes. Other sources of data are laboratory and pharmacy data and other databases that were created to help manage priority conditions.²⁰ Gaps in the database include patients’ histological data and any cause of death information.²⁰

Reference standard: Appendectomy cases and controls

The reference standard comprised appendectomy cases and other general surgery operation controls identified through searching the operative registries of four hospitals known to maintain electronic records. Patients who had a procedure that included the word ‘appendectomy’ were identified as cases. Each hospital’s clerical or clinical staff searched the general surgery operative registries. Variables included the patient’s name, surname, date of birth, medical record number, operation date and procedure code or description.

Data management and analysis

Microsoft^® Excel was used to manage the data. The reference standard was shared with the database data scientists as a password-protected Microsoft^® Excel file. Data scientists deidentified and pseudonymised any data shared with the researcher by assigning each patient a unique study identifier.³⁴

Data analyses were performed using STATA, a statistical software, version 18 (Stata Corp LP, College Station, Texas, United States [US]). For each phase of the algorithm development, we calculated Sensitivity (Se) and Specificity (Sp) for variables and combinations that met the eligibility criteria (see Equation 1 and Equation 2):

In this study, a true positive was a patient listed for appendectomy on the operative list who also had evidence in the provincial database consistent with an appendectomy case. A true negative was a control patient (non-appendectomy) on the operative list who was not identified as a case in the database. False negatives were patients listed as a case on the operative list but not identified as a case in the database using the included variables. False positives were a control on the operative list but were picked up as a case in the database. Various combinations were made within and across selected variables and corresponding values, which was necessary to improve the performance of the final algorithm (Figure 1).

Sensitivity and Specificity thresholds set for the algorithm

A prior study carried out in Denmark validated an appendicitis and appendectomy algorithm and reported a Se and Sp of 92.8% and 99.5%.⁵ We selected slightly lower thresholds because we expected the data coding and quality to be poorer in an LMIC setting. Therefore, we selected a Se of ≥ 85% to maximise the number of cases identified and a higher Sp of ≥ 95% to minimise the false positives included.

Study procedure

Phase 1: Variable exploration and selection

All available variables for reference dataset 1 were extracted from the database and explored. Case data were used for algorithm development, and the control data informed the Se and Sp analysis (Table 1). Variables were included if they were clinically relevant for appendectomy, had complete data and met the predefined Se or Sp thresholds (Figure 1). If any of the criteria were not met, the variable was excluded.

Phase 2: Algorithm development

Step 1: Variables and corresponding values that met the eligibility criteria were used to extract potential cases from the database. Se and Sp for each value were calculated and compared with the reference dataset 2. If no individual values met the Se or Sp threshold, values were combined to improve overall performance. Pairwise, incremental and systematic combinations were created within variables, and Sp and Se were calculated for each combination. To optimise Sp, values within a variable were combined using the ‘AND’ operator. To optimise Se, values were combined using the ‘OR’ operator.

Step 2: Intervariable combinations were then created to optimise performance further. If the Se and Sp thresholds were achieved at this stage, the algorithm proceeded to validation (Phase 4). If not, the algorithm was refined.

Phase 3: Algorithm refinement

Phase 2 was repeated by extracting data from the database and creating combinations between the selected variables to achieve the desired Se and Sp by comparing the results to reference dataset 3 (Figure 1). This would continue until the optimal Se and Sp were met, and the algorithm could be validated.

Phase 4: Validation

To validate the final algorithm, the combination of variables and values with the highest Sp and Se was used to extract cases. The final Sp and Se were determined and compared with the reference dataset 4 (Figure 1).

Ethical considerations

This study was approved by the Health Research Ethics Committee of Stellenbosch University (S21/02/031 [PhD]), the Health Research Ethics Committee of the University of Cape Town (Project 2021/226) and the Western Cape Department of Health (WC_202104_030).

Results

All appendectomy cases (n = 1177) and general surgery procedure controls (n = 5057) from the study hospitals were included. The reference standard was divided into four datasets for the different study phases (Figure 1).

Phase 1: Variable exploration and selection

Two variables in the database met the eligibility criteria, procedure and diagnostic coding. Distinguishing between primary and secondary codes was crucial given that surgical conditions and procedures are often associated with multiple diagnostic and procedure codes. Sixteen primary procedure codes were recorded for cases. While all the codes were ≥ 95% Sp, none on their own met the ≥ 85% Se threshold (Table 1). Laparoscopic appendectomy (47.01) achieved the highest Se (23.8%) with an Sp of 99.9% (Table 1), which meant that at least 76.2% of appendectomy cases were missed.

Twenty-seven diagnostic codes were associated with the cases (Table 2). Similarly, the diagnostic codes were Sp for the cases, while the highest recorded Se was 28.5%.

Given the overall low Se of all procedure and diagnostic codes, based on clinical association with appendectomy, procedure codes with the words ‘appendectomy’, ‘laparotomy’ OR ‘laparoscopy’ OR diagnostic codes containing the words ‘appendicitis’ OR ‘appendix’ were included in the algorithm (Table 1 and Table 2).

Phase 2: Algorithm development

The selected codes were used to extract potential cases from the database and compared with the reference dataset 2. All but one of the codes (47.09 – other appendectomy) were ≥ 95% Sp; however, none were ≥ 85% Se (Table 3).

Stepwise combinations were made within procedure codes to optimise the algorithm’s performance. The ‘OR’ combinations led to a higher Se while reducing the algorithm’s Sp. Combinations with ‘AND’ showed the opposite results, with a very low Se and nearly 100% Sp. Similar findings were observed for diagnostic codes. In the process of combining procedure and diagnostic codes separated with the ‘OR’ operator, two procedure codes were dropped – ‘Exploratory laparotomy’ and ‘Other appendectomy’ – because of their effect on the overall Se of the algorithm. The highest Se achieved was 64%, with a lowered Sp of 90% by P&D Combo53_or (Procedure codes [‘Appendectomy’ OR ‘Laparoscopic appendectomy’ OR ‘Other laparotomy’] OR Diagnostic codes [‘Acute appendicitis, other and unspecified’ OR ‘Acute appendicitis with generalised peritonitis’ OR ‘Acute appendicitis with localised peritonitis’]). Therefore, the algorithm required further refining before it could be finalised.

Phase 3: Algorithm refinement

The algorithm combinations from Phase 2 were further refined. This meant reordering the codes to achieve the highest Se and Sp. The results are shown in Table 4. Any one of two procedures or nine diagnostic codes (P&D Combo1_or) could identify a case in the provincial database.

Phase 4: Validation

The final algorithm comprised procedure codes (‘Appendectomy’ OR ‘Laparoscopic appendectomy’) OR diagnostic codes (‘Acute appendicitis, other and unspecified’ OR ‘Acute appendicitis with generalised peritonitis’ OR ‘Acute appendicitis with localised peritonitis’ OR ‘Unspecified appendicitis’ OR ‘Disease of appendix, unspecified’ OR ‘Fistula of appendix’ OR ‘Other appendicitis’ OR ‘Hyperplasia of appendix’ OR ‘Other specified diseases of appendix’) and was validated using reference dataset 4. A final Se of 91.3% and an Sp of 96% were confirmed (Table 5).

Discussion

Data on surgical procedures are severely lacking in South Africa. The provincial database in the Western Cape, South Africa, is one of the few databases in the country and is continually refined. Several algorithms have been developed to identify pregnant women and persons with certain comorbidities for use in these databases.^27,28,29,30 However, surgical conditions and procedure algorithms are more challenging to develop owing to the lack of a unique clinical signature.²⁰ Our study highlights the potential for similar approaches to be applied to other surgical procedures in comparable settings such as South Africa.

Our study found similar results to an HIC study, validating diagnostic and procedure codes as the best variables to identify appendectomy cases. Initially, we had hypothesised that other variables might be more Se and Sp to identify appendectomy cases because coding can be a real challenge in LMICs. However, data completeness varied across facilities, and only two variables, procedure and diagnostic codes, met the eligibility criteria for inclusion in the algorithm. Many public hospitals still rely on surgical logbooks and other paper-based systems, which leads to incomplete, inaccessible and inconsistent patient data in LMICs.^35,36 Although e-health systems are used in some facilities, integrating them into a unified national system remains a challenge in LMICs.³⁷ Barriers include a lack of national leadership and coordination, insufficient understanding of the requirements of scaling up, workforce shortages and a lack of investment in infrastructure.³⁷

In our dataset, appendectomy cases were associated with 16 procedures and 27 diagnostic codes, and nine of each variable were selected based on the eligibility criteria. These codes had high Sp but low Se, highlighting the quality of coding practices in the public health sector in LMICs compared to HICs. Similar findings were observed in other studies that showed International Classification of Diseases (ICD) coding often has a high Sp but low Se.^{7,8,9,10,11,12,13,14,15,16,17} The coding quality could be affected by insufficient numbers of adequately trained staff or a lack of a large enough workforce. A South African study assessing a training and support programme for discharge ICD coding at two central hospitals found improved completeness but no change in the accuracy of primary ICD codes.¹⁹ The authors recommended that policymakers consider implementing affordable components of such interventions to enhance coding quality.¹⁹ Increasing the workforce supply could also potentially improve the quality of coding practices, enabling adequate time to dedicate to these practices.

Beyond its immediate application to appendectomy, this study demonstrates the potential of validated algorithms to address the broader gap in data on the number and types of surgical procedures performed over a defined period in LMICs. This is a crucial Lancet indicator and serves as a measure of the strength of the national surgical system while contributing to our understanding of surgical capacity. In many LMICs, including South Africa, these metrics are poorly documented because of fragmented data systems and the absence of dedicated operative registries. By showing that a reliable algorithm can be developed from routinely collected health information system data, our work provides a methodological framework that could be adapted to identify other surgical procedures, enabling more accurate measurement of surgery. This, in turn, could strengthen surgical surveillance, inform workforce and infrastructure planning and guide policy decisions to improve access to safe, timely surgical care.

Limitations

The creation of the algorithm was limited by the quality of healthcare records and the number of variables that were Sp and sufficiently completed for appendectomy. Despite these limitations, the final algorithm achieved high Se and Sp. Our algorithm was based on the coding used in the ICD-9 and ICD-10 coding systems and should be revisited with the adoption and use of ICD-11. The reference standard comprised patients from four selected hospitals where diagnostic and procedure codes were 100% complete. However, this level of completeness might not be consistent across facilities in South Africa. Therefore, a follow-up study is recommended to assess the algorithm’s performance across healthcare settings.

Conclusion

This study demonstrates the feasibility of developing and validating a high-performing appendectomy algorithm within an LMIC database. Using routinely collected data from the Western Cape provincial database, the algorithm achieved high Se and Sp despite variability in coding completeness and quality across facilities. Differences in data infrastructure, coding practices, and resource availability influence algorithm performance in LMICs compared to HICs. Importantly, this work addresses a critical gap in LMIC surgical data by providing a reproducible method for identifying a specific surgical procedure from routine health information system data. Such algorithms can be adapted to improve the availability and accessibility of data on surgical conditions and procedures regarding range and volume on a large scale. Strengthening these data streams is essential for monitoring surgical capacity, guiding resource allocation and informing policy to improve equitable access to safe, timely surgical care.

Acknowledgements

The authors would like to acknowledge the data scientists at the Provincial Health Data Centre, and the health care providers at the study hospitals for their assistance with the data required for the study.

This article is based on research originally conducted as part of J.L.’s doctoral thesis titled ‘Exploring Pathways to Accessing Care for Appendectomy in the Western Cape Government Health System, South Africa’, submitted to the Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University in 2025. The thesis was supervised by Kathryn Chu, Rene English, and Peter Nyasulu. The article has since been revised and adapted for journal publication.

The author, P.N., serves as an editorial board member of this journal. P.N. has no other competing interests to declare.

The author reported that they received funding from the National Research Foundation, which may be affected by the research reported in the enclosed publication. The author has disclosed those interests fully and has implemented an approved plan for managing any potential conflicts arising from their involvement. The terms of these funding arrangements have been reviewed and approved by the affiliated university in accordance with its policy on objectivity in research.

K.C. and J.L. conceived the study and contributed to the design of the study. J.L. analysed the data and wrote the first draft of the article. K.C., P.N. and R.E. reviewed the first draft and edited the article. All authors reviewed and approved the final version of the article.

This research was funded by the National Research Foundation.

The data that support the findings of this study are available from the corresponding author, J.L., upon reasonable request.

The views and opinions expressed in this article are those of the authors and are the product of professional research. They do not necessarily reflect the official policy or position of any affiliated institution, funder, agency or that of the publisher. The authors are responsible for this article’s results, findings and content.

References