Chronic Kidney Disease Progression Prediction Using Transformer Models

Chronic Kidney Disease has shown a concerning trend over the past few decades, due to its prevalence nearing ten percent of the global population. It is also closely tied with heart disease, making it a particular concern for healthcare providers. It has been known for quite some time that earlier interventions, if possible, could slow down someone's decline into more severe stages of Chronic Kidney Disease. Much of the work in this field up to this point has centered around the prediction of the presence of Chronic Kidney Disease.

In terms of modeling future outcomes, the current state-of-the-art centers around the use of Recurrent Neural Networks (RNN) to predict future states. The central idea in this Praxis is to prove that there exists a better modeling architecture that can help with regards to the longitudinal nature of the problem found with CKD modeling and the use of longitudinal Electronic Health Record (EHR) data. The Transformer is such a model. This Praxis highlights the benefits of this architecture, along with improvements that can be made to it to help it perform well in this situation. The main improvements posited in this Praxis are the use of temperature scaling and a custom loss function that combines both binary cross-entropy (BCE) and Matthews Correlation Coefficient (MCC).

The Transformer model, as shown through various tests in the Praxis, can achieve a greater than nine percent improvement in the accuracy metric, reaching as high as 99% across different tests. The performance of this architecture remains robust across different thresholds of patients. SMOTE and stratified sampling were used, as well, to validate the performance of this approach. Other metrics were benchmarked as well such as AUROC, AUPRC, and MCC, keeping it in line with the seminal paper in this space. This highlights its usefulness in accurate detection of future decline in Chronic Kidney Disease Stage, which can lead to interventions happening sooner, such as having the patient start taking Angiotensin-Converting Enzyme (ACE) or Angiotensin Receptor Blocker (ARB) medications or getting an appointment with a nephrologist.