Improving the prognostication of patients receiving palliative radiotherapy

Introduction

Oncologists often have difficulty identifying the terminal period of cancer where treatment cannot realistically prolong survival. This in turn affects prognostication of these patients (1-4). Accurate prognostication is, however, crucial in deciding whether or not to offer any intervention versus best supportive care.

Radiotherapy (RT) is effective in treating patients suffering from symptoms due to a local tumour. Its effectiveness is, however, limited in situations where death is imminent, when multiple competing symptoms are unable to be relieved by RT, or when the treatment course is long (5,6). It is essential that good prognostication tools are available to the treating radiation oncologist for appropriate and timely allocation of resources.

Clinical reviews on prognostication models suggest that good models encompass both clinical and laboratory variables (7-9). Two of the better-known models in the palliative RT setting—the number of risk factors (NRF) and TEACHH models—include only clinical variables. Chow et al. developed the NRF prognostic model looking at Karnofsky Performance Status (KPS), site of disease and site of metastases to describe the functional status and disease burden of a patient (10). The TEACHH model developed by Krishnan et al. stratified patients into three prognostic groups based on cancer type, Eastern Cooperative Oncology Group (ECOG) performance status, age, prior palliative chemotherapy, prior hospitalisation and hepatic metastases (11). Both models have been externally validated in several studies, and have shown to be effective tools in the Caucasian population (10,12-14). Currently, data on the performance of these models in Asian populations is limited.

Since the development of these two models, new prognostic factors have been identified for advanced cancer patients such as the Palliative Performance Scale (PPS), Glasgow Predictive Score and the neutrophil-lymphocyte ratio (NLR) (15-18). The usefulness of these new factors for the prognostication of patients undergoing palliative RT is unknown.

This study aims to create and validate a new prognostic model for Asian patients receiving palliative RT, validate the NRF and TEACHH model in Asian patients, and compare the performance of the new prognostic model with the NRF and the TEACHH model. As part of model development, the prognostic value of PPS and NLR will be assessed. We present the following article in accordance with the TRIPOD reporting checklist (available at http://dx.doi.org/10.21037/tro-20-59).

Methods

Patient recruitment

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by Singhealth Institutional Review Board with waiver of consent (No. 2013/530/B). We prospectively recruited patients treated with external beam RT with palliative intent at the National Cancer Centre Singapore (NCCS) from July 2013 to January 2014. Patients who did not have the capacity to answer questions coherently and not represented by primary caregivers were excluded from the study.

Study assessments

Clinical data were extracted from patient’s clinical notes and electronic medical records. Where available, latest routine haematology and biochemistry tests taken in the month prior to the start of RT were recorded.

Patients were assessed on the PPS by trained radiation therapists (RTTs) during the pre-RT briefing. RTTs involved in the measurement of the PPS received training on the job by the principal investigator to assess the score. RTTs provided unbiased PPS assessments as they do not have to evaluate the patient’s functional status or be cognizant of their treatment history and extent of disease during routine work. Each patient was assessed once by a RTT due to practical reasons and the high inter-observer reliability of PPS (19).

RT treatment and follow up

Patients received RT as prescribed by their respective oncologist, independent of their assessed PPS status. Patients were followed up about 4–6 weeks post completion of RT and subsequently every 3–6 months at the discretion of the treating oncologist. All recruited patients were followed until 2 November 2017 or death.

Statistical methods

Formal sample size calculations were not performed as there was no generally accepted approaches to determine an adequate sample size for developing a prognostic model. To ensure that there would be a reasonable number of deaths for identification of risk factors during model development and around 100 deaths for model validation, recruited patients were followed up until at least 250 deaths were observed before analysis was conducted.

Patients with an uniform random number ≤0.6 were assigned into a 60% training cohort and those with a corresponding number >0.6 were assigned into a 40% testing cohort for model development and validation, respectively. Differences of continuous and categorical characteristics between the two cohorts were compared using Mann Whitney U test and Fisher’s Exact test, respectively.

Overall survival (OS) was defined as the time from start of RT to date of death from any cause. OS distribution was estimated using the Kaplan-Meier method, and differences between OS curves were compared using the log-rank test. Univariate Cox proportional hazard (PH) regression analyses were used to assess the association of various variables with OS via hazard ratio (HR). Derived continuous variables, such as NLR and platelet-lymphocyte ratio (PLR), were categorised based on the minimum P value method (20). The ordinal PPS was analysed based on 10–50% vs. 60–100% grouping; this categorisation was chosen because it provided the largest discrimination in OS and a sufficiently large number of patients in each group. Prognostic factors were identified using multivariable Cox regression by applying backward elimination method on variables with univariate P<0.05. The PH assumption was verified based on Schoenfeld residuals.

Parametric survival models were fitted to the training cohort with the identified prognostic factors included as covariates to see if the survival predictions from these models were better than those based on the Cox model. Three parametric survival models were assessed—Weibull accelerated failure time (AFT) model, log-logistic AFT model and flexible Royston-Parmar (RP) model (21). The choice of scale (proportional hazards, proportional odds or probit) to model the baseline function and the number of knots to characterise the restricted cubic spline function for the optimal RP model were selected based on the minimum Akaike Information Criterion and Bayesian Information Criterion. For our data, the optimal RP model was the PO(3) model in which a restricted cubic spline with two knots was used to model the logarithm of the baseline cumulative odds function.

Model fit was assessed based on the exponential QQ plot of Cox-Snell residuals and explained variation R² (R-square) (22). Model discrimination was assessed based on the Royston and Sauerbrei’s D statistics (D-stats) and the Harrell’s concordance index for censored data (c-index) (23,24). Model calibration was assessed graphically by comparing the models’ predicted survival estimates against the empirical Kaplan-Meier estimates.

The NRF and TEACHH models were validated based on patients with non-missing data for all the prognostic factors included in each model. The total number of risk factors possessed by each patient under each model was calculated. Approximations were made for three risk factors based on the data collected in this study: (I) KPS ≤60 in the NRF model was approximated by ECOG performance status ≥2 following the conversion rules used in Chow’s paper (25); (II) >2 prior palliative chemotherapy lines in the TEACHH model was approximated by chemotherapy within 3 months prior to start of RT; and (III) prior hospitalisation in the TEACHH model was approximated by inpatient at start of RT. Patients were stratified into 3 prognostic groups under each model viz. Chow (0–1, 2 and 3 risk factors) and TEACHH (0–1, 2–4 and 5–6 risk factors).

Complete-case analyses were performed in this study. All reported P values were 2-sided and statistically significant if P<0.05. All analyses were performed using Stata version 14.2 (StataCorp, College Station, TX, USA).

Results

Clinical characteristics and survival outcomes

A total of 288 patients were recruited in this study; 175 were randomly assigned to the training cohort and 113 to the testing cohort. In the training cohort, median OS was 5.2 months (95% CI, 3.4–7.7 months) and 3-month OS was 62.2%. For the testing cohort, median OS was 6 months (95% CI, 4.0–8.0 months) and 3-month OS was 66.9%. At the data cut-off in November 2017, 36 patients were alive, of which 16 (8 from each cohort) had survived >2 years. Clinical and treatment characteristics of patients were similar between the two cohorts (Table 1).

Table 1 Clinical characteristics and treatment outcomes of patients in training and testing cohort
Full table

Prognostic factors and NCCS prognostic model

Univariate Cox regression analyses revealed that non-breast cancer, metastases other than bones, low albumin, low haemoglobin, NLR ≥6, high PLR, inpatient at start of RT, PPS score ≤50% and chemotherapy within 3 months prior to start of RT were adversely associated with OS (Table 2). On multivariable analysis, NLR ≥6 (P=0.001), inpatient at start of RT (P=0.05), PPS score ≤50% (P<0.001) and chemotherapy within 3 months prior to start of RT (P=0.001) remained statistically significant, and were selected as independent prognostic factors.

Table 2 Training cohort: prognostic factors of overall survival using Cox regression analyses
Full table

NLR failed PH assumption, and the fit and discrimination ability of the Cox model were poor (Figure 1A). Among the parametric survival models assessed, the log-logistic AFT model had better model fit and discrimination than the Weibull AFT and RP PO(3) model (Figure 1B,C,D). The log-logistic AFT model was chosen as the NCCS prognostic model, and survival of a patient at month t was estimated asS(t)=11+(tα)β, where β=1.33 was the shape parameter of the log-logistic distribution, α = exp(2.60 − 0.75I_NLR≥6 − 0.75I_INPATIENT − 0.73I_{PRIOR CHEMO} − 1.14I_PPS≤50%) was the scale parameter, and I_x was an indicator variable for factor x. Based on this model, the estimated survival of patients with NLR ≥6 was about half of that of patients with NLR <6 [accelerated factor (AF) 0.47; 95% CI, 0.30–0.74]. The AF was 0.47 (95% CI, 0.28–0.78) for inpatient at start of RT, 0.32 (95% CI, 0.19–0.55) for PPS score ≤50%, and 0.48 (95% CI, 0.30–0.78) for chemotherapy prior to start of RT.

Figure 1 Training cohort: diagnostic checks and discrimination ability of various models fitted with prognostic factors of overall survival. (A) Cox PH model; (B) Weibull AFT model; (C) log-logistic AFT model; (D) Royston-Parmar PO(3) spline model. Each plot depicts the exponential QQ plot of Cox-snell residuals of the fitted model; the closer the residuals to the 45^o line, the better the fit of the model.

To validate the NCCS prognostic model, patients were stratified into a low risk group (0 risk factor), a moderate risk group (1–2 factors) and a high-risk group (3–4 factors). Population averaged survival predictions for each risk group derived based on the prognostic model were close to their corresponding empirical Kaplan-Meier estimates in the training cohort (Figure 2A). The model achieved good calibration and discrimination based on the testing cohort (Figure 2B).

Figure 2 Temporal validation of the NCCS model: overall survival by prognostic groups according to number of prognostic factors in (A) training cohort and (B) testing cohort. Estimated survival probabilities estimated based on the NCCS model (dotted lines) and the actual outcomes estimated based on the Kaplan-Meier method (solid lines) are shown.

Validation of NRF model and TEACHH model and comparison with NCCS prognostic model

Based on NRF model, 20% of patients in our study had 0-1 risk factors (median OS 8 months; 95% CI, 3.2–16.7 months), 58% had 2 risk factors (6.6 months; 95% CI, 4.0–8.7 months), and 22% had 3 risk factors (2.8 months; 95% CI, 1.4–3.6 months) (Figure 3A). The NRF model was unable to separate the NCCS patients into distinct prognostic groups, with the OS curves of the 0–1 and 2 risk factors groups intertwined for the first 6 months after start of RT. Model discrimination was poor with D-stats =0.510 and c-index =0.567.

Figure 3 Overall survival by prognostic groups according to the number of prognostic factors based on (A) NRF model, (B) TEACHH model, and (C) NCCS model.

Based on the TEACHH model, 17% of patients had 0–1 risk factor (15.4 months; 95% CI, 8.0–20.2 months), 79% had 2–4 risk factors (3.5 months; 95% CI, 3.0–5.2 months), and 4% had 5–6 risk factors (0.6 months; 95% CI, 0.2 to undefined) (Figure 3B). Compared with the NRF model, the TEACHH model achieved a greater degree of discrimination (D-stats =0.656 and c-index =0.580).

The discrimination ability of the NCCS prognostic model was highest with D-stats at 1.020 and c-index at 0.657 (Figure 3C). Median OS (95% CI) was 13.5 months (10.6–17.4 months) for patients with no risk factors, 4.4 months (3.2–6.5 months) for 1–2 risk factors, and 1.3 months (0.7–1.6 months) for 3–4 risk factors.

Discussion

Development of the NCCS prognostic model

This study is the first to create a prognostic tool via parametric survival modelling for Asian patients undergoing palliative RT. A log-logistic AFT model with 4 adverse factors (PPS score ≤50%, NLR ≥6, inpatient at start of RT, and chemotherapy within 3 months prior to start of RT) predicted survival most accurately in these patients.

Performance status was measured based on the PPS in our model, compared to KPS in the NRF model and ECOG in the TEACHH model. Bivariate Cox regression analysis with ECOG and PPS in our patients showed that ECOG was not significantly associated with survival after accounting for PPS (results not shown). The PPS has been shown to prognosticate survival in a variety of populations and performs well in inter- and intra-observer reliability studies (26-30). While PPS assessment is less straightforward compared to ECOG and KPS, it provides a more comprehensive measurement of functional status of palliative patients. Its use also facilitates decision making and communication in the palliative care setting.

In addition, despite the availability of NLR from routine full blood count and the association of elevated NLR with poorer survival across various types of metastatic cancers reported in multiple studies (31-35), it has not been included in any of the prognostic models for palliative RT. Compared to other studies where the NLR thresholds were often arbitrarily set based on percentiles of its distribution, the threshold in our study was determined based on the minimum P value method in which all potential NLR values were systematically tested to obtain a value that best separated the OS of patients, while controlling for the overall false positive rate.

Inpatient status at start of treatment has not yet been reported as a prognostic factor for patients with metastatic cancer. There is an increased rate of hospitalisation toward the end of life of cancer patients. Studies have found that about half of cancer-related deaths occurred in acute hospitals (36-38). While the reasons for admission were not captured in our study, it would not be unreasonable to infer that the inpatients were more symptomatic and required a higher level of care that could not be administered in the outpatient setting. Several authors have reported that clinical symptoms have an independent predictive value for survival in patients with terminal cancers (39-41).

At least 75% of the patients enrolled in this study did not have chemotherapy 3 months pre-RT. Based on the NCCS prognostic model, the survival for these patients post RT was about 2.1 times longer than that of patients with chemotherapy. Earle et al. found that patients who received chemotherapy were more likely to die in hospital than those who did not (34.9% vs. 29.0%, P<0.001), and that 15.7% of those receiving chemotherapy were still undergoing treatment within 2 weeks of death (42). This is not unusual as while patients are on chemotherapy, they may experience toxicities from chemotherapy, including neutropaenic sepsis, that could put them at a higher risk of death.

Use of prognostic model in treatment planning

Short courses of RT have been demonstrated to be as effective as longer courses in treating various symptoms (43-45). Despite a long duration between the time of RT and symptom relief, many patients still receive RT near the end of their lives; they may even spend a significant portion of their remaining lives attending RT sessions (46,47). In our cohort, 6.7% of patients who died spent more than half of their remaining life span on RT. In patients with a more guarded prognosis, shorter courses or even withholding RT would be preferable, with aggressive management of symptoms in collaboration with a palliative care team.

While survival predictions are useful to guide treatment selection for individual patients, radiation oncologists should not rely solely on them when formulating treatment plans for patients. Human survival is a complex process that is unrealistic to predict with a high degree of accuracy (10). Our model was only able to account for 31% of the variability in the survival times of our patients, even though it had a high discrimination ability and was successfully validated. The low explained variability was broadly in line with those reported by the NRF model (15–24%) and the TEACHH model (17%).

Validation of the NRF and TEACHH models

Validation of the NRF and TEACHH models did not reveal any significant differences in the OS between patients included and those who were excluded (27% of the 288 patients due to missing ECOG status- results not shown). The TEACHH model appeared to perform better than the NRF model. The median OS of our patients with 0-1, 2, and 3 NRF risk factors were 8, 6.6 and 2.8 months respectively; these were much shorter than the 55–64, 19–28 and 9–10 months reported by Chow et al. for the three risk groups based on their training, temporal validation and external validation dataset. In comparison, the median OS of our patients with 0–1, 2–4 and 5–6 TEACHH risk factors were 15.4, 3.5 and 0.6 months respectively; these were similar to the corresponding 19.9, 5 and 1.7 months reported by Krishan et al. Similar results of the NRF and TEACHH models were reported by Dosani et al. based on a cohort of Canadian patients treated with palliative RT for spinal metastases (14). Of the three NRF factors, only site of disease and metastases were significantly associated with OS. That said, two of the TEACHH risk factors viz. >2 prior palliative chemotherapy lines and prior hospitalisation were approximated for our patients. Furthermore, different exclusion criteria were also used. Our validation of the TEACHH model has its limitations, and thus its performance may not entirely relate to the model itself. We await the results of another Asian institution to see if the TEACHH model performs better than the NRF model (48).

Strengths and limitations of the NCCS prognostic model

The NCCS model is developed based on log-logistic AFT, which is a parametric survival model. This overcomes many limitations of the Cox PH model, the most commonly-used model for prediction of patient survival (49-52), which have been widely described by various authors (53,54). With the survival function of the prognostic model fully estimated, the clinical utility of the model will be higher, as this allows individualised absolute survival prediction at all time points. Our model also utilises easily obtainable patient information which does not require medically trained personnel to assess each variable. However, data originated from a single institution and may not be generalised to all palliative patients. A relatively small sample size in the training cohort (n=175) was also used to develop the NCCS model compared with other studies (10,55-57). Hence, the model may not have included other potential prognostic factors for OS.

Conclusions

The utilisation of objective prognostic systems can improve the reliability of clinician’s prediction of survival. We developed a prognostic model for Asian patients undergoing palliative RT which had better discrimination ability than the NRF and TEACHH models. Further work to validate this model with data from other institutions are required to confirm its predictive accuracy.

Acknowledgments

The authors wish to thank the faculty and staff of the Division of Radiation Oncology, NCCS and all physicians who provided and cared for the patients in this study.

Funding: None.

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at http://dx.doi.org/10.21037/tro-20-59.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/tro-20-59). DSCQ serves as an unpaid editorial board member of Therapeutic Radiology and Oncology from Apr 2020 to Mar 2022. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by institutional review board of Singhealth (No. 2013/530/B) with a waiver of consent, as there was no more than minimal risk to the patients recruited for the study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Chow E, Harth T, Hruby G, et al. How accurate are physicians' clinical predictions of survival and the available prognostic tools in estimating survival times in terminally ill cancer patients? A systematic review. Clin Oncol (R Coll Radiol) 2001;13:209-18. [Crossref] [PubMed]
2. Vigano A, Dorgan M, Bruera E, et al. The relative accuracy of the clinical estimation of the duration of life for patients with end of life cancer. Cancer 1999;86:170-6. [Crossref] [PubMed]
3. Christakis NA. Predicting patient survival before and after hospice enrollment. Hosp J 1998;13:71-87. [Crossref] [PubMed]
4. McCusker J. The terminal period of cancer: definition and descriptive epidemiology. J Chronic Dis 1984;37:377-85. [Crossref] [PubMed]
5. Lutz S, Korytko T, Nguyen J, et al. Palliative radiotherapy: when is it worth it and when is it not? Cancer J 2010;16:473-82. [Crossref] [PubMed]
6. Jones JA, Lutz ST, Chow E, et al. Palliative radiotherapy at the end of life: a critical review. CA Cancer J Clin 2014;64:296-310. [Crossref] [PubMed]
7. Maltoni M, Caraceni A, Brunelli C, et al. Prognostic factors in advanced cancer patients: evidence-based clinical recommendations--a study by the Steering Committee of the European Association for Palliative Care. J Clin Oncol 2005;23:6240-8. [Crossref] [PubMed]
8. Krishnan M, Temel JS, Wright AA, et al. Predicting life expectancy in patients with advanced incurable cancer: a review. J Support Oncol 2013;11:68-74. [Crossref] [PubMed]
9. Vigano A, Dorgan M, Buckingham J, et al. Survival prediction in terminal cancer patients: a systematic review of the medical literature. Palliat Med 2000;14:363-74. [Crossref] [PubMed]
10. Chow E, Abdolell M, Panzarella T, et al. Predictive model for survival in patients with advanced cancer. J Clin Oncol 2008;26:5863-9. [Crossref] [PubMed]
11. Krishnan MS, Epstein-Peterson Z, Chen YH, et al. Predicting life expectancy in patients with metastatic cancer receiving palliative radiotherapy: the TEACHH model. Cancer 2014;120:134-41. [Crossref] [PubMed]
12. Chow E, James JL, Hartsell W, et al. Validation of a Predictive Model for Survival in Patients With Advanced Cancer: Secondary Analysis of RTOG 9714. World J Oncol 2011;2:181-90. [PubMed]
13. Angelo K, Dalhaug A, Pawinski A, et al. Survival prediction score: a simple but age-dependent method predicting prognosis in patients undergoing palliative radiotherapy. ISRN Oncol 2014;2014:912865 [Crossref] [PubMed]
14. Dosani M, Tyldesley S, Bakos B, et al. The TEACHH model to predict life expectancy in patients presenting for palliative spine radiotherapy: external validation and comparison with alternate models. Support Care Cancer 2018;26:2217-27. [Crossref] [PubMed]
15. McMillan DC. The systemic inflammation-based Glasgow Prognostic Score: a decade of experience in patients with cancer. Cancer Treat Rev 2013;39:534-40. [Crossref] [PubMed]
16. Anderson F, Downing GM, Hill J, et al. Palliative performance scale (PPS): a new tool. J Palliat Care 1996;12:5-11. [Crossref] [PubMed]
17. Partridge M, Fallon M, Bray C, et al. Prognostication in advanced cancer: a study examining an inflammation-based score. J Pain Symptom Manage 2012;44:161-7. [Crossref] [PubMed]
18. Mei Z, Shi L, Wang B, et al. Prognostic role of pretreatment blood neutrophil-to-lymphocyte ratio in advanced cancer survivors: A systematic review and meta-analysis of 66 cohort studies. Cancer Treat Rev 2017;58:1-13. [Crossref] [PubMed]
19. Campos S, Zhang L, Sinclair E, et al. The palliative performance scale: examining its inter-rater reliability in an outpatient palliative radiation oncology clinic. Support Care Cancer 2009;17:685-90. [Crossref] [PubMed]
20. Mazumdar M, Glassman JR. Categorizing a prognostic variable: review of methods, code for easy implementation and applications to decision-making about cancer treatments. Stat Med 2000;19:113-32. [Crossref] [PubMed]
21. Royston P, Lambert P. Flexible Parametric Survival Analysis Using Stata: Beyond the Cox Model. College Station, Texas: Stata Press, 2011.
22. Royston P. Explained variation for survival models. Stata J 2006;6:83-96. [Crossref]
23. Royston P, Sauerbrei W. A new measure of prognostic separation in survival data. Stat Med 2004;23:723-48. [Crossref] [PubMed]
24. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15:361-87. [Crossref] [PubMed]
25. Chow E, Abdolell M, Panzarella T, et al. Validation of a predictive model for survival in metastatic cancer patients attending an outpatient palliative radiotherapy clinic. Int J Radiat Oncol Biol Phys 2009;73:280-7. [Crossref] [PubMed]
26. Lau F, Downing GM, Lesperance M, et al. Use of Palliative Performance Scale in end-of-life prognostication. J Palliat Med 2006;9:1066-75. [Crossref] [PubMed]
27. Seow H, Barbera L, Dudgeon D, et al. The association of the palliative performance scale and hazard of death in an ambulatory cancer population. J Palliat Med 2013;16:156-62. [Crossref] [PubMed]
28. Ho F, Lau F, Downing MG, et al. A reliability and validity study of the Palliative Performance Scale. BMC Palliat Care 2008;7:10. [Crossref] [PubMed]
29. Head B, Ritchie CS, Smoot TM. Prognostication in hospice care: can the palliative performance scale help? J Palliat Med 2005;8:492-502. [Crossref] [PubMed]
30. Harrold J, Rickerson E, Carroll JT, et al. Is the palliative performance scale a useful predictor of mortality in a heterogeneous hospice population? J Palliat Med 2005;8:503-9. [Crossref] [PubMed]
31. Thio Q, Goudriaan WA, Janssen SJ, et al. Prognostic role of neutrophil-to-lymphocyte ratio and platelet-to-lymphocyte ratio in patients with bone metastases. Br J Cancer 2018;119:737-43. [Crossref] [PubMed]
32. Vano YA, Oudard S, By MA, et al. Optimal cut-off for neutrophil-to-lymphocyte ratio: Fact or Fantasy? A prospective cohort study in metastatic cancer patients. PLoS One 2018;13:e0195042 [Crossref] [PubMed]
33. Dolan RD, McSorley ST, Horgan PG, et al. The role of the systemic inflammatory response in predicting outcomes in patients with advanced inoperable cancer: Systematic review and meta-analysis. Crit Rev Oncol Hematol 2017;116:134-46. [Crossref] [PubMed]
34. Guthrie GJ, Charles KA, Roxburgh CS, et al. The systemic inflammation-based neutrophil-lymphocyte ratio: experience in patients with cancer. Crit Rev Oncol Hematol 2013;88:218-30. [Crossref] [PubMed]
35. Bowen RC, Little NAB, Harmer JR, et al. Neutrophil-to-lymphocyte ratio as prognostic indicator in gastrointestinal cancers: a systematic review and meta-analysis. Oncotarget 2017;8:32171-89. [Crossref] [PubMed]
36. DeCaria K, Dudgeon D, Green E, et al. Acute care hospitalization near the end of life for cancer patients who die in hospital in Canada. Curr Oncol 2017;24:256-61. [Crossref] [PubMed]
37. Bekelman JE, Halpern SD, Blankart CR, et al. Comparison of Site of Death, Health Care Utilization, and Hospital Expenditures for Patients Dying With Cancer in 7 Developed Countries. JAMA 2016;315:272-83. [Crossref] [PubMed]
38. Zhuang Q, Lau ZY, Ong WS, et al. Sociodemographic and clinical factors for non-hospital deaths among cancer patients: A nationwide population-based cohort study. PLoS One 2020;15:e0232219 [Crossref] [PubMed]
39. Reuben DB, Mor V, Hiris J. Clinical symptoms and length of survival in patients with terminal cancer. Arch Intern Med 1988;148:1586-91. [Crossref] [PubMed]
40. Morita T, Tsunoda J, Inoue S, et al. Survival prediction of terminally ill cancer patients by clinical symptoms: development of a simple indicator. Jpn J Clin Oncol 1999;29:156-9. [Crossref] [PubMed]
41. Chuang RB, Hu WY, Chiu TY, et al. Prediction of survival in terminal cancer patients in Taiwan: constructing a prognostic scale. J Pain Symptom Manage 2004;28:115-22. [Crossref] [PubMed]
42. Earle CC, Neville BA, Landrum MB, et al. Trends in the aggressiveness of cancer care near the end of life. J Clin Oncol 2004;22:315-21. [Crossref] [PubMed]
43. Hartsell WF, Scott CB, Bruner DW, et al. Randomized trial of short- versus long-course radiotherapy for palliation of painful bone metastases. J Natl Cancer Inst 2005;97:798-804. [Crossref] [PubMed]
44. Rades D, Dahm-Daphi J, Rudat V, et al. Is short-course radiotherapy with high doses per fraction the appropriate regimen for metastatic spinal cord compression in colorectal cancer patients? Strahlenther Onkol 2006;182:708-12. [Crossref] [PubMed]
45. Lutz S, Berk L, Chang E, et al. Palliative radiotherapy for bone metastases: an ASTRO evidence-based guideline. Int J Radiat Oncol Biol Phys 2011;79:965-76. [Crossref] [PubMed]
46. Guadagnolo BA, Liao KP, Elting L, et al. Use of radiation therapy in the last 30 days of life among a large population-based cohort of elderly patients in the United States. J Clin Oncol 2013;31:80-7. [Crossref] [PubMed]
47. Gripp S, Mjartan S, Boelke E, et al. Palliative radiotherapy tailored to life expectancy in end-stage cancer patients: reality or myth? Cancer 2010;116:3251-6. [Crossref] [PubMed]
48. Prognosis of Patient Evaluated for Palliative Radiotherapy (PROPER). Available online: https://clinicaltrials.gov/ct2/show/NCT03797755?term=TEACHH&cntry=HK&draw=2&rank=1.
49. Phung MT, Tin Tin S, Elwood JM. Prognostic models for breast cancer: a systematic review. BMC Cancer 2019;19:230. [Crossref] [PubMed]
50. Feng Q, May MT, Ingle S, et al. Prognostic Models for Predicting Overall Survival in Patients with Primary Gastric Cancer: A Systematic Review. Biomed Res Int 2019;2019:5634598 [Crossref] [PubMed]
51. Bradley A, Van Der Meer R, McKay CJ. A systematic review of methodological quality of model development studies predicting prognostic outcome for resectable pancreatic cancer. BMJ Open 2019;9:e027192 [Crossref] [PubMed]
52. Mahar AL, Compton C, Halabi S, et al. Personalizing prognosis in colorectal cancer: A systematic review of the quality and nature of clinical prognostic tools for survival outcomes. J Surg Oncol 2017;116:969-82. [Crossref] [PubMed]
53. Reid N. A Conversation with Sir David Cox. Statistical Science 1994;9:439-55. [Crossref]
54. Ng R, Kornas K, Sutradhar R, et al. The current application of the Royston-Parmar model for prognostic modeling in health research: a scoping review. Diagn Progn Res 2018;2:4. [Crossref] [PubMed]
55. Kao CY, Hung YS, Wang HM, et al. Combination of initial palliative prognostic index and score change provides a better prognostic value for terminally ill cancer patients: a six-year observational cohort study. J Pain Symptom Manage 2014;48:804-14. [Crossref] [PubMed]
56. Gwilliam B, Keeley V, Todd C, et al. Development of Prognosis in Palliative care Study (PiPS) predictor models to improve prognostication in advanced cancer: prospective cohort study. BMJ Support Palliat Care 2015;5:390-8. [Crossref] [PubMed]
57. Laird BJ, Kaasa S, McMillan DC, et al. Prognostic factors in patients with advanced cancer: a comparison of clinicopathological factors and the development of an inflammation-based prognostic system. Clin Cancer Res 2013;19:5456-64. [Crossref] [PubMed]