Accepted on 31 Jan 2020
Submitted on 11 Apr 2017
Work is important to a large number of cancer patients, as 42% of patients diagnosed with cancer are of working age. Only two-third of cancer survivors return to work after having finished cancer treatment (Mehnert, 2011). From a patients’ point of view, return to work is considered important for satisfaction, meaning, daily structure, social contacts, focus on other things than cancer and ‘not being regarded as a patient’ (Dutch Breast Cancer Federation, 2013). Returning to work following cancer is frequently experienced by patients as a sign of recovery (Kennedy, Haslam, Munir, & Pryce, 2007).
Individual differences have been found in the extent to which cancer survivors (do not) return to work (Van Muijen et al., 2013). Research has examined which factors may explain these individual differences. A review from 2013 into predictors of return to work in cancer survivors included 28 articles and categorized predictors in socio-demographic, disease-related and job-related domains (Van Muijen et al., 2013). A limited number of predictors for not returning to work was found: the level of physical exertion, treatment with chemotherapy or invasive surgery, an older age, a lower education and lower income (Van Muijen et al., 2013). Intriguingly, no psychosocial factors like stressful life-events, well-being, psychological symptoms, quality of life, or acceptance were included in these 28 prognostic studies as potential predictors, while cancer certainly has an emotional impact on a group of patients, which in turn may influence return to work. A meta-analysis of 10,071 oncology patients across 14 countries showed a 38% prevalence of emotional disorder according to DSM or ICD criteria (Mitchell, Ferguson, Gill, Paul, & Symonds, 2013). Apart from one study (Spelten et al., 2003), we are not aware of any literature on the prognostic value of anxiety and depression for the return to work in cancer patients.
Several models are known that aim to integrate different predictors of return to work in order to understand the concept of return to work after cancer (de Boer, Frings-dresen, & Feuerstein, 2016; Feuerstein et al., 2010; Mehnert, de Boer, & Feuerstein, 2013). Feuerstein et al. (Feuerstein et al., 2010) proposed a ‘cancer and work’ model to conceptualize barriers for return to work, which considered both clinical and workplace application. The first category in this model is health and well-being, the second major category is the presence and severity of symptoms such as fatigue, depression and anxiety, followed by levels of daily functioning as the third major category in the model. They reviewed the literature and concluded that prospective studies are limited and there is a clear need for such studies in this area. Since there is little knowledge about which factors in the first, second and third category are most important for returning to work, we decided to start with a Delphi study. We obtained consensus information from a group of cancer patients with psychological symptoms and a group of therapists experienced in working with cancer patients (“the Delphi sample”). We asked them which factors they thought to be important predictors of return to work.
The predictors as elicited during the Delphi study will subsequently be used in a quantitative study, with the aim to examine a prediction model of return to work. We used data from a subsample of cancer patients from a previous study on the evaluation of the psychological care at specialized psycho-oncology centres (POC) in the Netherlands (Garssen et al., 2016) (“the evaluation sample”). Factors that were important for returning to work as reported by therapists and patients (i.e. the Delphi sample) were included in a multivariate prediction model for return to work after psychological treatment in the evaluation sample.
The POC evaluation study was a naturalistic study with consecutive sampling from all seven specialized psycho-oncology institutions in the Netherlands. Cancer patients that applied for psychological care on referral of their doctor between September 2008 and March 2010 were asked informed consent. For more details on participant recruitment procedure, see Garssen et al. (Garssen et al., 2016). Participants who consented to participate filled in questionnaires at intake (T1, N = 384), after 3 months (T2, N = 167) and 9 months (T3, N = 146). Eligible participants were: (1) diagnosed with cancer (any type) and seeking psycho-oncological help, (2) able to complete questionnaires in Dutch. In addition, for the current study they had to be (3) employed in paid work, and (4) sick-listed at the start of psycho-oncological therapy. Concerning ethical approval, in the Netherlands, studies evaluating institutional services are not subject to the Medical Research involving Human Subjects Act and, therefore, approval from a Medical Ethics Committee was not required. The POC evaluation study was approved by the institutional boards of all seven participating POCs.
For the Delphi study we recruited two groups by purposive sampling: a group (n = 21; 61% women) of cancer patients with a mean age of 47.3 (SD = 14.7) years, and a group of therapists (n = 19; 79% women) aged 49.7 (SD = 11.4) years. Most patients and therapists at the Helen Dowling Institute received a paper questionnaire in their mailbox, all participants from other POCs filled out the questionnaire online via e-mail. Therapists were all working at one of the involved POCs and had substantial experience with psycho-oncological treatment.
We performed a Delphi study to obtain consensus on predictors for return to work. The Delphi process transforms personal opinions into group consensus (Hasson, Keeney, & McKenna, 2000; Sumsion, 1998). The Delphi participants scored the importance of all possible 20 variables for predicting work return at the end of treatment on a scale ranging from 0 (fully disagree) to 10 (fully agree) that were available from the evaluation sample. Variables were then selected for the prediction model if ≥75% of the participants rated the variable with a score ≥7 (Norder et al., 2012). This model was tested on data from the previous POC evaluation study, i.e. the evaluation sample (n = 384)(de Boer et al., 2016; Hasson et al., 2000; Norder et al., 2012).
The POC evaluation study was used as described in Garssen et al. (Garssen et al., 2016). The dataset and questionnaires were added to this manuscript separately to provide full disclosure of used resources.
Demographic characteristics (age, gender, education, children at home, work hours/week, disability or employment benefits) and medical characteristics (date of cancer diagnosis, cancer site, cancer stage (metastatic no, yes), cancer treatment, health care utilization, and comorbidity) were measured at intake (T1). Health care utilization was assessed by questions on visits to general practitioners, use of medication in the past four weeks, and treatment by therapists (Trimbos and iMTA questionnaire on Costs associated with Psychiatric illness). Comorbid disease was investigated with an item asking for disorders other than cancer.
After selection by the Delphi participants, predictor variables were included in a multivariate logistic regression model with return to work nine months after treatment (no = 0, yes = 1) as outcome variable. The logistic regression equation was then applied to predict the probability of return to work for each patient. The predicted probabilities were compared to the observed probability of return to work by using the Hosmer-Lemeshow (H-L) goodness of fit test (Steyerberg, 2009; Steyerberg et al., 2010). A non-significant (p > .05) H-L test result means that predicted probabilities do not deviate significantly from the observed probabilities. Thus, a non-significant H-L test result indicates that the prediction model adequately predicts return to work after treatment.
We wanted to investigate the prediction model as a prognostic tool for return to work after treatment. For this purpose, we must know whether the prediction model discriminates between patients with and without return to work after treatment. Therefore, the probabilities predicted by the logistic regression model were included in receiver operating characteristic (ROC) analysis. ROC analysis addresses each probability as a cutoff point and estimates the proportion of patients with return to work having a predicted probability higher than the cutoff point (sensitivity) and the proportion of patients without return to work having a predicted probability lower than the cutoff point (specificity). The ROC-curve plots sensitivity (on the y-axis) against 1–specificity (on the x-axis). The area under the ROC-curve (AUC) is a measure for discrimination between patients with and without return to work after treatment (Steyerberg, 2009). An AUC 0.9–1.0 represents excellent, 0.8–0.9 good, 0.7–0.8 fair, 0.7–0.6 poor, and <0.6 failing discrimination. For example, an AUC = 0.75 indicates that the prediction model will correctly discriminate between patients with without return to work in 75% of the cases.
Discrimination will be different in other samples of cancer patients, because the prediction model is based on a logistic regression equation fitted to the data of the patients in the study population. To get an idea of the prediction model’s discriminative ability in other samples of cancer patients, we used bootstrapping. This data simulation technique creates bootstrap samples by randomly drawing patients from the study population. Patients are randomly drawn with replacement, and therefore some can be drawn into a bootstrap sample twice or more frequently while other patients may not be drawn. Consequently, each bootstrap sample has a different data structure (Steyerberg, 2009; Steyerberg et al., 2001, 2010). In the present study, 250 bootstrap samples were drawn to validate the prediction model’s discrimination. The validated AUC represents discrimination that can be expected in other samples of cancer patients and thus increases the external validity of the study results.
The prediction model was developed with IBM SPSS Statistics for Windows, version 20.0 (IBM Corp., 2012) and validated in R (R Core Team, 2017) by using Harrell’s Regression Modeling Strategies (rms) package, version 3.2-0 (Harrell Jr, 2013).
Overall differences were small between therapists and patients, though patients valued domestic and social functioning, well-being and stressful life-events more important as predictors of return to work at T3, than therapists (Table 1).
Table 1
Delphi Study Results.
| Variable | Total (N = 40) | Patients (n = 21) | Therapists (n = 19) | Difference |
|---|---|---|---|---|
| Depressive symptoms | 9.00 (8.00–9.00) | 9.00 (8.00–9.00) | 8.00 (8.00–9.25) | P = 0.669 |
| Fatigue | 9.00 (8.00–9.00) | 9.00 (8.00–9.00) | 9.00 (8.00–10.00) | P = 0.667 |
| Anxiety | 8.00 (8.00–9.00) | 9.00 (8.00–9.00) | 8.00 (7.75–9.00) | P = 0.499 |
| Quality of life | 8.00 (8.00–9.00) | 8.00 (8.00–9.00) | 8.00 (6.00–8.00) | P = 0.168 |
| Well-being | 8.00 (8.00–9.00) | 9.00 (8.00–9.00) | 8.00 (7.00–9.00) | P = 0.043 |
| Social functioning | 8.00 (8.00–9.00) | 8.50 (8.00–9.00) | 7.00 (6.00–9.00) | P = 0.036 |
| Comorbidity | 8.00 (7.00–9.00) | 8.00 (7.00–9.00) | 8.00 (7.00–9.00) | P = 0.934 |
| Disease cognitions | ||||
| helplessness | 8.00 (7.00–9.00) | 8.00 (7.00–9.00) | 8.00 (7.00–9.00) | P = 0.911 |
| acceptation | 8.00 (7.00–9.00) | 9.00 (7.00–9.00) | 8.00 (7.00–9.00) | P = 0.341 |
| Mastery | 8.00 (7.00–9.00) | 8.00 (7.00–9.00) | 9.00 (8.00–9.00) | P = 0.500 |
| Stressful life-events | 8.00 (7.00–9.00) | 8.00 (8.00–9.00) | 8.00 (7.00–8.00) | P = 0.042 |
| Domestic functioning | 8.00 (7.00–9.00) | 8.00 (8.00–9.00) | 7.00 (6.00–8.00) | P = 0.023 |
| Attitude spouse | ||||
| motivating | 7.00 (6.00–8.00) | 8.00 (3.50–8.50) | 7.00 (6.00–8.00) | P = 0.510 |
| protective | 7.00 (6.00–8.00) | 8.00 (5.00–8.00) | 7.00 (6.00–8.00) | P = 0.537 |
| Use of medication | ||||
| analgesics | 7.00 (6.00–8.00) | 7.00 (6.00–8.00) | 6.00 (5.00–8.00) | P = 0.375 |
| tranquilizers | 7.00 (5.75–8.00) | 7.00 (4.25–8.00) | 7.00 (6.00–8.00) | P = 0.806 |
| sedatives | 7.00 (5.75–8.00) | 7.00 (5.25–8.75) | 7.00 (5.75–8.00) | P = 0.613 |
| antidepressants | 6.50 (4.00–8.00) | 7.00 (3.25–8.00) | 6.00 (4.75–8.25) | P = 0.740 |
| Age | 7.00 (5.00–8.00) | 7.00 (5.50–9.00) | 5.00 (4.00–8.00) | P = 0.101 |
| Children at home | 6.50 (4.25–7.75) | 7.00 (5.00–8.50) | 6.00 (3.00–7.00) | P = 0.052 |
| Therapeutic sessions | ||||
| number | 6.00 (5.00–8.00) | 7.00 (5.00–8.00) | 6.00 (5.00–7.00) | P = 0.210 |
| frequency | 6.00 (5.00–8.00) | 7.00 (4.00–8.00) | 5.00 (5.00–6.00) | P = 0.134 |
| type (individual or group) | 6.00 (2.50–8.00) | 7.00 (2.00–8.00) | 5.50 (2.75–7.00) | P = 0.558 |
| Time since diagnosis | 6.00 (4.00–8.00) | 6.00 (4.00–8.00) | 6.00 (4.00–8.00) | P = 0.773 |
| Previous psychological treatments | 6.00 (4.25–7.00) | 7.00 (4.00–8.00) | 5.00 (5.00–7.00) | P = 0.245 |
| Education | 6.00 (2.00–8.00) | 7.00 (2.50–8.00) | 5.00 (2.00–7.00) | P = 0.293 |
| Treatment by other therapists | 5.00 (2.00–7.00) | 6.00 (2.00–7.00) | 5.00 (3.00–7.00) | P = 0.966 |
Note: The table shows median (25th percentile–75th percentile) scores for potential predictors of sickness absence nine months after intake and the difference in opinion between patients and therapists (Mann-Whitney U-test); bold font indicates group consensus, which is achieved when >75% of both groups, patients and therapists, rated the variable as important >7). In the final column p-values are reported on the difference between therapist and patient scores.
Using the Delphi approach, consensus (i.e. >75% of both patients and therapists rated as important >7) was reached on the importance of 12 predictor variables. Earlier analyses showed considerable co-occurrence of depression, anxiety and fatigue in this group of patients (Zhu et al., 2016). Therefore, these three variables were merged into one category, that is, psychological symptoms. As a result, 10 variables were selected for the prediction model: psychological symptoms, quality of life, comorbidity, helplessness, acceptance, mastery, stressful life-events, well-being, domestic functioning, and social functioning.
At intake, 219 of 384 study participants were employed in a paid job, of whom 174 (79%) were sick-listed at the start of therapy. The data of these 174 participants were used to develop the model for predicting return to work at T3. At T3 119 (68%) of these patients did no longer report sickness absence from work. Table 2 shows the characteristics of the participants (166 women and 53 men). Their mean age was 48.7 (SD = 8.6) years and they worked on average 29.0 (SD = 9.8) hours per week. Most of them had secondary vocational (24%), higher vocational (37%), or academic education (18%). Forty-six participants reported comorbid disease, particularly pulmonary (9%), neurologic (migraine 9% epilepsy 1%), gastrointestinal (5%) and endocrine (thyroid 4%, diabetes 2%) disorders.
Table 2
Study Population Characteristics.
| Mean (SD) | n (%) | |
|---|---|---|
| Cancer diagnosis | ||
| breast | 109 (50) | |
| leukemia/lymphoma | 24 (11) | |
| gastrointestinal | 20 (9) | |
| head/neck | 17 (8) | |
| lung | 14 (6) | |
| gynaecologic | 12 (5) | |
| other | 23 (11) | |
| Metastatic | 77 (36) | |
| Years since diagnosis | 2.3 (4.1) | |
| Years since end of treatment | 1.0 (2.6) | |
| Still under treatment | 99 (45) | |
| Illness cognitions | ||
| helpless (range 6–24) | 12.8 (4.0) | |
| acceptance (range 6–24) | 13.0 (3.7) | |
| finding benefits (range 6–24) | 13.8 (4.2) | |
| Use of medication | ||
| analgetics | 84 (38) | |
| tranquilizers | 35 (16) | |
| sedatives | 47 (22) | |
| anitdepressants | 15 (7) | |
| Previous psychologic treatment | ||
| no | 126 (58) | |
| yes | 93 (42) | |
| Fatigue (range 8–56) | 35.5 (11.9) | |
| Anxiety (range 6–24) | 14.4 (3.5) | |
| Depressive symptoms (range 0–48) | 15.0 (8.6) | |
| Well-being (range 0–10) | 6.0 (1.6) | |
| Mastery (range 7–35) | 21.2 (4.7) | |
| Domestic functioning (range 4–20) | 10.2 (3.7) | |
| Social functioning (range 8–40) | 20.1 (7.3) | |
| Life-events past year | ||
| 0 | 56 (26) | |
| 1 | 74 (34) | |
| 2 | 43 (20) | |
| 3 or more | 46 (21) | |
Note: (N = 219).
None of the predictor variables valued important by patients and therapists in the Delphi procedure was significantly associated with return to work (Table 3). The multivariate logistic regression model including the variables selected by the Delphi panel explained 10% of the variance in return to work.
Table 3
Prediction Model Development.
| B (SE) | Wald | OR (95% CI) | p-value | |
|---|---|---|---|---|
| Intercept | –4.501 (2.881) | |||
| Psychological symptoms | 0.008 (0.018) | 0.177 | 1.008 (0.972–1.045) | P = 0.674 |
| Quality of life | –0.300 (0.190) | 2.507 | 0.741 (0.510–1.075) | P = 0.113 |
| Comorbidity | 0.136 (0.486) | 0.078 | 1.145 (0.442–2.970) | P = 0.780 |
| Helplessness | 0.086 (0.068) | 1.615 | 1.090 (0.954–1.245) | P = 0.204 |
| Acceptation | 0.008 (0.064) | 0.014 | 1.008 (0.890–1.141) | P = 0.905 |
| Mastery | –0.062 (0.050) | 1.563 | 0.940 (0.852–1.037) | P = 0.211 |
| Stressful life-events | 0.149 (0.127) | 1.375 | 1.161 (0.905–1.489) | P = 0.241 |
| Well-being | –0.020 (0.037) | 0.283 | 0.981 (0.913–1.054) | P = 0.594 |
| Domestic functioning | –0.109 (0.058) | 3.463 | 0.897 (0.800–1.005) | P = 0.063 |
| Social functioning | –0.024 (0.034) | 0.509 | 0.976 (0.914–1.043) | P = 0.475 |
Note: The table shows regression coefficients (B) and related standard errors (SE), Wald-statistics ([B/SE]2), odds ratios (OR) and related 95% confidence intervals (CI), and significance of associations between predictor variables and sickness absence nine months after intake.
The H-L test p-values was 0.41, which was non-significant. This indicated that the prediction model adequately predicted the probability of return to work. The validated AUC was 0.652 (95% CI 0.553–0.751), which means that the prediction model correctly discriminated between cancer patients with and without return to work in 65.2% of the cases. Although significantly better than discrimination by chance (i.e., AUC = 0.50), discrimination of this magnitude is poor (Figure 1) and not sufficient to use the prediction model as a prognostic tool for return to work after treatment (Figure 1).
Discrimination graph. The figure shows the receiver operating characteristic (ROC) curve. Discrimination improves with the area under the ROC curve; the diagonal indicates no discrimination above chance. the circle denotes a 49% cut-off risk with sensitivity 0.59 and specificity 0.74.
Using a Delphi approach, cancer patients who sought treatment for psychological symptoms and therapists offering psycho-oncological treatment agreed that the following factors were important for (not) returning to work: comorbidity, psychological symptoms (anxiety, depression and fatigue), well-being, quality of life, helplessness, acceptance, mastery, stressful life-events, domestic and social functioning. This is in line with the proposed Cancer and Work model in which well-being, psychological symptoms, fatigue, and functioning are considered most important for working with cancer (Feuerstein et al., 2010). Using data from a large national study in cancer patients in which psychological treatment was evaluated, results from multivariate analyses including the predictors identified by the Delphi participants showed that none of the predictors was significantly associated with return to work after nine months of receiving psychological treatment.
We found that, although patients and therapists agreed that factors like well-being, psychological symptoms, fatigue, and functioning are important for returning to work, none of the identified predictors was significantly associated with return to work after nine months. It might be that predictors for returning to work differ too much between patients, so that for one individual patient well-being may play a role, but not for the whole group of patients. This would imply that it is not feasible to make a prediction model for all patients at baseline. Another explanation may be that there was not enough variance in predictors at baseline (i.e. start of psychological treatment): the level of psychological symptoms for example was high for most patients. If we had analyzed data over a longer period of time and assuming that the level of psychological symptoms diminishes in a group of patients, this might have taught us more about the predictive value of psychological symptoms for returning to work in the longer term. However, the scope of the current study was to identify predictors of return to work among cancer patients, based on patients’ characteristics at the start of psycho-oncological therapy.
Studies have found that about 64% of cancer survivors return to work (with a wide range between 24% and 94%), and that returning to work is more difficult for patients with advanced stages and/or more intensive cancer treatment (Mehnert et al., 2013). The 68% return to work rate found in the present study is in line with these findings, even though 36% of the patients had advanced stage cancer and most patients had clinical levels of depression and/or anxiety and/or fatigue at baseline. This might indicate that the study included cancer patients who were very eager to resume work.
Quantitative studies showed that fatigue, anxiety and depressed mood often co-occur (Zhu et al., 2016), because these symptoms are causally related (Cramer, Waldorp, van der Maas, & Borsboom, 2010). We therefore merged the scores on depression, anxiety and fatigue into one variable ‘psychological symptoms’, to reduce the number of predictor variables and limit the risk of statistical overfitting. It would be interesting in future studies to differentiate between these symptoms when analyzing the effects on return to work. In addition, it would be important to know whether patients acknowledge fatigue, anxiety, depression equally important as a reason not to go back to work, or that for instance fatigue is more considered to be a legitimate reason.
The strength of our study lies in the fact that we used the knowledge of experts (both cancer patients and therapists) and focused on the role of psychosocial factors in predicting return to work, which have not been studied before among cancer patients who applied for psycho-oncological treatment. Several limitations need to be considered when interpreting the findings of this study. The most important limitation of this study is that we used a small sample of cancer patients, which restricted the statistical power of the multivariate analyses. The limited power may be a reason that none of the 10 predictors in the prediction model was found to be significant. Predictions by models developed in small samples are difficult to generalize to other populations of cancer patients. Therefore, predictions were internally validated in 250 bootstrap samples (Steyerberg et al., 2001). The validated AUC presented in this study is indicative of the discrimination that can be expected when the prediction model is applied to new samples of cancer patients. Another limitation was that we assessed return to work by asking about sickness absence in the past four weeks. It is known that this is a time period that most people can reliably remember. It is possible, however, that a patient did resume work during therapy, but got sick again the month before T3. Furthermore, information on causes of sickness absence at the end of treatment was not available from the POC evaluation study Future studies should gather information on sickness absence during the whole period of treatment.
In sum, the current study showed that though patients and therapists identified multiple predictors for return to work as important, none of these predictors were significantly associated with return to work. Moreover, a prediction model which correctly discriminated between cancer patients with and without return to work, did not sufficiently discriminate in order to use the prediction model as a prognostic tool for return to work after treatment. Following patients for a longer time before and after treatment could teach us more about the predictive value of psychological symptoms for returning to work in the longer term.
The additional files for this article can be found as follows:
Analysis and outputOutput analyse Corne Roelen Marije van der Lee Return to work. DOI: https://doi.org/10.5334/hpb.4.s1
Data file delphi studyDatabestand werkhervatting. DOI: https://doi.org/10.5334/hpb.4.s2
The authors wish to thank the Ingeborg Douwes Foundation and Pink Ribbon for funding the POC evaluation study; Rosalie van Woezik for her help with revising the article; Judith Dusseldorp for her help with the Delphi Study and all patients and therapists for their cooperation.
The authors have no competing interests to declare.
Beurskens, A. J. H. M. J., Bultmann, U., Kant, I. J., Vercoulen, J. H. M. M. H., Bleijenberg, G., & Swaen, G. M. H. (2000). Fatigue among working people: validity of a questionnaire measure. Occupational Environmental Medicine, 57(1351–0711 (Print)), 353–357. DOI: https://doi.org/10.1136/oem.57.5.353
Cramer, A. O. J., Waldorp, L. J., van der Maas, H. L. J., & Borsboom, D. (2010). Comorbidity: a network perspective. The Behavioral and Brain Sciences, 33(2–3), 137–150; discussion 150–93. DOI: https://doi.org/10.1017/S0140525X09991567
de Boer, A., Frings-dresen, M., & Feuerstein, M. (2016). Improving Return to Work in Cancer Survivors. In Handbook of Return to Work (pp. 481–503). Boston, MA: Springer. DOI: https://doi.org/10.1007/978-1-4899-7627-7
de Bruin, E. D., van Dijk, M. V., & Duivenvoorden, H. J. (1996). Assessing adjustment to cancer: the health and disease inventories (HDI). Lisse: Swets and Zeitlinger.
Dutch Breast Cancer Federation. (2013). B-force questionnaire: “Borstkanker en werk: wat is jouw ervaring? – Deel 1.” Retrieved from https://bforce.nl/nl/borstkanker-en-werk-wat-jouw-ervaring-deel-1
Evers, A. W. M., Kraaimaat, F. W., van Lankveld, W., Jongen, P. J. H., Jacobs, J. W. G., & Bijlsma, J. W. J. (2001). Beyond unfavorable thinking: The Illness Cognition Questionnaire for chronic diseases. Journal of Consulting and Clinical Psychology, 69(6), 1026–1036. DOI: https://doi.org/10.1037/0022-006X.69.6.1026
Feuerstein, M., Todd, B. L., Moskowitz, M. C., Bruns, G. L., Stoler, M. R., Nassif, T., & Yu, X. (2010). Work in cancer survivors: a model for practice and research. Journal of Cancer Survivorship, 4(4), 415–437. DOI: https://doi.org/10.1007/s11764-010-0154-6
Garssen, B., Van der Lee, M., Van der Poll, A., Ranchor, A. V., Sanderman, R., & Schroevers, M. J. (2016). Characteristics of patients in routine psycho-oncological care, and changes in outcome variables during and after their treatment. Psychology and Health, 31(10), 1237–1254. DOI: https://doi.org/10.1080/08870446.2016.1204447
Harrell, F. E., Jr. (2013). rms: Regression Modeling Strategies. R package version 3.2-0. City.
Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32(4), 1008–1015. DOI: https://doi.org/10.1046/j.1365-2648.2000.t01-1-01567.x
IBM Corp. (2012). SPSS Statistics for Windows, Version 20.0. Armonk, NY: IBM Corp.
Kennedy, F., Haslam, C., Munir, F., & Pryce, J. (2007). Returning to work following cancer: a qualitative exploratory study into the experience of returning to work following cancer. European Journal of Cancer Care, 16(16), 17–25. DOI: https://doi.org/10.1111/j.1365-2354.2007.00729.x
Mehnert, A. (2011). Employment and work-related issues in cancer survivors. Critical Reviews in Oncology/Hematology, 77(2), 109–130. DOI: https://doi.org/10.1016/j.critrevonc.2010.01.004
Mehnert, A., de Boer, A., & Feuerstein, M. (2013). Employment challenges for cancer survivors. Cancer, 119(S11), 2151–2159. DOI: https://doi.org/10.1002/cncr.28067
Millar, K., Purushotham, A. D., McLatchie, E., George, W. D., & Murray, G. D. (2005). A 1-year prospective study of individual variation in distress, and illness perceptions, after treatment for breast cancer. Journal of Psychosomatic Research, 58(4), 335–342. DOI: https://doi.org/10.1016/j.jpsychores.2004.10.005
Mitchell, A. J., Ferguson, D. W., Gill, J., Paul, J., & Symonds, P. (2013). Depression and anxiety in long-term cancer survivors compared with spouses and healthy controls: a systematic review and meta-analysis. The Lancet Oncology, 14(8), 721–732. DOI: https://doi.org/10.1016/S1470-2045(13)70244-4
Norder, G., Roelen, C. A. M., van Rhenen, W., Buitenhuis, J., Bültmann, U., Anema, J. R., Kivimäki, M., et al. (2012). Predictors of Recurrent Sickness Absence Due to Depressive Disorders – A Delphi Approach Involving Scientists and Physicians. PLoS ONE, 7(12), e51792. DOI: https://doi.org/10.1371/journal.pone.0051792
Pearlin, L. I., & Schooler, C. (1978). The Structure of Coping. Journal of Health and Social Behavior, 19(1), 2. DOI: https://doi.org/10.2307/2136319
R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.r-project.org
Schroevers, M. J., Sanderman, R., van Sonderen, E., & Ranchor, A. V. (2000). The evaluation of the Center for Epidemiologic Studies Depression (CES-D) scale: Depressed and Positive Affect in cancer patients and healthy reference subjects. Quality of Life Research, 9(9), 1015–1029. DOI: https://doi.org/10.1023/A:1016673003237
Spelten, E. R., Verbeek, J. H. A. M., Uitterhoeve, A. L. J., Ansink, A. C., van der Lelie, J., de Reijke, T. M., Sprangers, M. A. G., et al. (2003). Cancer, fatigue and the return of patients to work – a prospective cohort study. European Journal of Cancer, 39, 1562–1567. DOI: https://doi.org/10.1016/S0959-8049(03)00364-2
Steyerberg, E. W. (2009). Clinical prediction models: a practical approach to development, validation and updating. New York: Springer. DOI: https://doi.org/10.1007/978-0-387-77244-8
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J., Eijkemans, M. J., Vergouwe, Y., & Habbema, J. D. F. (2001). Internal validation of predictive models: Efficiency of some procedures for logistic regression analysis. Journal of Clinical Epidemiology, 54(8), 774–781. DOI: https://doi.org/10.1016/S0895-4356(01)00341-9
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Kattan, M. W., et al. (2010). Assessing the Performance of Prediction Models. Epidemiology, 21(1), 128–138. DOI: https://doi.org/10.1097/EDE.0b013e3181c30fb2
Sumsion, T. (1998). The Delphi Technique: An Adaptive Research Tool. The British Journal of Occupational Therapy, 61(4), 153–156. DOI: https://doi.org/10.1177/030802269806100403
van der Lubbe, P. M. (1995). De ontwikkeling van de Groningse Vragenlijst over Sociaal Gedrag (GVSG)[The development of the Groningen Questionnaire about Social Behavior (GQSB)]. Groningen: Rijksuniversiteit Groningen.
Van Muijen, P., Weevers, N. L. E. C., Snels, I. A. K., Duijts, S. F. A., Bruinvels, D. J., Schellart, A. J. M., & van der Beek, A. J. (2013). Predictors of return to work and employment in cancer survivors: A systematic review. European Journal of Cancer Care, 22(2), 144–160. DOI: https://doi.org/10.1111/ecc.12033
Zhu, L., Ranchor, A. V., van der Lee, M., Garssen, B., Almansa, J., Sanderman, R., & Schroevers, M. J. (2016). Co-morbidity of depression, anxiety and fatigue in cancer patients receiving psychological care. Psycho-Oncology. DOI: https://doi.org/10.1002/pon.4153
Health Psychology Bulletin has blind peer review, which is unblinded upon article acceptance. The editorial history of this article can be downloaded here:
PR File 1Questionnaire Delphi research version client. DOI: https://doi.org/10.5334/hpb.4.pr1
PR File 2Questionnaire Delphi research version healthcare provider. DOI: https://doi.org/10.5334/hpb.4.pr2