Stellungnahme zu „mistletoe therapy in oncology“ (cochrane-review 2008)

Comment on:
Mistletoe therapy in oncology (Cochrane-Review 2008)

(Horneber MA, Bueschel G, Huber R, Linde K, Rostock M: Mistletoe in oncology (Review).
2008 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd)
The Cochrane-Review ‘Mistletoe therapy in oncology’ (2008) concludes that the available
evidence is not sufficient to support mandatory administration of mistletoe therapy. This
summation will no doubt generally find agreement. – However, the additional conclusion that
trial evidence is generally weak requires critical examination.
The Cochrane-Review is neither complete nor updated, the formal evaluation procedure is
inadequate, and the factual assessments are not consistent and often incorrect. – Appropriate
correction of the assessments leads to a different overall picture of the mistletoe trials.
(For details see www.mistel-therapie.de).

How complete and updated is the Cochrane-Review?
The Cochrane-Review covers 21 randomized trials (RCTs). One HTA report and two system-
atic reviews are also discussed.
The following relevant publications were not taken into account: 9 RCTs investigating sur-
vival time, tumour behaviour and quality of life; 1 re-analysis of an RCT; 1 meta-analysis; 1
systematic review; 1 HTA report. The publications are distributed as follows:
- published 1987 [1]: 1 RCT
- published 1999 [2]: 1 RCT
- published 2005 [3]: 1 RCT
- published 2006 [4-7]: 1 RCT, 1 re-Analysis of an RCT, 1 HTA report
- published 2007 [8-12]: 5 RCTs und 1 systematic review
- published 2008 [13]: 1 meta-analysis.
(For further details, see www.mistel-therapie.de)
The Cochrane-Review is therefore incomplete and outdated.
The formal procedure for evaluation of study quality
The procedure chosen for evaluation of study quality in this Cochrane-Review is inadequate:
A number of formal evaluation scores are available. Different scores yield different results
when applied to identical studies (see e.g. [14]). The Cochrane-Review employs two scores
(the Jadad score and the Delphi list) which give the greatest weight to blinding of the study
intervention: blinding is allocated 2 points (40%) of the maximum 5 Jadad points, and 3
points (30%) of the maximum 9 Delphi points. By selecting these particular evaluation scales,
a poor performance of the mistletoe trials was predetermined, irrespective of how good or bad
they actually are. The reason is: Subcutaneous mistletoe injections regularly cause painful
swelling and reddening. Therefore, in principle, this therapy cannot be blinded. When blind-
ing is attempted, unblinding certainly occurs. (This has been repeatedly investigated and con-
firmed in clinical trials. For this reason many mistletoe trials refrain from using pro-forma
blinding.) Consequently, the Cochrane-Review disqualifies all mistletoe trials on this point,
including the trials formally conducted as double-blind trials. But the dilemma is not men-
tioned: Namely that trials investigating subcutaneous mistletoe therapy will, in principle, per-
form poorly in the evaluation scores used by the Cochrane-Review (because blinding is im-possible). The formal evaluation procedure chosen for this review is therefore inadequate. It may be inconvenient that mistletoe therapy is so incompatible with blinding. However, the same problem arises with other therapy forms such as physiotherapy, surgery and psychother-apy. It would be absurd if all these treatments should be disqualified a priori. The Cochrane-Review should have at least acknowledged that the evaluation procedure did not allow for mistletoe studies to be classified as a high-quality research. Furthermore, the general requirement of blinding (of patient, physician and observer) has bi-zarre consequences: It assumes, amongst other things, that a physician can only reliably de-termine a patient’s time of death if he or she does not know whether the patient received mis-tletoe therapy or not (i.e. is blinded). Inconsistencies in evaluation
For evaluation of study quality, cut-off values were set arbitrarily. Following the subtraction of “blinding points” mentioned above, no mistletoe trial could perform positively on the Jadad score, and on the Delphi list the subtraction of one additional point was sufficient for the trial to receive a poor quality rating. Therefore the remaining methodological evaluations require particular accuracy and balancing Furthermore, errors, lack of transparency, subjective ten-dencies, lack of recourse to the author of the trial etc. can quickly and unjustly tip the balance towards a positive or negative trial rating. Thus the question arises as to why, in the Cochrane Review, the trials with negative results (i.e. no superiority of mistletoe therapy over the control group) were so positively evaluated, and why the weaknesses of these studies, in part severe limitations, were not presented or dis-cussed. And why, on the other hand, trials with positive results (i.e. where mistletoe therapy was superior) were disqualified – sometimes with regard to aspects which they dealt with equally well or better than the trials with negative results. Or why trials with positive results were disqualified because some study features were not clearly described in the primary pub-lication, when the respective features were described in a secondary publication or trial report, or could have been identified by contacting the study authors. These issues will be illustrated with the following examples: Example 1: Handling of drop-outs in the Piao (2004) [15] und Steuer-Vogt (2001) [16;17]
trials.
Assessment in the Cochrane-Review: The Piao trial, which shows superiority of mistletoe
therapy, is criticised on the following grounds that not all patients completed the trial (“drop-
outs” are ubiquitous in almost all clinical trials), that according to the Cochrane review, the
reason for the drop-outs was not mentioned, that the number of drop-outs was unbalanced in
the two groups, and that these drop-outs were not part of the trial evaluation. In contrast, the
Steuer-Vogt trial, where mistletoe therapy showed no superiority, received a plus point with
respect to drop-outs.
The facts:
1.) Both studies were evaluated according to the intention-to-treat-principle, i.e. all patients were evaluated as if they actually received the treatment assigned by randomization, inde-pendently of whether this actually was the case or not. This is the contemporary standard evaluation technique. 2.) Furthermore, the drop-out rate in the Piao trial was only 4%. This rate is so marginal that any influence of dropout on the trial result would appear to be theoretical. In the Steuer-Vogt trial, the drop-out rate for survival was 9%; for quality of life, it was 32% and 53% after one and two years, respectively. In fact, the Piao study states the reasons for drop-outs very clearly (presented in a biometric trial report available on the Internet). In the Steuer-Vogt trial, however, the dropouts are for the most parts not even listed separately for the mistletoe and control groups. Therefore, contrary to the Cochrane-Review’s statements, “balance” of drop-outs in the two groups of the Steuer-Vogt trial cannot be assessed at all. The Cochrane-Review criticises the Piao trial for failure to include data of excluded patients in the quality of life evaluation. This does not make sense: quality of life can only be evalu-ated for available questionnaires, as opposed to outcomes such as survival time, time of death and tumour behaviour which can also be assessed outside of trial participation. In the litera-ture this problem is well-known and is still unsolved (see e.g. [18;19]). Therefore, there is no convincing factual basis for an a priory disqualification of the Piao trial in this respect. More-over, the same problem is also present in the Steuer-Vogt trial: With regard to the trial’s pri-mary outcome measure, a subset of the drop-outs (namely 4%: 495 randomized, 477 assessed) was also not included in the analyses, although this would have been technically possible—in contrast to Piao’s 4%. Furthermore, the 32% and 53% drop-out-rate concerning quality of life were not considered in the respective analyses of the Steuer-Vogt trial. In conclusion: With regard to handling of dropouts, the Steuer-Vogt trial is definitely not
superior to the Piao trial. At best, the two trials are comparable in this respect—but under
close scrutiny the Steuer-Vogt trial clearly comes off worse because of its high dropout rates
for quality of life assessment. Nonetheless, the Steuer-Vogt trial receives a plus point in this
regard and the Piao trial a minus point. This is not in accordance with the factual data and
creates an impression of biased assessment.
Example 2: Assessment of prognostic comparability in the Grossarth (2001a/b) [20;21]
and Kleeberg (2004) [22] trials.
Assessment in the Cochrane-Review: The assessment of comparability of trial groups regard-
ing prognostic factors results in a minus point for the Grossarth trials (that show superiority of
mistletoe therapy), in contrast to a plus point for the Kleeberg trial (no superiority).
The facts:
In the breast cancer trial by Grossarth, the patients (Mamma Ca., N>1, M=O) were matched systematically into pairs according to stage (IIIA and IIIB), menopause status, chemotherapy, radiation therapy, hormone therapy, age, and year of first diagnosis; the other Grossarth trial applied similar matching criteria. Accordingly, patients were comparable concerning these prognostic parameters. After matching had been completed, each patient in a matched and comparable pair was randomized to either mistletoe or control group. It is inexplicable why the Cochrane-Review assessed comparability of these studies as “not guaranteed”. The Kleeberg trial, on the other hand, had marked differences in gender distribution: Women were clearly under-represented in the mistletoe group (35.6% of patients) compared to the control group (46.5%), a relevant difference for survival time. For example, among patients in Stage IIb (half of the study patients: 49% and 48% respectively), women had a highly signifi-cant better survival (p=0.0009) than men. This uneven gender distribution between mistletoe and control groups disadvantages the mistletoe group. If the imbalance had been taken into account in the final multivariate analyses, a positive result for survival time in the mistletoe group might have occurred. In conclusion: Regarding comparability of prognostic factors, the Kleeberg trial is not at all
superior to the Grossarth trial. In fact, the Kleeberg trial shows a markedly uneven distribu-
tion relevant for survival time. It is inexplicable why the Kleeberg trial received a plus point
and the Grossarth trial a minus point in this respect. This assessment creates an impression of
bias.

Example 3: Blinding in the Borrelli (1999) [23] and Dold (1991) [24] trials
Assessment in the Cochrane-Review: In the Borrelli trial (which showed superiority of mis-
tletoe therapy), according to Dr. Borrelli (first author), the physicians and patients were
blinded towards mistletoe therapy. Nevertheless the trial was evaluated as “not blinded”.
On the other hand, in the Dold trial (which shows no superiority of mistletoe therapy concern-ing tumour remission and survival time) the physicians and patients were not blinded towards mistletoe therapy—this trial was devised and conducted as a non-blinded placebo-controlled trials. Yet, the Cochrane-Review allocates the Dold trial a plus point in the Delphi list regard-ing “adequate blinding of patients”. This is inexplicable. Furthermore, according to the Cochrane review’s description of the Dold trial, there was sup-posedly a genuine blinding of the “outcome assessor”. This claim also does not reflect the facts: The trial report states that the physicians were not blinded, and that these (not blinded) physicians collected the outcome data and entered them into the case report forms. Further inconsistencies
Grossarth trials [20;21], Borrelli trial [23]: The Cochrane-Review subtracts a quality point
for each of these trials because randomization was not concealed. Allocation concealment is
intended to prevent the physicians enrolling patients from guessing or estimating the group to
which each patient will be randomized, knowledge of which could lead to manipulation of
patient enrolment. The fact is, however: In the Borrelli trial randomization was carried out by
an independent person who had no contact with the enrolling physicians (= definition crite-
rion for “concealment” according to the Delphi list). In the Grossarth trials each unit to be
randomized (the matched patient pair) was already fully included in the trial before randomi-
zation. Because randomization took place after enrolment had been completed, randomization
was certainly concealed from the personnel enrolling patients. – Therefore subtracting a point
for supposed lack of concealment of allocation was indisputably incorrect for the Borrelli trial
and at least questionable regarding the Grossarth trials.
In addition, the Cochrane-Review claims that in the Grossarth trials it is unclear whether the
patients had consented to participation. The fact is: All patients were informed about the trial.
(“Consent to participate in the study was assumed after comprehensive information about the
study objectives and the study design and the patient’s explicit expression of willingness to
participate” [4]. – Randomization procedures were carried out according to Zelen’s Random-
ized Consent Design.)
The Cochrane-Review restricted inclusion criteria to randomized trials. This limitation is not
without problems, as can be seen with the Grossarth trials: Both Grossarth RCTs considered
in the Cochrane-Review (as well as other mistletoe RCTs by Grossarth) are part of a large
epidemiological cohort study comprising approximately 10,000 patients, within which several
large prospective matched-pair studies are embedded [4;5;8-10;13;20;25]. RCTs normally
have a very limited external validity, i.e. they provide very little evidence of the effectiveness
of a therapy in practice. For example, RCTs encompass highly selected patients (mostly less
than 1% of the relevant diagnosis group [26]), they exclude the most relevant concomitant
diseases; and RCTs differ markedly from real-world treatment conditions with respect to di-
agnosis, therapy, adjunctive therapies and follow-up [26;27]. The Grossarth cohort study, on
the other hand, is characterised by an extremely high external validity. The study does not
interfere with the therapy, nor does it intervene in the natural course of treatment with exten-
sive study documentation and diagnostic procedures. In this way, the Grossarth study enables
a comprehensive and undistorted evaluation of the patients’ treatment under everyday clinical
conditions. These features are combined with other design elements to assess and strengthen
the internal validity (that is, the highest possible distortion-free therapy evaluation), by em-
bedding a number of smaller RCTs within this larger cohort study. Thus Grossarth achieves
what almost all experimental trials are incapable of: to maximise internal and external validity
within a single research project. This achievement is lost when single embedded RCTs are
isolated and assessed independently from their research context, as in the Cochrane-Review.
In this respect the Cochrane Review represents a reductionistic tunnel vision that does not
enable any realistic evaluation of the Grossarth studies.
Against this background, the Cochrane-Review’s statement that lack of detailed information
about the therapies provided would limit the informational value of the Grosarth RCTs is also
put into perspective. Although the trials do not provide any information as to whether, for
example, Iscador A or Iscardor M is more effective for a particular indication, or whether mis-
tletoe dosage should be increased rapidly or slowly, they do assess the question of whether
mistletoe therapy administered in regular clinical settings has any benefit at all. Similarly, the
Cochrane-Review summary concludes with a global analysis and does not differentiate ac-
cording to mistletoe host tree and dosage. The Grossarth trials were also criticised for the lack
of detailed information about adjunct therapies administered apart from the investigational
therapy, but this represents customary practice in clinical trials. Similarly, in the Kleeberg or
Steuer-Vogt trials, there is no documentation on therapies administered apart from the trial
therapy – although one can assume that the patients did receive other therapies in the time
span from surgery to death.
Piao trial (2004) [15]: If blinding is not reliably possible, an active, effective therapy is a rea-
sonable and well-established alternative for the control group. This may lead to an underesti-
mation of the effects of the test therapy, which would not be in conflict with the prevailing
conservative attitude in clinical research. This solution was adopted in the Piao trial. In the
Cochrane-Review, however, the use of active control therapy was not acknowledged as a rea-
sonable alternative: the therapy benefit of mistletoe extracts was deemed unassessable since
the effects of the control therapy (Lentinan) were “unclear”. This statement is not quite cor-
rect because a number of clinical trials on Lentinan are available.
Gutsch trial (1988) [28]: This trial was excluded from the Cochrane-Review because of a
putative randomization error. However, no such error is reported in the publication by Gutsch;
instead protocol violations are described, i.e. the allocated therapy was not administered to all
patients. This occurs frequently in clinical trials. (For this reason analyses are mostly made
according to intention-to-treat and per-protocol; Gutsch analysed as treated [29]). Therefore
this trial should have been included as an RCT.
Kienle-Review (2003) [30]: According to the Cochrane Review, Kienle 2003 “failed to” in-
clude two unpublished trials, Lange 1993 and Schwiersch 1999. The fact is: “Unpublished”
was an exclusion criterion for the Kienle Review from 2003. – Furthermore, Kienle was said
to have also failed to include Borrelli’s published trial. This is true for Kienle 2003 but not for
Kienle’s HTA-report of 2006 [6;7] and the Kienle Review from 2007 [12]. According to the
Cochrane Review, Kienle 2003 also mistakenly included the above-mentioned Gutsch trial as
an RCT; in fact, however, this trial was indeed an RCT. – Kienle is said to have included Sal-
zer 1987 twice: Once as a randomized trial and once as a non-randomized trial. The fact is:
Two different trials were involved here, concerning bronchial carcinoma and breast cancer,
respectively. – The Cochrane-Review is also of the opinion that the trial described in the
Kienle Review 2003 as ‘”Salzer 1987” is identical to “Gutsch 1988” (in the Cochrane-Review
also described as “Günczler 1974”). This is also incorrect: Neither is Gutsch 1988 identical
with the RCT of bronchial carcinoma, nor with the quasi-randomized trial for breast carci-
noma which was carried out before the Gutsch trial. The randomized trial of bronchial carci-
noma in question was overlooked by the Cochrane Review and not included.
Further issues
The Cochrane Review has a substantial number of other errors of details, but not every prob-lematic aspect can be dealt with here. All non-randomized trials were excluded a priori, which is questionable since tumour re-sponse to mistletoe therapy was an explicit outcome parameter in the Cochrane Review and since tumour remission has been far more thoroughly investigated and represented in the non-randomized trials than in the randomized trials. Strangely, the decision to conduct this Cochrane Review is described as resulting from dis-crepancies between earlier reviews (e.g. Kienle 2003 [30], Ernst 2003 [31], Lange-Lindberg 2006 [32]) – whereas the protocol for the Cochrane Review was already available in the Cochrane Library long before these reviews were accessible. Overall picture of mistletoe trials
When the criticism against the Cochrane Review presented here is taken fully into account, a
different overall picture of the mistletoe trials emerges.
While most of the trials have strengths and weaknesses – to varying extent, as is the case with
other clinical trials – there are indeed a number carefully conducted trials. (Furthermore, the
mistletoe trials with a negative result, ranked as “high quality” by the Cochrane Review have
– in part substantial – quality deficiencies not discussed in the Review. See further details at
or [33]). Currently, the best evidence of mistletoe effects concerns
improvement of quality of life and improved tolerance of conventional oncological therapies.
Survival benefit from mistletoe therapy has been found in many trials, but not beyond cri-
tique. Tumour remissions have been reported, often in detail, in non-randomized trials, and
seem to depend on the type of administration and dosage of mistletoe extracts.
Hopefully the evidence base will be further broadened by future trials. However, “high qual-
ity” trials are frequently called for, but their practical implementation is subject to consider-
able difficulties. Due to huge bureaucratic hurdles today, industry-independent trials are
hardly feasible. (The cost per clinical RCT in the USA have been estimated at US$ 12 million
[34]). In Germany larger trials for mistletoe therapy are almost infeasible because of recruit-
ment problems and randomization refusals. For this reason, trials have to be conducted in
other countries. Blinding poses problems discussed above. Nonetheless, mistletoe therapy is
subject to vigorous research activities, the number of clinical trials has increased markedly in
recent years, and this trend is likely to continue in the future.
8th May 2008
Dr. med. Gunver S. Kienle, Dr. med. Helmut Kiene
Institute for Applied Epistemology and Medical Methodology
D-79111 Freiburg, Zechenweg 6, Germany
www.ifaemm.de
References
(1) Salzer G. 30 Jahre Erfahrung mit der Misteltherapie an öffentlichen Krankenanstalten. In: Leroi R, editor. Misteltherapie. Eine Antwort auf die Herausforderung Krebs.Stuttgart: Verlag Freies Geistesleben; 1987. p. 173-215. (2) Kim M-H, Park Y-K, Lee S-H, Kim S-C, Lee S-Y, Kim C-H, et al. Comparative study on the effects of a Viscum album (L.) extract (mistletoe) and doxycycline for pleurodesis in patients with malignant pleural effusion. 51th Meeting of The Korean Association of Internal Medi-cine. Translation by Helixor Heilmittel GmbH. Korean Journal of Medicine 1999;57(Suppl. II):S121. (3) Enesel MB, Acalovschi I, Grosu V, Sbarcea A, Rusu C, Dobre A, et al. Perioperative applica- tion of the Viscum album extract Isorel in digestive tract cancer patients. Anticancer Res 2005;25:4583-90. (4) Grossarth-Maticek R, Ziegler R. Prospective controlled cohort studies on long-term therapy of breast cancer patients with a mistletoe preparation (Iscador). Forsch Komplementärmed 2006;13:285-92. (5) Grossarth-Maticek R, Ziegler R. Randomized and non-randomized prospective controlled cohort studies in matched-pair design for the long-term therapy of breast cancer patients with a mistletoe preparation (Iscador): a re-analysis. Eur J Med Res 2006 Nov 30;11(11):485-95. (6) Kienle GS, Kiene H, Albonico HU. Anthroposophische Medizin in der klinischen Forschung. Wirksamkeit, Nutzen, Wirtschaftlichkeit, Sicherheit. Stuttgart, New York: Schattauer Verlag; 2006. (7) Kienle GS, Kiene H, Albonico HU. Anthroposophic Medicine: Effectiveness, Utility, Costs, Safety. Stuttgart, New York: Schattauer Verlag; 2006. (8) Grossarth-Maticek R, Ziegler R. Prospective controlled cohort studies on long-term therapy of cervical cancer patients with a mistletoe preparation (Iscador®). Forsch Komplementärmed 2007;14:140-7. (9) Grossarth-Maticek R, Ziegler R. Wirksamkeit und Unbedenklichkeit einer Langzeitbehand- lung von Melanompatienten mit einem Mistelpräparat (Iscador®). Schweizerische Zeitschrift für GanzheitsMedizin 2007;19:325-32. (10) Grossarth-Maticek R, Ziegler R. Prospective controlled cohort studies on long-term therapy of ovarian cancer patients with mistletoe (Viscum album L.) extracts Iscador. Arzneim -Forsch /Drug Res 2007;57(10):665-78. (11) Tröger W. Additional therapy with mistletoe extracts in breast cancer patients receiving che- motherapy. A prospective randomized open label pilot study. Phytomedicine 2007;14(VII):36. (12) Kienle GS, Kiene H. Complementary Cancer Therapy: A Systematic Review of Prospective Clinical Trials on Anthroposophic Mistletoe Extracts. Eur J Med Res 2007;12:103-19. (13) Ziegler R, Grossarth-Maticek R. Individual Patient Data Meta-analysis of Survival and Psy- chosomatic Self-regulation from Published Prospective Controlled Cohort Studies for Long-term Therapy of Breast Cancer Patients with a Mistletoe Preparation (Iscador). eCam 2008;doi:10.1093/ecam/nen025. (14) Jüni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 1999;282(11):1054-60. (15) Piao BK, Wang YX, Xie GR, Mansmann U, Matthes H, Beuth J, et al. Impact of complemen- tary mistletoe extract treatment on quality of life in breast, ovarian and non-small cell lung cancer patients. A prospective randomized controlled clinical trial. Anticancer Res 2004;24(1):303-9. (16) Steuer-Vogt MK, Bonkowsky V, Ambrosch P, Scholz M, Neiß A, Strutz J, et al. The effect of an adjuvant mistletoe treatment programme in resected head and neck cancer patients: a ran-domised controlled clinical trial. Eur J Cancer 2001;37:23-31. (17) Steuer-Vogt MK, Bonkowsky V, Scholz M, Fauser C, Licht K, Ambrosch P. Einfluss eines ML-1-normierten Mistelextraktes auf die Lebensqualität bei Patienten mit Kopf-Hals-Karzinomen. HNO 2006;54(4):277-86. (18) Fayers PM, Machin D. Quality of Life. The Assessment, Analysis and Interpretation of Pa- tient-reported Outcomes. 2 ed. Chichester: Wiley John & Sons; 2007. (19) Sprangers M, Moinpour CM, Moynihan TJ, Patrick DL, Revicki DA, The Clinical Signifi- cance Consensus Meeting Group. Assessing meaningful change in quality of life over time: a users' guide for clinicians. Mayo Clin Proc 2002;77:561-71. (20) Grossarth-Maticek R, Kiene H, Baumgartner S, Ziegler R. Use of Iscador, an extract of Euro- pean mistletoe (Viscum album), in cancer treatment: prospective nonrandomized and random-ized matched-pair studies nested within a cohort study. Altern Ther Health Med 2001;7(3):57-78. (21) Grossarth-Maticek R, Kiene H, Baumgartner S, Ziegler R. Addendum to Iscador article. Al- (22) Kleeberg UR, Suciu S, Bröcker EB, Ruiter DJ, Chartier C, Liénard D, et al. Final results of the EORTC 18871/DKG 80-1 randomised phase III trial: rIFN-2b versus rIFN- versus Iscador M versus observation after surgery in melanoma patients with either high-risk primary (thick-ness >3mm) or regional lymph node metastasis. Eur J Cancer 2004;40:390-402. (23) Borrelli E. Evaluation of the quality of life in breast cancer patients undergoing lectin stan- dardized mistletoe therapy. Minerva Medica 2001;92(Suppl. 1):105-7. (24) Dold U, Edler L, Mäurer HCh, Müller-Wening D, Sakellariou B, Trendelenburg F, et al. Krebszusatztherapie beim fortgeschrittenen nicht-kleinzelligen Bronchialkarzinom. Stuttgart, New York: Georg Thieme Verlag; 1991. (25) Grossarth-Maticek R. Systemische Epidemiologie und präventive Verhaltensmedizin chroni- scher Erkrankungen: Strategien zur Aufrechterhaltung der Gesundheit. Berlin, New York: Walter de Gruyter; 1999. (26) Rothwell PM. External validity of randomised controlled trials: "To whom do the results of this trials apply?". Lancet 2005;365:82-93. (27) Kienle GS. Gibt es Gründe für Pluralistische Evaluationsmodelle? Limitationen der Randomi- sierten Klinischen Studie. Z ärztl Fortbild Qual Gesundh wes 2005;99:289-94. (28) Gutsch J, Berger H, Scholz G, Denck H. Prospektive Studie beim radikal operierten Mamma- karzinom mit Polychemotherapie, Helixor und unbehandelter Kontrolle. Dtsch Zschr Onkol 1988;(4):94-100. (29) Sheiner LB, Rubin DB. Intention-to-treat analysis and the goals of clinical trials. Clin Phar- (30) Kienle GS, Berrino F, Büssing A, Portalupi E, Rosenzweig S, Kiene H. Mistletoe in cancer - a systematic review on controlled clinical trials. Eur J Med Res 2003;8:109-19. (31) Ernst E, Schmidt K, Steuer-Vogt MK. Mistletoe for cancer? A systematic review of random- ized clincial trials. Int J Cancer 2003;107:262-7. (32) Lange-Lindberg AM, Velasco Garrido M, Busse R. Misteltherapie als begleitende Behandlung zur Reduktion der Toxizität der Chemotherapie maligner Erkrankungen. GMS Health Technol Assess 2006; 2:Doc18 (20060919). 2006. (33) Kienle GS, Kiene H. Die Mistel in der Onkologie - Fakten und konzeptionelle Grundlagen. Stuttgart, New York: Schattauer Verlag; 2003. (34) Johnston SC, Rootenberg JD, Katrak S, Smith WS, Elkins JS. Effect of a US National Insti- tutes of Health programme of clinical trials on public health and costs. Lancet 2006;367:1319-27.

Source: http://www.ivaa.info/fileadmin/editor/file/teaser/Cochrane_Comment_Kienle_Kiene_English.pdf