Outcome switching: Difference between revisions
Content deleted Content added
m gr |
expand per request DYK |
||
| Line 1: | Line 1: | ||
'''Outcome switching''' is to the practice of changing the primary or secondary outcomes of a [[clinical trial]] after its initiation. Outcome switching can undermine the reliability of the trial, for instance when outcomes are switched after researchers have become aware of the data. |
'''Outcome switching''' is to the practice of changing the primary or secondary outcomes of a [[clinical trial]] after its initiation. Outcome switching can lead to bias and undermine the reliability of the trial, for instance when outcomes are switched after researchers have become aware of the data. |
||
== Problem and possible solutions == |
|||
| ⚫ | Primary outcomes for clinical trials need to be defined upfront to prevent a biased selection of outcomes. Some medical journal require adherence to the [[Consolidated Standards of Reporting Trials| |
||
Outcome switching can undermine the reliability of the trial, for instance when outcomes are switched when researchers already have access to the trial data. It can lead to [[Bias (statistics)|bias]] in terms of benefits and harms. For example, when the findings using the original protocol were statistically insignificant, a study may [[Cherry picking|cherry pick]] a new outcome measure that is [[Statistical significance|statistically significant]].<ref name="pmid28196813">{{cite journal |vauthors=Altman DG, Moher D, Schulz KF |date=February 2017 |title=Harms of outcome switching in reports of randomised trials: CONSORT perspective |url= |journal=BMJ (Clinical Research Ed.) |volume=356 |issue= |pages=j396 |doi=10.1136/bmj.j396 |pmid=28196813|s2cid=206916748 }}</ref> If there are a large number of outcomes to choose from, it is likely at least one will show significant findings, assuming no correction is made for [[multiple testing]].<ref>{{Cite journal |last=Wicherts |first=Jelte M. |date=2017 |title=The Weak Spots in Contemporary Science (and How to Fix Them) |url=https://www.mdpi.com/2076-2615/7/12/90 |journal=Animals |language=en |volume=7 |issue=12 |pages=90 |doi=10.3390/ani7120090 |issn=2076-2615}}</ref> Incomplete or misleading reporting of outcomes is an example of [[unethical research]] practice.<ref name="pmid28196813" /> |
|||
| ⚫ | Primary outcomes for clinical trials need to be defined upfront to prevent a biased selection of outcomes. Some medical journal require adherence to the [[Consolidated Standards of Reporting Trials|CONSORT]] standards, which requires authors to document any deviations from the initial study protocol and explain why changes were made. For instance, ''[[JAMA]]'' and the [[British Medical Journal|''British Medical Journal'']] require inclusion of the CONSORT checklist.<ref name="pmid28196813" /> When pre-specified outcomes are defined but unclear, researchers still have wriggle room to choose the most favourable option. A clear pre-defined outcome is therefore important to avoid outcome switching.<ref>{{Cite journal |last=Kahan |first=Brennan C. |last2=Jairath |first2=Vipul |date=2018-05-02 |title=Outcome pre-specification requires sufficient detail to guard against outcome switching in clinical trials: a case study |url=https://doi.org/10.1186/s13063-018-2654-z |journal=Trials |volume=19 |issue=1 |pages=265 |doi=10.1186/s13063-018-2654-z |issn=1745-6215 |pmc=PMC5932799 |pmid=29720248}}</ref> |
||
== Prevalence == |
|||
Despite problems with outcome switching, the practice is common. Changes in primary outcome metrics are present in nearly one in three studies.<ref name="pmid26581191">{{cite journal |vauthors=Jones CW, Keil LG, Holland WC, Caughey MC, Platts-Mills TF |date=November 2015 |title=Comparison of registered and published outcomes in randomized controlled trials: a systematic review |url= |journal=BMC Medicine |volume=13 |issue= |pages=282 |doi=10.1186/s12916-015-0520-3 |doi-access=free |pmc=4650202 |pmid=26581191}}</ref> Outcome switching also occurs frequently in follow-up studies.<ref name="pmid34004339">{{cite journal |vauthors=Kampman JM, Sperna Weiland NH, Hollmann MW, Repping S, Hermanides J |date=September 2021 |title=High incidence of outcome switching observed in follow-up publications of randomized controlled trials: Meta-research study |url= |journal=Journal of Clinical Epidemiology |volume=137 |issue= |pages=236–240 |doi=10.1016/j.jclinepi.2021.05.003 |pmid=34004339}}</ref> In [[oncology]], outcome switching is more common in studies with a male first author, and in studies funded by non-profits.<ref name="pmid28835682">{{cite journal |vauthors=Falk Delgado A, Falk Delgado A |date=August 2017 |title=Outcome switching in randomized controlled oncology trials reporting on surrogate endpoints: a cross-sectional analysis |url= |journal=Scientific Reports |volume=7 |issue=1 |pages=9206 |doi=10.1038/s41598-017-09553-y |pmc=5569019 |pmid=28835682|bibcode=2017NatSR...7.9206F }}</ref> One study analysed outcome switching in five top medical journals, writing letters for each misreported trial outcome. Journal editors and clinical trial authors typically responded dismissively when concerns were raised, misrepresenting ethical standards and including ''[[ad hominem]]'' attacks.<ref name="pmid30760328">{{cite journal |vauthors=Goldacre B, Drysdale H, Marston C, Mahtani KR, Dale A, Milosevic I, Slade E, Hartley P, Heneghan C |date=February 2019 |title=COMPare: Qualitative analysis of researchers' responses to critical correspondence on a cohort of 58 misreported trials |url= |journal=Trials |volume=20 |issue=1 |pages=124 |doi=10.1186/s13063-019-3172-3 |doi-access=free |pmc=6374909 |pmid=30760328}}</ref> |
Despite problems with outcome switching, the practice is common. Changes in primary outcome metrics are present in nearly one in three studies.<ref name="pmid26581191">{{cite journal |vauthors=Jones CW, Keil LG, Holland WC, Caughey MC, Platts-Mills TF |date=November 2015 |title=Comparison of registered and published outcomes in randomized controlled trials: a systematic review |url= |journal=BMC Medicine |volume=13 |issue= |pages=282 |doi=10.1186/s12916-015-0520-3 |doi-access=free |pmc=4650202 |pmid=26581191}}</ref> Outcome switching also occurs frequently in follow-up studies.<ref name="pmid34004339">{{cite journal |vauthors=Kampman JM, Sperna Weiland NH, Hollmann MW, Repping S, Hermanides J |date=September 2021 |title=High incidence of outcome switching observed in follow-up publications of randomized controlled trials: Meta-research study |url= |journal=Journal of Clinical Epidemiology |volume=137 |issue= |pages=236–240 |doi=10.1016/j.jclinepi.2021.05.003 |pmid=34004339}}</ref> In [[oncology]], outcome switching is more common in studies with a male first author, and in studies funded by non-profits.<ref name="pmid28835682">{{cite journal |vauthors=Falk Delgado A, Falk Delgado A |date=August 2017 |title=Outcome switching in randomized controlled oncology trials reporting on surrogate endpoints: a cross-sectional analysis |url= |journal=Scientific Reports |volume=7 |issue=1 |pages=9206 |doi=10.1038/s41598-017-09553-y |pmc=5569019 |pmid=28835682|bibcode=2017NatSR...7.9206F }}</ref> One study analysed outcome switching in five top medical journals, writing letters for each misreported trial outcome. Journal editors and clinical trial authors typically responded dismissively when concerns were raised, misrepresenting ethical standards and including ''[[ad hominem]]'' attacks.<ref name="pmid30760328">{{cite journal |vauthors=Goldacre B, Drysdale H, Marston C, Mahtani KR, Dale A, Milosevic I, Slade E, Hartley P, Heneghan C |date=February 2019 |title=COMPare: Qualitative analysis of researchers' responses to critical correspondence on a cohort of 58 misreported trials |url= |journal=Trials |volume=20 |issue=1 |pages=124 |doi=10.1186/s13063-019-3172-3 |doi-access=free |pmc=6374909 |pmid=30760328}}</ref> |
||
Revision as of 08:35, 18 February 2024
Outcome switching is to the practice of changing the primary or secondary outcomes of a clinical trial after its initiation. Outcome switching can lead to bias and undermine the reliability of the trial, for instance when outcomes are switched after researchers have become aware of the data.
Problem and possible solutions
Outcome switching can undermine the reliability of the trial, for instance when outcomes are switched when researchers already have access to the trial data. It can lead to bias in terms of benefits and harms. For example, when the findings using the original protocol were statistically insignificant, a study may cherry pick a new outcome measure that is statistically significant.[1] If there are a large number of outcomes to choose from, it is likely at least one will show significant findings, assuming no correction is made for multiple testing.[2] Incomplete or misleading reporting of outcomes is an example of unethical research practice.[1]
Primary outcomes for clinical trials need to be defined upfront to prevent a biased selection of outcomes. Some medical journal require adherence to the CONSORT standards, which requires authors to document any deviations from the initial study protocol and explain why changes were made. For instance, JAMA and the British Medical Journal require inclusion of the CONSORT checklist.[1] When pre-specified outcomes are defined but unclear, researchers still have wriggle room to choose the most favourable option. A clear pre-defined outcome is therefore important to avoid outcome switching.[3]
Prevalence
Despite problems with outcome switching, the practice is common. Changes in primary outcome metrics are present in nearly one in three studies.[4] Outcome switching also occurs frequently in follow-up studies.[5] In oncology, outcome switching is more common in studies with a male first author, and in studies funded by non-profits.[6] One study analysed outcome switching in five top medical journals, writing letters for each misreported trial outcome. Journal editors and clinical trial authors typically responded dismissively when concerns were raised, misrepresenting ethical standards and including ad hominem attacks.[7]
See also
References
- ^ a b c Altman DG, Moher D, Schulz KF (February 2017). "Harms of outcome switching in reports of randomised trials: CONSORT perspective". BMJ (Clinical Research Ed.). 356: j396. doi:10.1136/bmj.j396. PMID 28196813. S2CID 206916748.
- ^ Wicherts, Jelte M. (2017). "The Weak Spots in Contemporary Science (and How to Fix Them)". Animals. 7 (12): 90. doi:10.3390/ani7120090. ISSN 2076-2615.
{{cite journal}}: CS1 maint: unflagged free DOI (link) - ^ Kahan, Brennan C.; Jairath, Vipul (2018-05-02). "Outcome pre-specification requires sufficient detail to guard against outcome switching in clinical trials: a case study". Trials. 19 (1): 265. doi:10.1186/s13063-018-2654-z. ISSN 1745-6215. PMC 5932799. PMID 29720248.
{{cite journal}}: CS1 maint: PMC format (link) CS1 maint: unflagged free DOI (link) - ^ Jones CW, Keil LG, Holland WC, Caughey MC, Platts-Mills TF (November 2015). "Comparison of registered and published outcomes in randomized controlled trials: a systematic review". BMC Medicine. 13: 282. doi:10.1186/s12916-015-0520-3. PMC 4650202. PMID 26581191.
- ^ Kampman JM, Sperna Weiland NH, Hollmann MW, Repping S, Hermanides J (September 2021). "High incidence of outcome switching observed in follow-up publications of randomized controlled trials: Meta-research study". Journal of Clinical Epidemiology. 137: 236–240. doi:10.1016/j.jclinepi.2021.05.003. PMID 34004339.
- ^ Falk Delgado A, Falk Delgado A (August 2017). "Outcome switching in randomized controlled oncology trials reporting on surrogate endpoints: a cross-sectional analysis". Scientific Reports. 7 (1): 9206. Bibcode:2017NatSR...7.9206F. doi:10.1038/s41598-017-09553-y. PMC 5569019. PMID 28835682.
- ^ Goldacre B, Drysdale H, Marston C, Mahtani KR, Dale A, Milosevic I, Slade E, Hartley P, Heneghan C (February 2019). "COMPare: Qualitative analysis of researchers' responses to critical correspondence on a cohort of 58 misreported trials". Trials. 20 (1): 124. doi:10.1186/s13063-019-3172-3. PMC 6374909. PMID 30760328.