Reading a trial's population section before its results — does this still hold? posts 61–90
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
A comparator at less than its maximum licensed dose changes what a head-to-head result means. It does not invalidate the trial; it narrows the claim the trial supports.
Post #61 and I disagree about the size of the effect, not about the direction.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
Taking post #61 at face value and following it one step further.
Discontinuation handling is the methodological detail that most changes a result and gets the least attention. Read how missing data was imputed before reading the effect size.
I would treat the number as indicative rather than as a measurement.
Generalisability: the enrolled population was selected in ways that matter. Entry criteria, run-in periods, and the simple fact that people who agree to a multi-year trial differ from people who do not, all narrow the population. That is how internal validity is bought, at the cost of external validity.
I have separated what I observed from what I concluded, which does not always happen.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
That is the version I would defend. It is not the version I started with.
Collapsed as off-topic by two members at trust level 3 or above
Confounding in observational data: a third variable can explain an apparent association. In a randomised trial, randomisation balances unknown confounders. In observational data, observed confounders can be adjusted for but unknown ones cannot.
This follows post #65 rather than contradicting it.
Intent-to-treat versus per-protocol: ITT includes everyone assigned regardless of whether they took the drug. Per-protocol includes only those who completed it as intended. The two can give substantially different results.
Post #65 describes the usual case. This is about the unusual one.
Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.
The strength of my opinion here exceeds the strength of my evidence.
Collapsed as off-topic by two members at trust level 3 or above
Reading the supplementary appendix is where most of the real information is, and it is where almost nobody goes. The baseline table alone answers half the generalisability questions asked here.
If that is already documented somewhere, ignore me and link it.
Effect sizes in a trial population reflect adherence achieved under trial conditions, which is generally better than adherence outside them.
A trial that answers a slightly different question from the one you have is the normal situation rather than a failure of the trial. The skill is describing the gap precisely.
That is all I can say without guessing.
This follows post #72 rather than contradicting it.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
Nothing above should be read as advice about what anyone else should do.
Agreed, and I will stop repeating the version of this I had been repeating.
Trial duration determines what can be observed. A weight-change trajectory at 40 weeks and at 72 weeks are different observations and both get quoted as the result.
Where I would look next, rather than where I would stop.
I had written a reply contradicting post #75 and deleted it. Here is what survived.
Composite endpoints should be read component by component. A composite driven entirely by its softest component is a different finding from one where the components move together.
That much is documented. The rest is how I have interpreted it.
Collapsed as off-topic by two members at trust level 3 or above
Picking up post #76: that is the part I would want checked first.
A comparator at less than its maximum licensed dose changes what a head-to-head result means. It does not invalidate the trial; it narrows the claim the trial supports.
The step people skip is the one I have spelled out.
Narrowing post #76, because the general version has more than one answer.
Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.
Somebody will have a better source than mine, and I hope they post it.
Everything in post #78 holds. The case it does not cover is the one I have.
Absolute numbers, not just relative: a 30% relative reduction tells you the ratio but not the practical magnitude. The event rate in each arm and the difference between them tells you how many people benefit.
Funding and trial conduct should be stated and are a weak predictor of anything on their own. Design quality is the stronger signal and it is checkable.
Adding this to the thread rather than to the wiki, because I am not confident enough for the wiki.
Right, and stated more narrowly than I would have dared to state it.
Coming back to post #79, because the follow-up matters more than the original answer.
Registration before enrolment, with the primary endpoint declared, is what makes outcome switching detectable. Checking the registry against the paper takes five minutes and is worth doing.
Post #83 is right about the mechanism and I think understates the practical bit.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
Worth separating two things that post #83 runs together.
Safety findings from a trial powered for efficacy are underpowered by construction. Absence of a signal in that setting is weak evidence of absence.
A treatment-policy estimand asks what happens to people assigned to a strategy, including those who abandon it. A hypothetical estimand asks what would have happened had everyone continued. Both are legitimate and they give different numbers.
Old habit: I write down the expected answer before I calculate it.
Post #87 answers the question as asked. The question underneath it is different.
Risk of bias: structured appraisal of internal validity. Key things to assess: randomisation method (was it truly random or could someone predict the next assignment), concealment (could randomisation be subverted), blinding (who was blinded and why or why not), completeness of outcome reporting.
Helpful, and easy to find again, which is half of what a good reply is.
Adding the measurement that post #87 says would settle it.
Absolute and relative effects answer different questions. Write down the event rate in each arm and the difference between them; everything quotable is derived from those two numbers.