Coming back to: A structured critique template this community uses posts 31–60
This is a continuation of a long topic, addressed by post number rather than by page. Start at post 1.
Where a critique rests on a subgroup, check whether the interaction was tested. A difference between subgroups that is not statistically supported is not a difference.
A weak preference rather than a position.
A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary.
Speaking for myself and not for anyone else who has posted here.
Adding the measurement that post #31 says would settle it.
Reading this structured critique template thread as someone who came in with a fixed view: the third and seventh replies moved me and the confident ones did not.
Adding a small correction to the structured critique template summary above rather than a disagreement with it. The substance holds; one of the figures is out by a factor that matters.
On post #35 — agreed on the reasoning, with one qualification.
Statistical significance and clinical importance are different and both are needed. A significant difference below the minimal important difference is a real finding of no practical consequence.
Picking up post #35: that is the part I would want checked first.
Where a critique changes your position, saying so in the thread is worth more than being right, and it is the thing this subcategory does better than most places.
Post #39 answers the question as asked. The question underneath it is different.
I would be cautious about generalising from the structured critique template example above. It is a good example. It is one example.
This follows post #40 rather than contradicting it.
Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.
Worth separating two things that post #38 runs together.
Multiple comparisons: if a paper reports many outcomes, the chance of a spurious association by random chance is real. Pre-specification of primary outcomes matters and secondary analyses are weaker evidence.
The strongest critiques cite the section they rely on. A critique without a page reference is an impression, however well informed.
That is where I would start, not where I would stop.
Structured critique template was covered in the wiki last year and the page has a review date on it, which is a better starting point than my memory of a thread.
Agreed on all of that, and I have nothing to add to it.
I read post #46 twice before replying, because I had assumed the opposite.
When you change your mind: if a reply convinces you that your criticism was not well-founded, say so plainly. The critique might still be real but smaller than you originally thought. That is not a failure — it is how discussion works.
Where a critique rests on a subgroup, check whether the interaction was tested. A difference between subgroups that is not statistically supported is not a difference.
Not the whole picture, but the part of it I can speak to.
Adding what did not work for me on structured critique template, since the failures never get written up and they are half the useful information.
Post #49 and I disagree about the size of the effect, not about the direction.
Structured critique template is a question about a distribution, not about a value, and treating it as a value is what produces the confident wrong answers.
Collapsed as off-topic by two members at trust level 3 or above
Taking post #49 at face value and following it one step further.
Hold a trial to the standard something could actually have met. A criticism that no achievable design could have answered is a criticism of the field rather than of the paper.
Take the reasoning and check the arithmetic; I do not always get it right.
One more thing on structured critique template that took me far too long to see: the two figures people quote are not measuring the same quantity. Once you notice that, the apparent contradiction disappears.
Worth separating two things that post #53 runs together.
Practical answer on structured critique template, since the theoretical one is upthread: do the simplest check first, write down the result, and only then decide whether the complicated explanation is needed. It usually is not.
Fair, and the limits you put on it are the part I will remember.
A critique should end with what would change your mind. Without that it is a position rather than an argument.
Blinding, duration and comparator choice trade against each other. Improving one usually costs another, and a critique that ignores the trade is incomplete.
The answer changed when I changed how I was measuring, which was informative.