Measurement error in a self-reported exposure — setting out what I have, and where I think it stops being reliable.
Reading back through what has been written here about Measurement error, three questions come up every time and only one has ever been answered properly.
Listing all three, with what I think the state of the answer is for each, so the thread can start further along than the last one did.
Clear enough that I do not think I have a follow-up, which is unusual.
Staff rationale: Hidden by community flags. Contained an unredacted laboratory report with identifiers visible (R12). The analytical content has been quoted back into the topic without them.
Narrowing post #2, because the general version has more than one answer.
I think the Measurement error question is answerable and has not been answered, which is a more optimistic position than most of this thread.
The corresponding entry is in the public moderation log. Hidden posts are never deleted.
Post #2 put the caveat in the right place and I want to underline it.
Two claims get bundled together under Measurement error and they need separating. The descriptive one — this is what was observed — is usually well supported. The causal one — this is why — usually is not.
Almost every disagreement in threads like this one dissolves once you say which of the two you are making.
Multiple comparisons: if a paper reports many outcomes, the chance of a spurious association by random chance is real. Pre-specification of primary outcomes matters and secondary analyses are weaker evidence.
The short answer was in the first line; everything after is the working.
Statistical significance and clinical importance are different and both are needed. A significant difference below the minimal important difference is a real finding of no practical consequence.
The right answer here may simply be that it has not been measured.
The most useful thing anyone has posted about Measurement error in this category was a table of what had been measured and by whom. That is what I would want again.
Thank you for taking the time. That was more work than a reply usually is.
On post #11 — agreed on the reasoning, with one qualification.
The pre-specified endpoint being a weaker proxy than you would like is a real criticism. It is a smaller one than saying the result was chosen after the fact.
Post #11 is right about the mechanism and I think understates the practical bit.
I have been on both sides of the Measurement error argument in this category within eighteen months, which should tell you how strong the evidence for either side is.
Second this, and I would have said it less carefully.
Whatever the answer on Measurement error turns out to be, the method for getting there is the same: state the assumption, do the arithmetic in public, invite the correction.
Taking post #16 at face value and following it one step further.
Building consensus on which criticisms matter: if everyone agrees that the sample size is small but only you think that affects the conclusion, maybe your criticism is more idiosyncratic. That does not make it wrong but it is worth noticing.
Adding this to the thread rather than to the wiki, because I am not confident enough for the wiki.
Post #18 and I disagree about the size of the effect, not about the direction.
Reading this Measurement error thread as someone who came in with a fixed view: the third and seventh replies moved me and the confident ones did not.
Adding the measurement that post #19 says would settle it.
Generalisability and validity are separate axes. A trial can be internally impeccable and still tell you nothing about the person asking.
Where I would look next, rather than where I would stop.
I had written a reply contradicting post #19 and deleted it. Here is what survived.
What would change my mind on Measurement error is a second dataset collected by someone with no stake in the first. Until then I hold it loosely and I would rather say so than pretend to more.
Practical note on Measurement error: write down what you expect before you look. The number of times I have found what I went looking for is higher than chance would allow.
Picking up post #23: that is the part I would want checked first.
Criticism is more useful when it is narrower. "The trial answers a different question from the one being asked" is actionable; "the trial is flawed" is not.
That is the honest state of it as of this week.
Source for the Measurement error figure, since it was asked for. It is in the discussion rather than the abstract, which is why the version circulating is stronger than the paper is.
Reading the surrounding paragraph is worth the two minutes. The authors are more careful than their summarisers.
A request rather than an answer: could whoever has the primary source for Measurement error post it? I have seen the claim three times this month and each version had lost a qualifier.
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
A structured critique template this community uses
A structured critique template this community uses — setting out what I have, and where I think it stops being reliable. Structured critique template — I have the observation and I do not trust my…
|
2 | 18k | 1d | |
|
About the Study critique category
Structured criticism of a specific published study. Criticise the method, not the author. This post is a community wiki: any member at trust level 3 or above can edit it, and every edit is recorded with its…
|
4 | 114 | 16mo | |
|
Selection into a registry and what it does to the estimate
Selection into a registry and what it does to the estimate — setting out what I have, and where I think it stops being reliable. Working notes on selection rather than a conclusion. I would rather post the…
|
2 | 55k | 7mo | |
|
Selection into a registry and what it does to the estimate — a second dataset
Posting this under the heading it deserves: Selection into a registry and what it does to the estimate — a second dataset Everything below is what sits behind that. Posting a small dataset on Selection. It is…
|
+28 | 32 | 1.3k | 1d |
|
Immortal time bias in a claims-database study
Immortal time bias in a claims-database study — setting out what I have, and where I think it stops being reliable. Asking about immortal time bias directly, because I have read four threads on it and each…
|
+29 | 34 | 28k | 11mo |
| Topic | Participants | Replies | Views | Activity |
|---|---|---|---|---|
|
Journal club: semaglutide in MASH, and surrogate endpoints — does this still hold?
Journal club: semaglutide in MASH, and surrogate endpoints — does this still hold? I have a specific reason for asking rather than idle curiosity, and the context is below. A narrow question about semaglutide…
|
3 | 28k | 6mo | |
|
Confounding by indication, explained with a concrete example
On the subject in the title: Confounding by indication, explained with a concrete example Working notes rather than a conclusion. Comparing SURMOUNT-1 ( N Engl J Med , 2022) with SURPASS-2 ( N Engl J Med ,…
|
+21 | 25 | 21k | 11mo |
|
Risk-of-bias tools and the judgement they conceal
Risk-of-bias tools and the judgement they conceal Writing it up because I had to work it out twice and would rather nobody else did. I have spent a fortnight trying to pin risk-of-bias tools down and I want…
|
+8 | 12 | 21k | 11mo |
|
Coming back to: Prediction intervals and why they are more honest than confidence intervals
Prediction intervals and why they are more honest than confidence intervals Writing it up because I had to work it out twice and would rather nobody else did. Prediction intervals — I have the observation and…
|
2 | 65 | 22mo | |
|
Journal club: STEP 3 and the intensive behavioural therapy floor — does this still hold?
The question in the title: Journal club: STEP 3 and the intensive behavioural therapy floor — does this still hold? I will give what I have already checked below so nobody repeats it. Asking about STEP 3 on…
|
2 | 15k | 10mo |