Evidence · Study critique

Immortal time bias in a claims-database study

Evidence › Study critique risk of bias observational data disputed confounding

Closed

mi.almeidaTL214 Jul 2025#1

Immortal time bias in a claims-database study — setting out what I have, and where I think it stops being reliable.

Asking about immortal time bias directly, because I have read four threads on it and each answered a slightly different question.

The version I want answered is the narrow one: given the method stated below, is the result within what anyone else has seen? I am not asking what it means yet.

Method, numbers and the two assumptions I am aware of making are below. If the assumptions are wrong that is more useful to me than agreement.

54 likes 12mo

cohort_driftTL3Regular17 Jul 2025#2

Second-hand on immortal time bias, so weight it accordingly — someone whose method I trust told me this and I have not verified it myself.

14 likes 12mo

s.okonkwoTL219 Jul 2025#3

cohort_drift, post #2: Second-hand on immortal time bias, so weight it accordingly — someone whose method I trust told me this and I have not verified it myself. Go to post

Publication bias: a single published positive trial is weaker evidence than multiple published trials with consistent results. Asking whether there are unpublished negative trials is a fair critical question.

Anyone who has looked at this more carefully, please correct the record.

5 likes in reply to #2 12mo

isotonic_driftTL1Member21 Jul 2025#4

If you are new and reading this thread for the answer to immortal time bias: the answer is conditional, the conditions are in the third reply, and the rest of the thread is worth skipping.

0 likes 12mo

n.nakamuraTL223 Jul 2025#5

Understood. Thank you for being specific about the limits of it.

21 likes 12mo

c.niemelTL3Regular25 Jul 2025#6

Narrowing post #3, because the general version has more than one answer.

Reframing immortal time bias slightly, because I think the disagreement is about the question rather than the answer. If the question is "does it happen", yes. If it is "how often", nobody here knows.

9 likes 12mo

n.dziedzicTL226 Jul 2025#7

isotonic_drift, post #4: If you are new and reading this thread for the answer to immortal time bias: the answer is conditional, the conditions are in the third reply, and the rest of the thread is worth skipping. Go to post

Per-protocol and intention-to-treat analyses answer different questions and neither is the honest one by default. Reporting both is the practice worth insisting on.

2 likes in reply to #4 12mo

BramleyTL2Member28 Jul 2025 · edited#8

s.okonkwo, post #3: Publication bias: a single published positive trial is weaker evidence than multiple published trials with consistent results. Asking whether there are unpublished negative trials is a fair critical question. Anyone who has looked at this more carefully, please correct the record. Go to post

Reporting bias within a paper — a result in a figure that is absent from the abstract — is checkable and is worth checking when the claim matters.

That is the version I use. It may not be the version that is correct.

0 likes in reply to #3 12mo

l.trevinoTL229 Jul 2025#9

Generalisability and validity are separate axes. A trial can be internally impeccable and still tell you nothing about the person asking.

That is what I would do. It may not be what is correct.

15 likes 12mo

e.okaforTL231 Jul 2025#10

The arithmetic in post #7 is right; the assumption feeding it is the part to check.

Bias towards the null and bias away from the null: different criticisms have different directions. Differential dropout might bias away from null; conservative statistical analysis might bias toward null.

I looked this up rather than remembered it, which is the right order.

6 likes 12mo

f.wojcikTL21 Aug 2025#11

l.trevino, post #9: Generalisability and validity are separate axes. A trial can be internally impeccable and still tell you nothing about the person asking. That is what I would do. It may not be what is correct. Go to post

Taking post #8 at face value and following it one step further.

A run-in period that excludes non-responders before randomisation changes what the trial is estimating. It is legitimate design and it must be stated in any summary.

The reasoning is more useful than the number, which is why I have shown it.

27 likes in reply to #9 12mo

f.fonsecaTL23 Aug 2025#12

cohort_drift, post #2: Second-hand on immortal time bias, so weight it accordingly — someone whose method I trust told me this and I have not verified it myself. Go to post

Post #10 and I disagree about the size of the effect, not about the direction.

Immortal time bias looks different depending on whether you are reading the primary literature or the summaries of it, and the difference is not in our favour.

0 likes in reply to #2 12mo

v.bhattacharyaTL24 Aug 2025 · edited#13

2 likes 12mo

s.rasmussenTL26 Aug 2025#14

This is the first time the answer has come with its own limits attached. Appreciated.

9 likes 12mo

a.sorensenTL27 Aug 2025#15

Building on post #12 rather than restating it.

Careful with the language on immortal time bias. "Not detected" and "not present" are different findings and the first is a statement about the method.

20 likes 12mo

s.duarteTL28 Aug 2025#16

Post #15 put the caveat in the right place and I want to underline it.

Two questions I would want answered before drawing anything from the immortal time bias data above: how were the cases selected, and what happened to the ones that dropped out.

0 likes in reply to #3 12mo

n.nakamuraTL210 Aug 2025#17

Generalisability: do the inclusion/exclusion criteria narrow the population so much that results do not apply to real people asking about it? This is a fair criticism but requires specificity about which real people and why the difference matters.

That much is documented. The rest is how I have interpreted it.

0 likes 12mo

a.salcedoTL3Regular11 Aug 2025#18

Immortal time bias: I have looked for the primary source twice and failed twice. Either it does not exist or it is somewhere I do not know to look, and I would like to know which.

5 likes 12mo

blank_injectionTL2Analytical chemist12 Aug 2025#19

f.fonseca, post #12: Post #10 and I disagree about the size of the effect, not about the direction. Immortal time bias looks different depending on whether you are reading the primary literature or the summaries of it, and the difference is not in our favour. Go to post

Post #16 is the version of this I will quote in future. One addition.

What I want from this immortal time bias thread is the list of things that would need to be true for the claim to hold. If we can write that list, we can check it.

0 likes in reply to #12 12mo

n.villalobosTL213 Aug 2025 · edited#20

I would rather say I do not know than round it up to an answer.

2 likes 11mo

vial_slopeTL3Regular15 Aug 2025#21

Something worth flagging about immortal time bias: the strongest-sounding claims in this thread are the ones with no source attached, which is the usual pattern and not a coincidence.

0 likes in reply to #3 11mo

n.hartmannTL216 Aug 2025#22

I have three months of notes on immortal time bias and the honest summary is that the trend is real and the week-to-week numbers are noise. I nearly drew the opposite conclusion from the first fortnight.

21 likes 11mo

l.parkinsonTL2Member17 Aug 2025#23

I read post #19 twice before replying, because I had assumed the opposite.

Surrogate endpoints are not automatically bad and their validity is compound-specific and population-specific. The question is whether this surrogate has been validated for this use.

This is where my knowledge stops and I would rather mark the edge than blur it.

5 likes 11mo

ma.nascimentoTL218 Aug 2025#24

A criticism that would apply equally to every trial in the field is worth stating once and is not a reason to discount a particular paper.

That is what the documentation says. What happens in practice is usually close.

0 likes 11mo

IMainwaringTL3Regular19 Aug 2025#25

c.niemel, post #6: Narrowing post #3, because the general version has more than one answer. Reframing immortal time bias slightly, because I think the disagreement is about the question rather than the answer. If the question is "does it happen", yes. If it is "how often", nobody here knows. Go to post

Reporting bias within a paper — a result in a figure that is absent from the abstract — is checkable and is worth checking when the claim matters.

0 likes in reply to #6 11mo

k.karlsenTL221 Aug 2025#26

mi.almeida, post #1: Immortal time bias in a claims-database study — setting out what I have, and where I think it stops being reliable. Asking about immortal time bias directly, because I have read four threads on it and each answered a slightly different question. The version I want answered is the narrow one: given the method stated below, is the result… Go to post

Taking post #23 at face value and following it one step further.

Immortal time bias has a well-known answer and a correct answer, and the interesting work is establishing that they are the same. Nobody has done that here yet.

28 likes in reply to #1 11mo

NLoughranTL3Regular22 Aug 2025 · edited#27

Agreed, and I will stop repeating the version of this I had been repeating.

9 likes 11mo

v.malinowskiTL223 Aug 2025#28

The documentation on immortal time bias is better than this thread and I say that as someone who has posted in the thread.

2 likes 11mo

FFaulknerTL3Regular24 Aug 2025#29

e.okafor, post #10: The arithmetic in post #7 is right; the assumption feeding it is the part to check. Bias towards the null and bias away from the null: different criticisms have different directions. Differential dropout might bias away from null; conservative statistical analysis might bias toward null. I looked this up rather than remembered it, which… Go to post

Post #28 describes the usual case. This is about the unusual one.

Statistical significance and clinical importance are different and both are needed. A significant difference below the minimal important difference is a real finding of no practical consequence.

The mechanism is plausible, which is not the same as established.

22 likes in reply to #10 11mo

so.mbekiTL225 Aug 2025#30

Thank you for the correction. I would rather find out here than later.

10 likes 11mo

Sponsored Sigma-Aldrich — certified reference materialsReference standards, analytical reagents and certified materials for method calibration. If you are validating a method, this is where the standard comes from.Reference standards →