Thread by @EpiEllie

Now that the #epibookclub is over, my weekends are free for reading #RCTs. So, how about a #tweetorial about #pragmatic trials, causal questions, and #landmark analyses, inspired by the #SCOTHEART trial?

#epimethodsclub #cardiotwitter

If we’re going to talk about #pragmatic trials, then we need to start with a definition. The simplest definition I’ve seen is from a trialist I interviewed for my recent @JClinEpi paper (authors.elsevier.com/a/1XS1b3BcJPuv…)

pRCTs “...attempt to address effectiveness in real world settings”

Sure, it’s a bit vague but it’s useful because, like Tolstoy’s unhappy families, #pragmatic trials are all different.

So, what makes me think #SCOTHEART is pragmatic?

My 2¢: lack of blinding, inclusion criteria based on symptoms not diagnoses, & standard care as comparator.

#pragmatic trials often aim to inform clinical practice, and this means the causal questions of interest may differ from more traditional #RCTs.

In particular, the intention-to-treat effect tells us the effect of randomization — but that’s not always the effect of treatment.

If everyone adheres perfectly, the effect of randomization is the effect of treatment (assuming no extra problems like loss to follow-up!).

But non-adherence changes things.

This is important for #pragmatic trials because randomization is not part of our clinical decisions.

In #pragmatic trials, we usually want to answer multiple questions:
•the effect of randomization
•the effect of the treatment protocol
•trial-specific effects

In #SCOTHEART we might want to know about how the effect of treatment works — that’s a type of causal question too!

So, what question(s) do they answer in the recent #SCOTHEART paper?

First, the intention-to-treat effect. I tried to write out the question this was answering based on their trial design & reported analyses but it was too long so you’ll have to read the picture👇🏽👇🏽

Second, they (tried to) estimate a per-protocol effect. From what I can tell they wanted to answer this question👇🏽👇🏽

But, the analysis that they used was a #landmark analysis. That means (1) it answered an importantly different question(👇🏽), and (2) it might not even have doesn’t answer this question correctly.

The problem with #landmark analyses follows directly from their design:

A landmark analysis restricts to the population of individuals who survive under follow-up and without the event of interest until some arbitrary time after randomization.

In #SCOTHEART, time=12 months.

Also, both the intention-to-treat and landmark effect definitions compared groups based on *randomization*, but the per-protocol effect compared *treatment* protocols.

That’s the 1st problem w/ landmark analysis: it doesn’t answer a question we care about!

The 2nd problem with landmark analysis is that it’s hard to get the right answer, even if we *did* care about that effect!

Why? Because we are restricting on a post-randomization variable: surviving event-free until 12 months (or whenever we picked).

Here’s the causal diagram

In this causal diagram, we can see that randomization and the event should only be related because of adherence to treatment. If you’re familiar w/ causal graph rules, you may also see that when we restrict on 12-month status, we induce an association bt/wn randomization & event.

If you aren’t familiar with the rules, here’s how it works:

If you’re event-free at 12 months, maybe it’s b/c you were assigned to & received CTA (and CTA helped). But, maybe it’s b/c your chest pain was never going to lead to an event anyway....

So at 12 months, CTA group will include some people with less severe chest pain *and* people for whom CTA was (at least short-term) protective, while the other group will now have proportionally *more* people with less severe chest pain.

We have basically created confounding!

We call this type of bias “collider bias” or “selection bias”.

If you want to know more about how to read that bias from the causal diagram, Chapter 4 of @yudapearl’s #bookofwhy explains the rules nicely.

So there are 2 last points to address: (1) how do we answer the question about the landmark effect? and (2) how do we answer the question about the per-protocol effect?

The answer is the same, although the specific details will differ.

When we care about effects of things that happen AFTER randomization, we need to use a statistical method that adjusts for confounding AFTER randomization *and* accounts for the fact that our exposure may also cause that confounding! (We call that treatment-confounder feedback)

For a nice review of the available methods, see this paper @NEJM👇🏽 nejm.org/doi/full/10.10…

And, that’s a wrap! Thanks for sticking around. I hope you learned something!

Which effect would you like to see #SCOTHEART estimate (using appropriate methods) to supplement their intention-to-treat (ITT) effect?