Kristen Olson Profile picture
Aug 1, 2018 30 tweets 10 min read Twitter logo Read on Twitter
#JSM2018 The brilliant Susan Murphy is this year’s Fisher Lecture award recipient!
#JSM2018 Murphy Lab does sequential experimentation in improving health. Some for companies.
#JSM2018 Murphy Experimentation and continual optimization is key. How do we use learning as an experiment is put into the field to improve outcomes for individuals? Mobile interventions are key here. Intervention may be either a push intervention or pull intervention
#JSM2018 Murphy Pull intervention requires you to be aware that you need help. Push can go without you doing anything but can have negative impact (people delete app)
#JSM2018 Murphy Sense2Stop is designed as a smoking cessation program. Use stress reduction method to buffer real-like stressors. Should device notify you to go to exercises?
#JSM2018 Murphy Participant wears sensors that measures physiological responses. Has machine learning algorithm determining when you are stressed. Is a reminder effective? Does it vary by context? Thus, stratified micro-randomized trials
#JSM2018 Murphy Looking at time intervals of every minute for 10
Days straight. !!! Data comes in at different levels throughout the day (sensor, weather, etc.)
#JSM2018 Murphy Contains an indicator of whether appropriate to try to provide a treatment (no treatment while driving, for example).Only randomize if there is more than one treatment available at the time. If randomized intro treatment, then remind to access mindfulness exercise
#JSM2018 Murphy Proximal response is what you did within an hour of the treatment. (Can pick other time frames)
#JSM2018 Murphy Randomization means we can assess causal effects of the reminder and whether varies by context. Need to stratify so that you can actually have time points where you are actually stressed and not stressed.
#JSM2018 Murphy On average, for every 1 minute stressed, participants had 6 minutes not stressed
#JSM2018 Murphy Need to give a budget for how often it’s appropriate to try to provide treatment. Constrained this experiment to about 1.5 times per day for times stressed and not stressed
#JSM2018 Murphy I just made up an optimization criterion of 1.5 times per day on average for each stressed/not stressed time. Have uniform distribution of pinging participants across all times of day (not just in morning)
#JSM2018 Need to forecast expected number of times you will be stressed during today. Probability of getting treatment depends on desire number of treatments per day, how many treatments received, and anticipated number of future stressed times
#JSM2018 Murphy Schematic for the study
#JSM2018 Murphy What about the causal treatment effect? Call it causal excursion effect. Coming from a potential outcomes framework.
#JSM2018 Murphy Following Rubin’s potential outcomes. All of the treatments that occur could affect your outcome to all time up to time up to and including 59 minutes later from the treatment.
#JSM2018 Murphy Have a collection of all the Ys that could occur on your forehead. Researchers see a subset of these.
#JSM2018 Murphy All excursions have a time dimension on them - time of all treatments up to now and getting the treatment vs same collection of treatments and not getting the treatment
#JSM2018 Murphy Of course, individual level causal effect is not estimable. So need to look at averages. Causal excursion effect at time t beginning in strata x
#JSM2018 Murphy Now we need to place our bets on a particular hypothesis and one particular question. Want to contrast two treatments now and for the subsequent hour. So primary test - is there a signal going on here at all? Test of main effect
#JSM2018 Murphy Concerned about diminishing returns to treatment. So alternative is a quadratic decreasing effect and a linear decreasing effect.
#JSM2018 Murphy Can develop models that are very similar to GEE - familiar to many practicing statisticians. Use weighted and centered least squares.
#JSM2018 Murphy Can use contrast coding - adds robustness to test. Use a projection of excursion effect through time.
#JSM2018 Murphy Then use simulation based sample size calculator. Low dimensional alternative hypotheses increase power
#JSM2018 Murphy Treatment design and experimental design intertwined here. Treatment design is an algorithm! Developing part of the treatment, not just evaluating the treatment.
#JSM2018 Murphy has a music cue! This should be our mantra, Murphy urges.
#JSM2018 Murphy We are the ones who enrich science through experimentation.
#JSM2018 Murphy This requires lots of collaboration
#JSM2018 And this is exactly why Susan Murphy is a MacArthur Genius!

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Kristen Olson

Kristen Olson Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @olson_km

Aug 2, 2018
#JSM2018 Tobias Schmidt Looking at interviewer experience and interview duration
#JSM2018 Schmidt In this survey, duration linked to interviewer salaries.
#JSM2018 Schmidt Looking at interviewer experience over the course of survey and respondent experience within survey and experience over repeated surveys. Looking in particular at experience within panel survey for both Iers and Rs
Read 9 tweets
Aug 2, 2018
#JSM2018 Celine Wuyts from KU Leuven up next.
#JSM2018 Wuyts Interested in within-survey workload. Use call history data and interview time data. Some Measure workload by fixed measures of experience and interview order cumulated over the field period. They use actual number of cases assigned at time t in field period
#JSM2018 Wuyts Use Paradata to create new measures of interview workload, based on sample units assigned on given day
Read 7 tweets
Aug 2, 2018
#JSM2018 Rebecca Powell from @RTI_Intl talking about an experiment on Add Health shifting from interviewer administered to self administered survey
#JSM2018 Powell moved to a 55 self-administered survey from 90 minutes interviewer administered. Worried about response burden with this length of self-admin survey. Randomized n=7600 into either full 55 minute survey or 2 modules- one 35 minutes then 20 minutes.
#JSM2018 Powell Could select to continue on the web. In paper, had to first complete module A, then sent module B. Cover letters told about modules in the incentive part, but not up front. $55 incentive total in each condition
Read 12 tweets
Aug 1, 2018
#JSM2018 Next up Hubert Hamer from NASS talking about NASS Small Area Estimation
#JSM2018 Hamer NASS has Agriculture Loss Coverage County Option program. Payments triggered based on county crop revenue falling below program guarantee. NASS surveys used to make this decision, along with other data
#JSM2018 Hamer Program paid out $3.7 billion on 2016. Small changes can affect payments
Read 13 tweets
Aug 1, 2018
#JSM2018 Peter Miller appearing as a Northwestern University emeritus professor, providing comments on the CNSTAT reports
#JSM2018 Miller Survey paradigm vs multiple data source paradigm. Surveys may become irrelevant b/c they are slow, not granular, not nimble, costly, not sustainable
#JSM2018 Miller Multiple Data sources require new: methods, computing resources, privacy protections, training, data quality frameworks. Not cheap. What does this give us?
Read 9 tweets
Aug 1, 2018
#JSM2018 Panel on CNSTAT report on Federal Statistics, Multiple Data Sources, and Privacy Protection, with @fraukolos kicking off The discussion
#JSM2018 @fraukolos Goal of panel to evaluate combining data sources to possibly replace / augment surveys. Two reports out of the panel.
#JSM2018 @fraukolos Conclusions: Current Federal Statistical Agencies face threats from falling response rates, rising costs, increased desire for granularity and timelines
Read 13 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!


0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy


3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!