#JSM2018 Waiting for John Eltinge from @uscensusbureau to deliver the Deming Lecture: “Improving the Quality and Value of Statistical Information: 14 Questions on Management”
#JSM2018 Eltinge Deming and his work shaped by his time-1900 to 1993. Big changes in industry, government, how citizens related to science.
#JSM2018 Eltinge Deming involves in both management and sampling. thus, control charts!
#JSM2018 Eltinge Will apply Deming’s 14 points to 14 questions for management.
#JSM2018 Eltinge Our profession cares about quality of statistical information and applying it to statistical products (tables, microdata)
#JSM2018 Eltinge Currently experiencing major changes: non-designed Data, changes in tools we have for analysis, changes in expectations about privacy as well as granularity of Data desired
#JSM2018 Eltinge Example: Integrate survey data by linking to admin/commercial data. (Append data to surveys) Example B: “Backbone and Bridge” Admin data as frame/basis for sample survey
#JSM2018 Eltinge we have measure of quality, risk and cost; then environmental factors; then design decisions. High dimension vectors.
#JSM2018 Eltinge Important to distinguish between the occurrence of an event that may affect quality and the actual effect on quality (missing data indicator vs effect of NR bias or efficiency of an estimator)
#JSM2018 Eltinge Surveys care about multiple dimensions of cost.
#JSM2018 Eltinge The environment can have N effect on the quality, risk, and cost of surveys. Can affect some things we can control, or that we can’t control. Use design to account for environmental factors, affecting quality profiles, for particular user group
#JSM2018 Eltinge High quality statistical work requires a lot of resources (cash, human, other intangibles).
#JSM2018 Eltinge “Walk humbly with our data” Citing Phil Kott and others.
#JSM2018 Eltinge Hard to optimize for survey quality when environment affects potential design factors, don’t know when design may have differential effects on quality for different environments (interaction effects). Also experience “slippage” b/w ideal and actual operations
#JSM2018 Eltinge Connect ideas behind statistical control to design - is a problem easily identified by common and special causes? Changes in environment? Slippage in design implementation?
#JSM2018: Eltinge Has 14 questions for us to think about
#JSM2018 Eltinge What is the well-defined use for a particular piece of statistical information? (Use value) what is the value of this piece of statistical information for future use? (Option value)
#JSM2018 Eltinge Can we foster an institutional culture that will allow us to change based on environmental changes?
#JSM2018 Eltinge Even the best-designed systems don’t run on auto-pilot.
#JSM2018 Eltinge How do we improve investments on intangible capital? how do we use our capital to improve an organization? What are the skills that we need now? In the future? And how do we put incentives in place to get people to work together?
#JSM2018 Eltinge Statistical groups struggle with developing shared software for multiple projects. How to incentivize this?
#JSM2018 Eltinge “Problems of people” (Deming phrase) can dominate all problems of management.
#JSM2018 Do we have realistic approaches to training on what skills we need for a new job? In a perfect world, everyone knows everything, but that isn’t realistic
#JSM2018 Eltinge What can an organization do to help employees thrive in changing environment?
#JSM2018 Eltinge What are characteristics of management/ leadership? Need to build organizational consensus, identify values, communicate them
#JSM2018 Eltinge When integrating multiple data sources, what does this mean for Data as public goods?
#JSM2018 Eltinge Think long and hard about human condition in management
• • •
Missing some Tweet in this thread? You can try to
force a refresh
#JSM2018 Tobias Schmidt Looking at interviewer experience and interview duration
#JSM2018 Schmidt In this survey, duration linked to interviewer salaries.
#JSM2018 Schmidt Looking at interviewer experience over the course of survey and respondent experience within survey and experience over repeated surveys. Looking in particular at experience within panel survey for both Iers and Rs
#JSM2018 Wuyts Interested in within-survey workload. Use call history data and interview time data. Some Measure workload by fixed measures of experience and interview order cumulated over the field period. They use actual number of cases assigned at time t in field period
#JSM2018 Wuyts Use Paradata to create new measures of interview workload, based on sample units assigned on given day
#JSM2018 Rebecca Powell from @RTI_Intl talking about an experiment on Add Health shifting from interviewer administered to self administered survey
#JSM2018 Powell moved to a 55 self-administered survey from 90 minutes interviewer administered. Worried about response burden with this length of self-admin survey. Randomized n=7600 into either full 55 minute survey or 2 modules- one 35 minutes then 20 minutes.
#JSM2018 Powell Could select to continue on the web. In paper, had to first complete module A, then sent module B. Cover letters told about modules in the incentive part, but not up front. $55 incentive total in each condition
#JSM2018 The brilliant Susan Murphy is this year’s Fisher Lecture award recipient!
#JSM2018 Murphy Lab does sequential experimentation in improving health. Some for companies.
#JSM2018 Murphy Experimentation and continual optimization is key. How do we use learning as an experiment is put into the field to improve outcomes for individuals? Mobile interventions are key here. Intervention may be either a push intervention or pull intervention
#JSM2018 Next up Hubert Hamer from NASS talking about NASS Small Area Estimation
#JSM2018 Hamer NASS has Agriculture Loss Coverage County Option program. Payments triggered based on county crop revenue falling below program guarantee. NASS surveys used to make this decision, along with other data
#JSM2018 Hamer Program paid out $3.7 billion on 2016. Small changes can affect payments
#JSM2018 Peter Miller appearing as a Northwestern University emeritus professor, providing comments on the CNSTAT reports
#JSM2018 Miller Survey paradigm vs multiple data source paradigm. Surveys may become irrelevant b/c they are slow, not granular, not nimble, costly, not sustainable
#JSM2018 Miller Multiple Data sources require new: methods, computing resources, privacy protections, training, data quality frameworks. Not cheap. What does this give us?