Kristen Olson Profile picture
Jul 31, 2018 38 tweets 12 min read Twitter logo Read on Twitter
#JSM2018 A highly distinguished all-women panel for the 1 year update on the Commission on Evidence-Based Policy Making. Katharine Abraham, Nancy Potok, Amy O’Hara, and Julia Lane
#JSM2018 Abraham Commission co-sponsored by Paul Ryan & Patty Murray. 2/3 appointed based on substantive issues re: data, 1/3 appointed based on expertise in privacy.
#JSM2018 Abraham Had one year to do report - 7 public meetings, public hearings held across US, put out request for comments, talked to federal offices. 8 months of data gathering, 4 months of writing. Wrote a report that had unanimous support from all commissioners.
#JSM2018 Abraham No clear partisan splits in writing recommendations. Biggest split between people who came wanting to increase access to data for research, and those who had privacy background
#JSM Potok Data Stewardship issues were the area with the strongest discussions. How do people access data in a secure way?
#JSM2018 Abraham Researchers were frustrated with lack of access to data; officials said access to data would make work better. Public was concerned about student unit data ban. Also concerned about evaluation capacity of federal government.
#JSM2018 Abraham In 1960s, proposal for bringing all government data together for research purposes. People were not happy about this. So, commission was aware of this and decided to not make this recommendation.
#JSM2018 Potok Decided to not look at structure of federal statistical system.
#JSM2018 Abraham Most important findings: improve access to data for research purposes. In report, access to data is for statistical purposes only. Especially improve access to data for linking multiple sources.
#JSM2018 Abraham Data live at states that can be hard to access; hard to share across agencies. Recommend developing national secure data service where propose project, data are compiled/linked, do the project, go through review, then disassemble data.
#JSM2018 Abraham Need to have changes to legal barriers to make data sharing easier.
#JSM2018 Potok Looked at dissemination end - recommended mandatory review for privacy, disclosure risk. Splintered landscape across federal agencies re: data governance and data stewardship. Recommended having senior official in charge of privacy
#JSM2018 Potok Consent statements become important- consent to use data for statistical programs. But program administration may not do this. Needs to be an inventory of all of the datasets available, all projects being done. Need to build capacity for evaluation in agencies
#JSM2018 Potok Agencies should have a learning agenda, including what want to learn from Data. Ask bigger, important questions about how effectively programs are being administered. Have chief evaluation officers at agencies to identify that studies are rigorous
#JSM2018 Potok Hard to find good models for evaluation at agencies. Federal statistical agencies have good models for data protection; other agencies don’t have any framework. Privacy Act is implemented differently across agencies at Cabinet level
#JSM2018 O’Hara Folks outside government didn’t have thoughts that big changes would happen right away. Thought recommendations would make a dramatic difference. Incentives need to be placed differently to make data available.
#JSM2018 O’Hara State, county agencies want to know whether shared access to data would benefit the program administration.
#JSM2018 Lane Research community is standing back to see what happens. Not sufficient to just put Data in an environment. Need metadata on the data. Computer scientists interested in this. Need to gather interest of data science programs to increase capacity
#JSM2018 O’Hara Importance of getting buy-in from agencies can’t be understated.
#JSM2018 Lane Need sufficient resources. It’s a long, brutal effort to do this. Need institutional support, other funding support.
#JSM2018 O’Hara Need to have willingness for private sector computer to engage in this work.
#JSM2018 Abraham Commission decided to focus on federal data, but expanding to private companies in future relevant.
#JSM2018 Lane Computer scientists can help with building information retrieval system that can code metadata on disparate datasets, built for non-statistical projects.
#JSM2018 O’Hara Current data access very difficult, especially at Census Bureau. Can’t access if not approved for particular project (this is good in general), but getting approval for particular projects very challenging
#JSM2018 Potok Looking at Data access through RDCs. How to streamline and make it easier for researchers to access data? Use FSRDCs as trial point for access to data. Can there be one application, one portal for researchers to get data access in the RDCs?
#JSM2018 Abraham Legislation passed House for Evidence-Based policy making. Legislation has been sitting in the Senate. Needs a champion to get it passed.
#JSM2018 Potok OMB is playing pro-active role. Creating federal data strategy. Long term effort of looking into future of data. Looking at access and use, governance issues, categories of data including open data, and decision making
#JSM2018 Potok is the place for this federal data strategy. Help give something back to agencies who share data. Will look at initial projects for barriers to access to data.
#JSM2018 Potok OMB required agencies to submit senior evaluation official and learning agendas when submitting budgets. Get budget people focused on understanding how programs work, get resources for incubator projects.
#JSM2018 Potok Collaboration with academic researchers to develop common vocabulary for Linkage, etc. Have FCSM develop working papers with recommendations. Proposed moving BLS into Department of Commerce to facilitate Data sharing
#JSM2018 Moving BLS into Commerce facilitates sharing across agencies, including sharing tax data. Spend 2019 and 2020 to start planning effort
#JSM2018 Lane Get agency buy-in by creating products that have clear value to agency. Can’t be done in a void - engage foundations, people in agencies. This is sustainable b/c agencies care about it. Get lots of ideas generated, see which ideas are doable, scalable
#JSM2018 Potok Agencies are really eager to start using data differently.
#JSM2018 Lane In New Zealand, all policy has to be evidence-based now.
#JSM2018 O’Hara Both survey data and admin data have errors, need agenda on blended statistics from these initial projects
#JSM2018 Potok Has been working with Deputy Secretaries to develop learning agendas. OMB says have to come talk with them about learning agendas.
#JSM2018 Potok One group looking at FSRDCs is looking at financial structure. May include future structure that involves fees. But Potok thinks most effective and efficient way is to put money into budget for this work
#JSM2018 Report available at for one more year

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Kristen Olson

Kristen Olson Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @olson_km

Aug 2, 2018
#JSM2018 Tobias Schmidt Looking at interviewer experience and interview duration
#JSM2018 Schmidt In this survey, duration linked to interviewer salaries.
#JSM2018 Schmidt Looking at interviewer experience over the course of survey and respondent experience within survey and experience over repeated surveys. Looking in particular at experience within panel survey for both Iers and Rs
Read 9 tweets
Aug 2, 2018
#JSM2018 Celine Wuyts from KU Leuven up next.
#JSM2018 Wuyts Interested in within-survey workload. Use call history data and interview time data. Some Measure workload by fixed measures of experience and interview order cumulated over the field period. They use actual number of cases assigned at time t in field period
#JSM2018 Wuyts Use Paradata to create new measures of interview workload, based on sample units assigned on given day
Read 7 tweets
Aug 2, 2018
#JSM2018 Rebecca Powell from @RTI_Intl talking about an experiment on Add Health shifting from interviewer administered to self administered survey
#JSM2018 Powell moved to a 55 self-administered survey from 90 minutes interviewer administered. Worried about response burden with this length of self-admin survey. Randomized n=7600 into either full 55 minute survey or 2 modules- one 35 minutes then 20 minutes.
#JSM2018 Powell Could select to continue on the web. In paper, had to first complete module A, then sent module B. Cover letters told about modules in the incentive part, but not up front. $55 incentive total in each condition
Read 12 tweets
Aug 1, 2018
#JSM2018 The brilliant Susan Murphy is this year’s Fisher Lecture award recipient!
#JSM2018 Murphy Lab does sequential experimentation in improving health. Some for companies.
#JSM2018 Murphy Experimentation and continual optimization is key. How do we use learning as an experiment is put into the field to improve outcomes for individuals? Mobile interventions are key here. Intervention may be either a push intervention or pull intervention
Read 30 tweets
Aug 1, 2018
#JSM2018 Next up Hubert Hamer from NASS talking about NASS Small Area Estimation
#JSM2018 Hamer NASS has Agriculture Loss Coverage County Option program. Payments triggered based on county crop revenue falling below program guarantee. NASS surveys used to make this decision, along with other data
#JSM2018 Hamer Program paid out $3.7 billion on 2016. Small changes can affect payments
Read 13 tweets
Aug 1, 2018
#JSM2018 Peter Miller appearing as a Northwestern University emeritus professor, providing comments on the CNSTAT reports
#JSM2018 Miller Survey paradigm vs multiple data source paradigm. Surveys may become irrelevant b/c they are slow, not granular, not nimble, costly, not sustainable
#JSM2018 Miller Multiple Data sources require new: methods, computing resources, privacy protections, training, data quality frameworks. Not cheap. What does this give us?
Read 9 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!


0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy


3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!