#13 - Claire Walsh on testing which policies work & how to get governments to listen to the results

J-PAL Government Partnership Initiative (GPI)

• Structure: A multi-million dollar fund supporting three primary activities:
  • Funding policy-relevant randomized control trials (RCTs) in partnership with governments facing specific policy problems.
  • Providing technical assistance (human resources) to help governments build political will and technical knowledge to scale existing evidence-based interventions.
  • Supporting the creation of institutional mechanisms (evaluation labs, innovation funds, nudge units) to make evidence-informed decision-making the norm within governments.
• Scale: The initiative has funded 21 partnerships across 12 countries in Southeast Asia, South Asia, Sub-Saharan Africa, and Latin America.
• Selection Criteria: Governments must commit upfront to using the study results to inform specific policy decisions; the initiative is agnostic to region and sector.
• Impact: Since 2003, over 300 million people have been reached by programs originally evaluated by J-PAL-affiliated professors and subsequently scaled by partners.
• High-Impact Examples:
  • Deworming programs and free distribution of anti-malarial bed nets.
  • ID card reforms for national social protection programs in Indonesia.
  • Chlorine dispensers for safe water access.
  • Education interventions using the "Teaching at the Right Level" (Pratham) approach to improve numeracy and literacy.

Research and Policy Communication Strategies

• Core Philosophy: J-PAL aims to translate complex econometric papers (often 60 pages) into one-page summaries for policymakers.
• Digital Resources:
  • An evaluation clearinghouse featuring over 800 RCT summaries searchable by country and sector, with options to access publicly available raw data.
  • A "Policy Lessons" page highlighting areas with the strongest evidence of effectiveness.
  • "Research Resources" providing open-source Stata code, survey guides, and programming tutorials for aspiring researchers.
• Sector Focus: Research covers education, health, governance, environment, and energy; J-PAL actively challenges the misconception that impact evaluation is difficult in energy and environment sectors.
• Notable Findings:
  • A recent RCT in Uganda on payments for environmental services reduced deforestation rates from 7–10% to 2–5%.
  • Solar microgrid pilots in off-grid communities are used to assess willingness to pay and impacts on education and time allocation.
  • Experiments test drought/flood-resistant seed varieties to help smallholder farmers cope with climate change.

Career Insights and Advice from Claire Walsh

• Career Trajectory: Walsh advanced to Senior Policy Manager by age 29, attributing her rapid rise partly to the "APSEA" group of public policy master's programs which facilitate entry into senior roles in the late 20s/early 30s.
• Skill Gaps: She identified a lack of management, accounting, finance, fundraising, and technical skills (statistics, econometrics, impact evaluation) when starting in non-profits post-undergrad.
• Education Path:
  • A master's degree is recommended over a PhD for those aiming for policy action rather than pure academic research.
  • J-PAL and MIT now offer a low-cost, flexible MicroMasters and Blended Masters program in Data, Economics, and Public Policy for those unable to pursue full-time study.
• Networking and Experience:
  • Living in a developing country for at least two years is highly valuable for gaining cultural understanding, credibility, and operational skills in resource-poor environments.
  • Unpaid internships in developing countries are common entry points for young professionals.
  • Key conferences for networking include the American Economic Association (AEA) and the American Evaluation Association (AEA).
• Future Career Paths:
  • Professionals often transition from J-PAL to government roles (e.g., Office of Evaluation Services, Consumer Financial Protection Bureau, DC Lab) where they can directly influence spending decisions.
  • Alternatives include working for foundations to allocate endowment resources or joining organizations like Innovations for Poverty Action (IPA) or Ideas 42.

Challenges in Evidence-Based Policymaking

• Barriers to Adoption: Evidence is rarely the top factor in government decision-making (often ranked 8th or 9th); political priorities, constituent demands, and bureaucratic hiring restrictions often slow implementation.
• Risk of Harm: While low, risks include evaluating ineffective programs that waste resources or partnering taking away time from program implementation; Walsh argues the potential for "low-hanging fruit" gains outweighs these risks.
• Data Quality Concerns:
  • "Policy-based evidence-making" (slanted studies) exists, requiring rigorous scrutiny of balance tests, attrition, and spillovers.
  • Replication of RCTs is critical before scaling to prevent discrediting the evidence-based movement; however, representative sampling at scale can sometimes substitute for multiple small trials.
• Empiricism in Economics: Development economics is now over 90% empirical, with the majority of published papers focusing on data collection and analysis.

Philosophy and Global Equity

• Local Capacity: J-PAL regional offices are staffed by local nationals (e.g., J-PAL South Asia in Delhi/Chennai), emphasizing the importance of citizens from developing countries leading reforms within their own governments.
• Civil Society Role: Organizations like Twaweza in East Africa enable citizens to hold governments accountable for service delivery, offering an alternative to direct government work.
• Personal Impact Trade-off: Walsh acknowledges she could likely generate more immediate impact by earning high income and donating to cash transfers but prioritizes institutional strengthening to ensure sustainable, high-quality government service delivery.