Randomized Controlled Trials and Financial Capability: Why, When, and How

“Policy researchers agree that randomized controlled trials (RCTs) often produce the highest standard of evidence about an intervention’s effectiveness. Given the promise that RCTs hold, but also the difficulty of successfully implementing these studies, the Urban Institute and the Consumer Financial Protection Bureau (CFPB) convened a roundtable on the benefits and challenges of RCTs in April 2013. In addition to Urban Institute and CFPB staff, 26 evaluators, funders, and program staff—all of whom were involved in ongoing or recent evaluations of financial literacy, service, or education programs—discussed their experiences and shared practical insights into successful strategies and pitfalls of rigorous evaluations of financial capability interventions. This brief shares those insights with other researchers, practitioners, and funders undertaking or contemplating rigorous effectiveness research in the financial capability realm” (p. 1).

Recommendations regarding implementation and proper use of randomized control studies discussed in this brief are clustered under five topic areas:

1. Why conduct financial capability research?
2. What is a randomized controlled trial?
3. When is a randomized controlled trial suitable?
4. What hinders a strong study? How can these challenges be overcome?
5. How does this discussion shape practice, policy and funding?

Specific recommendations for practitioners, evaluators, funders, and policymakers outlined in the fifth topic area are defined below:

Two key steps for practitioners:

• Encourage a ‘culture of data’ in their operations, and prioritize tracking participants and collecting as much information as possible on their activities and outcomes.
• Use data to isolate key impact metrics, those indicators that programs feel they are able to shift the needle on.

Three key steps for evaluators, together with funders and policymakers:

• Develop strategies and systems to make data collection easier, faster, less expensive, and more efficient. Both researchers and practitioners will benefit from standardized, affordable, and easy-to-use data collection and management programs.
• Work with programs to develop ‘priority outcomes,’ accepted financial capability outcomes shared across evaluation studies.
• Design standardized data-collection efforts around the priority outcomes, once established, and ensure that these outcomes are included in subsequent evaluations.

• Last Updated:
• Created:
• Resource Publication Date: 2014
• Author(s): Brash, Rachel; Emam, Dina; Simms, Margare;t Sharygin, Claudia; Theodos, Brett
• Organizational Author(s): The Urban Institute
• Funding Source: Funding source not identified
• Resource Availability: Publicly available
• Posted by: Wesley Peterson
• Posted in: Workforce System Strategies