A Meta Analysis of 46 Career Pathways Impact Evaluations

In this study, the researchers conducted a meta-analysis of career pathways impact evaluations to summarize the effectiveness of these programs. Because the meta-analysis is based on impact evaluations that lead to causal conclusions, it allows the researchers to determine with high confidence whether career pathways programs impact educational progress and labor market outcomes. It also enables them to describe the magnitude of those impacts as an empirically computed average across the evaluations included in the meta-analysis.

Further, this meta-analysis seeks not only to determine the overall impact of the career pathways approach to workforce development but also to understand which particular characteristics of career pathways programs associate with those impacts. This second part of the analysis draws on patterns in the data. As such, it leads to suggestive but not causal conclusions.

In sum, this study contributes to the evidence base by:

1. Documenting the characteristics of 46 impact evaluations and the career pathways programs on which they focus.
2. Descriptively analyzing how those characteristics relate to one another and to program impacts.
3. Empirically aggregating the impact results to support causal conclusions about the impacts of the career pathways approach (as represented in this study’s 46 evaluations).

(This report is 145 pages.)

The researchers identified the following selected findings:

• The career pathways approach increased educational progress (measured as credential receipt) by a relatively large amount, corresponding to a 155 percent relative gain (based on 33 evaluations), an increase from 18 percent in the control/comparison group to 45 percent in the program group.
• The career pathways approach increased employment in the industry trained for by much more than it increased general employment. For industry-specific employment, the career pathways approach led to a 72 percent relative increase, an increase from 26 percent in the control/comparison group to 45 percent in the program group (based on 24 evaluations). However, for general employment, the approach led to only a 9 percent relative increase, from 60 to 66 percent of participants employed (based on 37 evaluations).
• The career pathways approach increased short-term earnings (i.e., earnings up through 35 months) by a very small amount, a 6 percent relative increase, corresponding to about $260 per quarter, from an average of $4,081 in the control/comparison group to $4,342 in the program group (based on 37 evaluations).
• The career pathways approach did not meaningfully increase medium/long-term earnings (i.e., earnings measured 36 or more months out, based on 16 evaluations), meaning that the program and control/comparison groups’ earnings were quite similar (just over $10,000 in both groups).

Researcher recommendations:

The researchers outlined several implications for policy and practice. First, the researchers suggest that policymakers and practitioners should consider how to translate the strong gains in educational progress into strong gains in labor market outcomes. The meta-regression explored some characteristics of programs that are associated with greater effectiveness, but the findings are exploratory and suggest avenues for future research. The researchers suggested that program design consider testing program components that are associated with effectiveness in past programs and that are suitable to the population served and the context in which the program operates.

• Last Updated:
• Created:
• Resource Publication Date: 2021
• Author(s): Peck; Schwartz; Strawn; Weiss.; Juras; Mills de la R.; Siobhan; Nathan; Tori, M.; Gabriel; Lloyd, C.
• Funding Source: U.S. Department of Labor, Employment and Training Administration
• Resource Availability: Publicly available
• Posted by: Wesley Peterson
• Posted in: Workforce System Strategies