Using the National Assessment of Educational Progress as an Indicator for College and Career…

“For more than 10 years, the National Assessment Governing Board (NAGB) and the National Center for Education Statistics…have been exploring whether the Grade 12 National Assessment of Educational Progress (NAEP) can be used to assess the college and career preparedness of high school students.…NAGB defines [college and career] ‘preparedness’ more narrowly, as ‘reading and mathematics knowledge and skills needed for placement into entry-level, credit-bearing, non-remedial courses in broad access 4-year institutions and, for 2-year institutions, the general policies for entry-level placement, without remediation, into degree bearing programs designed to transfer to 4-year institutions’.…As the only nationally representative measure of what 12th-grade students know and can do in reading and mathematics, NAEP provides a unique opportunity for measuring the college and career preparedness of the nation’s 12th graders” (p.1).

 

“This study contributes to the research on NAEP as an indicator of college and career preparedness by using longitudinal data on individual student achievement to relate NAEP scores to subsequent observed levels of postsecondary success. Using postsecondary institution records and state unemployment system data from the state of Virginia, this study examines the relationship between NAEP scores and a set of the most important markers of postsecondary success, ranging from progression to retention to completion, plus a critical measure of student success in the labor market after college completion.…

 

Using data on individual students spanning seven years for the cohort of recent high school graduates who entered four-year institutions in Virginia in the 2006–07 school year, this study addresses two questions:

1.       What is the empirically determined optimal (by criteria discussed below) NAEP mathematics cut score for distinguishing success?

2.       What are observed levels of success in college and in the labor market at various potential cut scores?” (p.1-2).

 

In total, “[t]he study population included the 35,293 students who had graduated high school within the previous 12 months, were under the age of 21, and enrolled in a 4-year institution in the 2006–07 academic year” (p.4).

(Abstractor: Author and Website Staff)

Full publication title: Using the National Assessment of Educational Progress as an Indicator for College and Career Preparedness

The authors present demographic characteristics of the study population and define the study’s four measures of success: progression, retention, completion, and earnings. The authors then report findings for each research question. The authors report “the empirically determined optimal [NAEP mathematics] cut scores for each success measure” (p.9). •“For Progression (earning more than 30 credits in the first year), the optimal cut score for predicting success was 181, or 5 points above the cut score for NAEP Proficient. •For Retention (returning for a second year), the optimal cut score was 174, or 2 points below the NAEP Proficient cut score. •For Completion (earning a bachelor’s degree within six years), the optimal cut score was 177, or 1 point above the NAEP Proficient cut score. •For Earnings (earning more than twice the poverty line in the year after graduation), the optimal cut score was 179, or 3 points above the NAEP Proficient cut score of 176; however, the estimated relationship between earnings and NAEP scores did not meet one of the study’s two pre-established criteria for accuracy and results should be interpreted with caution” (p.9). The authors then “examine the levels of student success at different points on the NAEP scale” (p.11). The authors find that the “…the levels of success across cut points for various outcomes differ. What is key…, however, is for a given measure, how different the levels of success are at the NAEP Proficient cut score (176) and the NAGB college academic preparedness cut score (163). The results indicate that between these two scores, the rates of success differ by the following: •7.9 percentage points for Progression •3.0 percentage points for Retention •8.6 percentage points for Completion •0.5 percentage points for Earnings The largest difference occurred in the levels of Completion: at the NAGB college academic preparedness cut score, about two thirds of the students were counted as successful; at the NAEP Proficient cut score, about three fourths were counted as successful” (p.11). The “study population..is likely not reflective of the entire national grade 12 population.... For more generalizable results, these analyses should be replicated with a full population of 12th graders at the state level or with a nationally representative sample of 12th graders” (p.13). (Abstractor: Author and Website Staff)

• Last Updated:
• Created:
• Resource Publication Date: 2015
• Author(s): Schneider, Mark; Kitmitto, Sami; Muhisani, Helen; and Zhu, Bo.
• Organizational Author(s): American Institutes for Research
• Funding Source: Funding source not identified
• Resource Availability: Publicly available
• Posted by: Laura Aron
• Posted in: Workforce System Strategies