Merit Aid, College Quality and College Completion: Massachusetts’ Adams Scholarship as an In-Kind Subsidy

We analyze a Massachusetts merit aid program in which high-scoring students received tuition waivers at in-state public colleges with lower graduation rates than available alternative colleges. A regression discontinuity design comparing students just above and below the eligibility threshold finds that students are remarkably willing to forgo college quality for relatively little money and that marginal students lowered their college completion rates by using the scholarship. These results imply that college quality has a substantial impact on college completion rates and that students likely do not understand this fact well. The theoretical prediction that in-kind subsidies of public institutions can reduce consumption of the subsidized good is shown to be empirically important.


Introduction
Recent research has emphasized troubling trends in U.S. college completion rates over the past few decades (Belley andLochner 2007, Bailey andDynarski 2011). Among students entering college, completion rates are lower today than they were in the 1970s, driven largely by the low completion rates of men and students from lower socioeconomic backgrounds. The factors driving these low completion rates are a matter of current debate. Poor academic preparation of high school students is thought to play some role, given that college enrollment has expanded to larger and larger portions of the high school population, but that alone does not seem sufficient to explain low completion rates. Controlling for rich sets of student characteristics does not eliminate wide variation among postsecondary institutions in completion rates (Bowen et al. 2009).
Much more of the completion rate puzzle is explained by the changing characteristics of colleges themselves, particularly the resources available at such institutions. Bound and Turner (2007) show that large cohorts within states have relatively low college completion rates, which likely stems from decreased resources per student in those cohorts given states' tendencies to change public postsecondary budgets slowly. Bound et al. (2010) argue that the vast majority of the decline in completion rates can be statistically explained by decreasing resources per student within institutions and, even more importantly, shifts in enrollment toward the relatively poorly funded public sector. Much of the literature on the impact of college quality on degree completion focuses on the community college sector, reaching conflicting conclusions about whether access to and quality of community colleges affects educational attainment (Rouse 1995, Leigh and Gill 2003, Sandy et al. 2006, Calcagno et al. 2008, Stange 2009, Reynolds 2012. The central challenge of estimating the impact of college quality on college completion is the non-random selection of students into institutions of different apparent quality. The aforementioned research attempts to eliminate such selection bias by either by conditioning on observable characteristics of students or by instrumenting college quality with distance from or tuition of nearby colleges. Neither of these approaches is entirely satisfactory and leave open the strong possibility that unobserved student-level characteristics may be driving the conflicting results such research reports. Similar problems confound estimates of the impact of college quality on labor 1 market earnings, many of which are positive (Loury and Garman 1995, Brewer et al. 1999, Chevalier and Conlon 2003, Black and Smith 2004, Black and Smith 2006, Long 2008, Hoekstra 2009), some of which are zero Krueger 2002, Dale andKrueger 2011), and some of which suggest that that earnings differences dissipate once the job market properly understands graduates' underlying ability (Brand andHalaby 2006, Lang andSiniver 2011).
Of all the research on college quality of which we are aware, only Hoekstra (2009) uses an identification strategy that convincingly eliminates selection bias. There, the author identifies the impact of attending a flagship state university on earnings by exploiting an admissions cutoff rule that allows for a fuzzy regression discontinuity design. Unfortunately, he only observes the enrollment decisions of those admitted by this rule and can only guess at the counterfactual enrollment decisions driving the observed differences in earnings. It is thus quite rare to find a plausibly exogenous variation in college quality for a given student. We have, however, found such a source of variation in college quality in a setting where we observe the enrollment decisions of all students.
The policy we exploit is a Massachusetts merit aid program in which high school students with test scores above multiple thresholds were granted tuition waivers at in-state public colleges of lower quality than the average alternative available to such students. The scholarship value was high enough to induce substantial variation in college choice but low enough that the induced shift in college quality was vastly more important than the money itself.
We use this program to provide the first clear causal evidence on the impact of college quality on students' postsecondary enrollment decisions and rates of degree completion, where college quality is defined by a variety of measures including on-time graduation rates. A binding score regression discontinuity design comparing students just above and below these thresholds yields two main findings. First, students are remarkably willing to forego college quality for relatively small amounts of money. Second, choosing a lower quality college significantly lowers on-time completion rates, a result driven by high-skilled students who would otherwise have attended higher quality colleges. For the marginal student, enrolling at an in-state public college lowered the probability of graduating on time by more than 40%. The low completion rates of scholarship users imply that the program had little impact on the in-state production of college degrees.
More broadly, these results suggest that the critically important task of improving college quality requires steps beyond merely changing the composition of the student body.
The structure of the paper is as follows. In section 2, we describe the merit scholarship program in detail. In section 3, we describe the data and the characteristics of the students under consideration. In section 4, we explain our empirical strategy, a regression discontinuity design that accounts for the multiple thresholds students must cross in order to be eligible for aid. In section 5, we present estimates of the impact of college quality on enrollment decisions and completion rates. In section 6, we discuss the implications of our results for the state of Massachusetts.
In section 7, we conclude by discussing implications postsecondary policy more generally.

The Adams Scholarship
All Massachusetts public high school 10th graders take the Massachusetts Comprehensive Assessment System (MCAS), which includes an English language arts portion and a mathematics portion. Scores on each portion range in multiples of two from 200 to 280, with 260-280 categorized as "advanced" and 240-258 as "proficient". In January 2004, Massachusetts Governor Mitt Romney proposed the John and Abigail Adams Scholarship Program, which would waive tuition at in-state public colleges for any student whose total MCAS score placed him or her in the top 25% of students statewide. 1 Romney's primary goals were to keep highly talented students in state and improve the quality of the state's public postsecondary institutions. In his January 15, 2004 State of the State speech to the Massachusetts legislature, Governor Romney explained that "I want our best and brightest to stay right here in Massachusetts." 2 Concerned that Governor Romney's statewide standard would assign scholarships largely to 1 The eponymous couple cared deeply about education. John Adams wrote, in the Massachusetts Constitution, that "Wisdom, and knowledge, as well as virtue... as these depend on spreading the opportunities and advantages of education in the various parts of the country, and among the different orders of the people, it shall be the duty of legislatures and magistrates, in all future periods of this commonwealth, to cherish the interests of literature and the sciences, and all seminaries of them; especially the university at Cambridge, public schools and grammar schools in the towns" (Chapter V, Section II). Abigail Adams, disturbed by the 18th century gender gap, wrote that "It is really mortifying, sir, when a woman possessed of a common share of understanding considers the difference of education between the male and female sex, even in those families where education is attended to" (Letter to John Thaxter, February 15, 1778). students in wealthy, high-performing school districts, the state Board of Higher Education ultimately approved a modified version of the program in October 2004, nine months after the original proposal. 3 Under the approved policy, a student receives a tuition waiver if his or her MCAS scores fulfill three criteria. First, he or she must score advanced on one portion of the exam. Second, he or she must score at least proficient on the other portion of the exam. Third, the student's total MCAS score must fall in the top 25% of scores in his or her school district. 4 The scores used to determine eligibility come from each student's first attempt at taking the grade 10 MCAS tests in ELA and mathematics. To receive the scholarship, a student must be enrolled in and graduate from a Massachusetts public high school in his or her senior year. The graduating class of 2005 was the first to receive the scholarships according to these eligibility criteria and the program has continued in this form to the present day. Figure 1 provides a graphical interpretation of scholarship eligibility in a typical mediumperforming school district. About half of the graduates attend such districts (those with cutoff scores between 502 and 518), where the cutoff score interacts with the proficient/advanced threshold in a complex way. Students must pass the proficient/advanced threshold represented by the thick solid line and must also achieve their own district's cutoff score, represented by the thick dashed line. Scholarship winners are those students whose test scores fall in the shaded region.
In low-performing districts (with cutoff scores of 500 or lower), the cutoff is so low that passing the proficient/advanced threshold is sufficient to win a scholarship, whereas in high-performing districts (with cutoff scores of 520 or higher), the cutoff is so high that passing it alone is sufficient to win. These complexities will inform construction of the eligibility variable to be used in the regression discontinuity estimates. We also note that MCAS scores have risen since the inception of the program, as shown in Figure 2. Each year a higher fraction of high school graduates qualifies for the Adams Scholarship, with the district cutoff more often becoming the relevant eligibility threshold.
Scholarship winners are automatically notified in the fall of their senior year, eliminating the selection bias that confounds evaluations of aid programs requiring applications. The scholarship waives tuition at any of fifteen (two-year) community colleges, nine (four-year) state colleges, or four University of Massachusetts (U. Mass.) campuses. 5 Receipt of the scholarship does not, however, eliminate the cost of college attendance. Figure 3 shows the tuition and fees at the University of Massachusetts at Amherst and Bridgewater State College, the two largest campuses in their respective sectors. Strikingly, at both campuses and nearly all other public Massachusetts colleges, tuition has remained constant in nominal terms over the past decade. Mandatory fees have, however, risen dramatically. 6 The letter that Governor Romney sent to the first class of scholarship recipients promised in bold-faced and underlined letters "four years of free tuition." Perhaps due to confusion this engendered in students and families unaware of the distinction between tuition and the overall cost of attendance, more recent letters to recipients have emphasized that the scholarship "covers tuition only, and does not include college fees." 7 Thus, for the first class of Adams scholars in fall 2005, the tuition waiver was annually worth $1,714 if used at U. Mass. Amherst or $910 if used at Bridgewater State. Given mandatory fees of $7,566 at U. Mass. Amherst and $4,596 at Bridgewater State, the Adams Scholarship thus respectively represented a 17-18% reduction in the direct cost of attendance. By the fall of 2010, fees had risen by roughly a third, so that the Adams Scholarship represented only a 13-15% reduction in the cost of attendance. These percentages would be substantially lower if room, board and other expenses were added to the cost of attendance. The Adams Scholarship thus lowers the cost of college attendance by well under 20%, is worth at most $6,856 (4*$1,714) over four years, and is about one half the value of a Georgia HOPE award and one sixth the value of a CalGrant award (Dynarski, 2008;Kane, 2003). By all of these measures, the Adams Scholarship represents a relatively small amount of financial aid. within four years of high school graduation. We also examine more recent classes but can only observe college enrollment, and not completion, for such students.
College outcomes come from DESE's merge of its data on high school graduates with the National Student Clearinghouse (NSC) database, which covers 94% of undergraduates in Massachusetts. 10 We observe for each high school graduate every college enrollment spell through 2011, including the specific college attended, dates of attendance, and college location and type.
8 Scholarship users must also be a U.S. citizen or permanent resident of the U.S. and must have been a permanent legal resident of Massachusetts for at least one year prior to entering college as a freshman. 9 98.3% of those who receive the Adams scholarship letter in the fall of 12th grade ultimately graduate from high school. Regression discontinuity analysis, as described further below, suggests that receipt of this letter had no impact on high school graduation rates. This sample restriction therefore does not create selection bias. 10 The remaining 6% come largely from for-profit institutions and those whose highest degrees take less than two years to complete. Such institutions tend to enroll students with relatively low academic skill, so that the overall match rate for those eligible for the Adams Scholarship is likely substantially higher than 94%. 6 We also observe graduation if it occurs. We merge on additional college characteristics such as college costs and quality measures from the U.S. Department of Education's Integrated Postsecondary Education Data System (IPEDS) and the 2009 Barron's rankings of colleges. We separate colleges into Adams eligible institutions (U. Mass. campuses, state colleges and community colleges), private in-state institutions, and out-of-state institutions. For each student and type of college, we construct two primary outcome indicators, one for enrolling full-time by the fall following high school graduation and one for earning a college degree within four years of high school graduation. Table 1 shows the mean characteristics of high school graduates from the classes of 2005-2007.
Column (1) contains the full sample of students. Columns (2) and (3) split the sample by Adams Scholarship eligibility. Column (4) restricts the sample to those who use the Adams Scholarship by attending a public college within Massachusetts. Column (5) restricts the sample to students sufficiently close to the eligibility threshold to be used in our subsequent regression discontinuity analysis, as will be described below. Comparing columns (2) and (3) reveals that Adams eligible students are less likely than Adams ineligible students to be low income, black or Hispanic, or enrolled in special education, because these characteristics are negatively associated with the test scores determining eligibility. Column (2) also reveals that 82% of Adams eligible students enroll in college full-time by the fall following their high school graduation. Of these, 30% enroll in-state at public colleges, 22% enroll in-state at private colleges, and 31% enroll out of state. More than half of the in-state public enrollment occurs at U. Mass. campuses. Comparing columns (2) and (4) shows that those who use the Adams Scholarship are more likely to be low income and have slightly lower test scores than the average Adams eligible student.
Perhaps the most striking fact in this table is the large gap between college enrollment rates in panel C and graduation rates in panel D. In the full sample, only 30% of students graduate within four years, relative to the 63% who enroll, a completion rate of 48% (.30/.63). This completion rate is higher among higher-scoring Adams eligible students, at 63% (.52/.82), but lower among those who use the Adams Scholarship, at 41% (.41/1.00). This suggests that Massachusetts' public colleges have low four-year completion rates relative to other colleges available to Massachusetts' 7 students. over 60%. Panel B shows these completion rates for the full set of college enrollers, including those who we do not observe graduating within seven years. Fewer than 20% of students who enroll at Massachusetts' public four-year colleges graduate within four years, relative to 30% who enroll at alternative colleges. In all sectors, over 50% of students fail to complete their degrees within seven years. This evidence makes clear that, in terms of degree completion time, Massachusetts' public four-year colleges are clearly inferior to the alternative colleges available to Massachusetts students. and state colleges have four-year graduation rates of 36% and 25% respectively, far lower than the 53% rate for in-state private and out-of-state colleges. The SAT scores in each sector suggest that the academic skill of incoming students may explain part of the difference in completion rates for state colleges but can not explain it for the U. Mass. campuses, which look quite similar to in-state private and out-of-state colleges. The latter are more selective, based on admission rates, so they may be screening students along other dimensions of academic skill. The in-state private and out-of-state colleges also have better student-faculty ratios (13.6 and 15.1) than U. Mass. campuses and state colleges (17.6 and 17.9). Bound et al. (2010) argue that the student-faculty ratio may be the best simple measure of resources available to college students because, unlike expenditure figures, it is not inflated by rising prices of productive factors such as labor. By this measure, Massachusetts public four-year colleges are providing about 20% fewer resources to students than 8 available private and out-state alternatives. That this resource gap may reduce students' access to coursework or to academic support necessary to complete such coursework may help explain the low completion rates at Massachusetts' public colleges.
No one of these variables perfectly captures the single dimension referred to as college quality. Black and Smith (2006) argue that, because each such variable measures college quality with error, relationships between them and outcomes of interest will be biased toward zero. We adopt their suggestion to measure college quality by combining information from multiple variables, in order to reduce such measurement error. Specifically, we construct "college quality" from our studentlevel data as the first component from a principal component analysis of each college's SAT math 75th percentile, admissions rate and student-faculty ratio. We think that the first two capture some element of student quality while the last is a proxy for available resources. We do not include four-year graduation rates in this measure because this is the outcome we are ultimately trying to explain. 11 We then standardize this quality measure to have mean zero and standard deviation one. The final row of panel A shows that U. Mass. campuses and state colleges are 0.34 and 0.88 standard deviations lower than the average quality college attended by Massachusetts high school graduates, while in-state private and out-of-state colleges are 0.24 and 0.15 standard deviations higher in quality. These figures accord with the measures underlying the quality measure itself.
Panel B may explain why students would choose to attend one of the U. Mass. campuses or state colleges given these large quality differences. In the fall of 2004, the total annual costs of enrolling in a U. Mass. campus and state college, including room, board and books, were roughly $18,000 and $14,000 respectively. Grant aid brought those net prices to under $11,000 and $8,000.
In contrast, the total costs of in-state private and out-of-state colleges were $38,000 and $35,000, with net prices of $23,000 and $21,000. For students enrolling in four-year colleges, Massachusetts public institutions were less than half as expensive as the alternatives and thus are quite attractive to students, particularly those facing financial constraints. 12 11 Black and Smith construct their quality measure using nearly identical variables but also include first-year retention rates. We think this is too close the graduation rate outcome of interest here and thus omit it. This makes little difference to the quality measure given the high correlations between retention rates and the variables we do include in the quality measure.
12 In-state community colleges, at which the scholarship could also be used, are essentially open admissions campuses. In fall 2004, they charged on average $831 in tuition, $2,073 in fees, and $5,797 in other expenses, so that their 9

Empirical Strategy
We now turn toward estimating the causal impact of the the Adams Scholarship on students' college outcomes. Comparing outcomes of those eligible and ineligible for the Adams Scholarship would confound the impact of the scholarship with the fact that eligible students have higher academic skill than ineligible ones. We eliminate this source of omitted variable bias by using a regression discontinuity design that compares students just above and below the eligibility thresholds.
Students just above and just below these thresholds are similar to each other except for receipt of the scholarship. Though the scholarship may incentivize students to raise their test scores and qualify for the aid, there is little scope for manipulation of test scores around eligibility thresholds for two reasons. First, exams are centrally scored and raw scores transformed into scaled scores via an algorithm unknown to students, their families or teachers. Second, at the time of test administration, the district-level 75th percentile threshold is impossible to gauge accurately. As a result, the assumptions underlying regression discontinuity design are exceedingly unlikely to be violated here.
The three eligibility conditions required by the Adams Scholarship complicate the implementation a regression discontinuity design. Only a few papers in the economics of education have dealt with such a situation (Papay et al., 2010(Papay et al., , 2011a. For simplicity, we use a binding-score regression discontinuity design in which we collapse the three dimensions of eligibility into a single measure that that perfectly determines treatment assignment (Reardon and Robinson, 2012).
To do so, we use DESE's data to determine each school district's 75th percentile cutoff. We then define each student's distance from the eligibility threshold as the minimum of the following three quantities: the difference between her total MCAS score and the district's 75th percentile cutoff, the difference between her lowest MCAS subject score from the proficient threshold at 240, and the difference between her highest MCAS subject score from the advanced threshold at 260.
In mathematical terms, we define the running variable GAP ijt for student i in district j and sticker and net prices were roughly two-thirds those of state colleges.
high school graduating class t as follows: where T OT ijt is student i's combined math and ELA MCAS scores, CU T jt is the district specific cutoff for the top 25% of MCAS scores, BEST ijt is the higher score of the student's two MCAS subjects, and W ORST ijt the lower score of the student's two MCAS subjects. 13 For example, a student that scores 254 in ELA, scores 272 in math, and resides in a district with a cutoff of 522 will have GAP ijt = min(526 − 522, 272 − 260, 254 − 240) = 4. In her case, the district cutoff is the forcing variable. In summary, GAP ijt assigns to each student her distance from the eligibility threshold that she was least above. This construction implies that any student with a non-negative value of GAP is eligible for the Adams Scholarship while any student with a negative value of GAP is ineligible. GAP can theoretically range in value from -160 for the lowest-scoring students to 20 for the highest-scoring students. We will thus focus on our analysis on students within 20, or more frequently 10, points of the threshold.
To estimate the causal effect of the Adams Scholarship, we use the now one-dimensional running variable to estimate linear probability models of the following form: where A is an indicator for Adams Scholarship eligibility (GAP ijt ≥ 0), δ t is a high school graduating class fixed effect, and X i is a vector of the demographic controls listed in panel A of Table   1. The causal effect of winning the Adams Scholarship on an outcome, Y ijt , is β 1 . 14 Assuming 13 Students can actually qualify for the top 25% criterion based on either 11th grade district of enrollment or 12th grade district of enrollment. Starting with the class of 2006, students at charter schools or who participate in school choice or Metco can also become eligible based on their 11th or 12th grade district of residence. We therefore use the lowest cutoff among each of these thresholds as CU Tjt.
14 We do not worry about manipulation of the running variable for a number of reasons. First, exams are centrally graded and neither students nor teachers know how the state will translate raw scores into the scaled scores relevant for the eligibility calculation. Second, it is impossible for students or teachers to know ahead of time the 75th percentile cutoff that also enters the eligibility calcluation. Third, as we show in Table A.1, there is little evidence of discontinuities in observable characteristics of students across the threshold. The only significant difference, in low income rates, is practically quite small. We control for such characteristics in our main specifications but show in Table 7 that such controls have no impact on our central results. that treatment effects are homogeneous along different parts of the eligibility threshold, this coefficient measures a local average treatment effect for students near the threshold, weighted by the probability of a given student being near the threshold itself. Our baseline regression specification limits the sample to students students within 10 scaled MCAS points of the eligibility threshold (i.e. |GAP | ≤ 10). We later show that our results are robust to choice of bandwidth and demographic controls.
As a preview of our regression results, we plot in Figure 5 four-year college outcomes by distance to the eligibility threshold as measured by GAP . We include the high school classes of 2005-2007 for whom we can observe college graduation within four years. Panel A plots the fraction of students enrolling in four-year college in the fall following their high school graduation.
A clear discontinuity at the eligibility threshold in the bottom series suggests the Adams Scholarship incentivized a substantial fraction of students to enroll in in-state four-year public colleges.
A similar though somewhat smaller discontinuity in the middle series suggests that the scholarship succeeded in keeping students in-state for initial college enrollment. The lack of a clearly visible discontinuity in the top series suggests that the scholarship did little to increase overall enrollment, implying that students choosing Massachusetts public colleges would simply have attended another college in the absence of the scholarship. Panel B, which plots the fraction of students graduating within four years from four-year institutions, shows much smaller discontinuities in the bottom two series, suggesting that relatively little of the induced college enrollment in-state at public institutions translated into on-time completion of degrees. Most striking is that the top series appears to contain a discontinuity in the opposite direction, suggesting that students barely eligible for the Adams Scholarship were less likely to graduate any college within four years than those barely ineligible. We now turn to regression analysis to measure these findings more rigorously.

Enrollment and Completion
Panel A of Table 3 shows the coefficient on the Adams eligible indicator, β 1 , for enrollment by the fall following high school graduation. This panel includes high school classes from 2005-2007, for whom we also observe graduation outcomes. 15 All outcomes are restricted to four-year colleges, exception of the two-year college outcome in column (8). The scholarship induced 8.3% of students near the threshold to attend in-state four-year public institutions. The vast majority of these students enrolled at a U. Mass. campus, with the remaining students enrolling in less selective four-year state colleges. 16 Columns (4) and (5) reveal that 2.2% (one fourth) of these students would have attended in-state four-year private colleges while 5.1% (more than one half) would have attended out-of-state colleges. Columns (6) and (7) reveal that the scholarship thus raised in-state enrollment by 6.1 percentage points but had no statistically significant impact on overall enrollment rates at four-year colleges. 17 To summarize, receipt of the Adams Scholarship induces a substantial number of students to choose in-state public four-year colleges over private and out-of-state alternatives, and attracts a small number of students into the two-year sector who would not otherwise have enrolled in college at all. 18 Overall, the scholarship does keep students in state who would otherwise have left, but also shifts some students from in-state private colleges.
There is no impact on four-year college enrollment. These effects are in line with those found in Goodman (2008), which used a coarser outcome measure to explore these questions.
We now turn to the impact of the scholarship on four-year college graduation rates in panel B. Each regression in this panel uses as an outcome an indicator for graduation from that college sector within four years of high school graduation. Column (1) is striking. Of the 8.3% of students induced by the scholarship to enroll at in-state four-year public colleges, only 2.2% graduate within four years, a ratio of 27%. Within the U. Mass. sector that ratio is an even lower, with only 1.2% of students graduating even though 6.0% were induced to enroll. This ratio of 22% is less than half that of the average U. Mass. student at the eligibility threshold, who has a 47% (0.069/0.146) chance of graduating within four years. This suggests that the marginal student induced to switch to a U. Mass. campus by the Adams Scholarship is much less likely to graduate on time than the average student with similar test scores who chooses a U. Mass. campus.
One possible explanation for the low completion rate from Massachusetts public colleges is that students may transfer to other institutions. Panel C explores this by defining as outcomes indicators for enrolling in a given college sector but failing to graduate from any college within four years. In all columns the coefficient in panel C is almost exactly the difference between the coefficients in panels A and B, suggesting that transfers to other college sectors are not an important part of this story.
The fact that U. Mass. campuses and other in-state public colleges have such low on-time completion rates implies two further facts about the policy's effects. First, as seen in column (6), though the scholarship successfully increased the fraction of students enrolling in-state for college by a substantial 6.1 percentage points, it increased the fraction of students completing college instate and within four years by only 1.1 percentage points. Raising local college completion rates is more difficult than raising local college enrollment rates. An evaluation of this program based on short-run effects would judge it much more favorably than a longer-run evaluation, pointing the importance of follow-up studies that look beyond immediate effects of such programs.
The second and more striking fact, seen in column (7), is that scholarship eligibility lowered the fraction of students completing four-year colleges on time by 2.2 percentage points. Given that this regression discontinuity estimate compares students of similar academic skill, we can rule out differences in such skill as an explanation for differences in completion rates. The only explanation must be that the choice of college itself affected completion rates. In fact, this estimated negative effect likely understates the true impact of college choice, given that the value of the Adams Scholarship itself should have mitigated financial constraints that may be responsible for some fraction of the failure to graduate on time. The offer of aid should have left students strictly better, or at least no worse, off. Instead, by inducing them to switch into lower quality Massachusetts public colleges, the scholarship hurt their longer-run outcomes.

Quality and Cost
The most plausible explanation for the negative impacts on on-time graduation is that the scholarship induced students to trade off college quality for a relatively small tuition subsidy. Table   4 explores the precise quality and cost tradeoffs that the Adams Scholarship induces. In the first three columns, we categorize colleges by the competitiveness category Barron's assigned them to in 2009. None of Massachusetts' public colleges fall into Barron's three highest categories of "most competitive," "highly competitive," and "very competitive," which includes private institutions such as Boston University, Tufts University, Simmons College, and Lesley University. All of the U.
Mass. campuses and nearly all of the state colleges fall into the fourth category of "competitive." Private universities in this category are Suffolk University and Wentworth Institute of Technology. The fifth category of "not competitive" includes a couple of state colleges and all community colleges. Columns (1)-(3) reveal that scholarship eligibility induced 2.9% of students to forego colleges in the highest two categories of most and highly competitive. A further 1.3% chose to forego very competitive colleges as a result of the scholarship. These result was to increase the fraction of students choosing competitive colleges, the second lowest Barron's category and the one containing nearly all Massachusetts public four-year colleges. Students did not simply switch into the public sector from private or out-of-state colleges of similar quality. More than half of the students induced to switch colleges would have enrolled in more selective alternatives in the absence of the scholarship.
Other measures of college quality point to a similar pattern. For students enrolling in four-year colleges, scholarship eligibility lowered the four-year graduation rate of their chosen colleges by 2.6 percentage points, an unsurprising finding given the previously documented differences in graduation rates between Massachusetts public colleges and other sectors. Columns (5)-(7) similarly show that switching into Massachusetts' public colleges lowered college quality as measured 15 by SAT scores, selectivity and student-faculty ratio. The result in column (8), combining all three measures as previously described, is that scholarship eligibility lowered college quality by 0.09 standard deviations. Table 4 assigns to each student the cost and financial aid for the average student at a given college as computed by IPEDS, again limiting the sample to those enrolled at four-year colleges. 19 The Adams Scholarship induces students to attend four-year colleges with $2,200 lower tuition and fees, $1,100 less in grant aid, but identical loan availability. These figures suggest that those eligible for the scholarship have annual net costs roughly $1,300 lower than the ineligible. This is not unexpected given how much less expensive Massachusetts' public four-year colleges are than the available alternatives, as we saw in Table 2.

Panel B of
Finally, we note that column (7) of panel B shows eligibility implied a $479 discontinuity in the average annual value of the tuition waiver. This discontinuity combines the discontinuity in enrollment for marginal students with the discontinuity in aid eligiblity for infra-marginal students.
In column (8) we estimate the discontinuity just for marginal students by assigning hypothetical Adams aid even to students below the eligibility threshold. This suggests that one-fourth of the scholarship funds, or $123 out of $479, flowed to marginal students. Combined with column (8) from panel A, this implies a willingness to forego 0.75 (0.092/123) standard deviations of college quality per $1,000 in annual aid, a remarkably high number. Students are surprisingly willing to forego college quality for relatively small amounts of financial aid.

The Impact of College Quality on Completion
The previous analysis has convincingly shown that scholarship eligibility both induced students to forego college quality and lowered their on-time college completion rates. In Table 5, we directly estimate the impact of college quality on students' on-time completion rates, using scholarship eligibility as an instrument for different measures of college quality in each column. Formally, we 19 In-state and out-of-state costs and aid are chosen appropriately when possible.
16 are estimating a system of equations of the following form: (3) where A is an indicator for Adams Scholarship eligibility (GAP ijt ≥ 0) as before, but here we use it as a instrument, λ t and κ t are high school graduating class fixed effects, and X i is a vector of the demographic controls listed in panel A of Table 1. In Equation 3, the first stage, we are estimating a measure of school quality, Q ijt , such as enrollment at an in-state public university or the student-faculty ratio. In Equation 4, the second stage, we use on-time graduation from a four- year institution as the outcome, Y ijt . The causal effect of interest, γ 1 , is interpreted as the effect on probability of graduation within four years for the marginal student induced to an institution with quality Q ijt . The first row of Table 5 replicates estimates seen in previous tables of the impact of scholarship eligibility on the given measure of college quality, which are the first stage estimates.
The second row replicates the impact of scholarship eligibility on the probability of four-year graduation from panel B of Table 3 Table 3, 30% (0.102+0.194) of students just below the scholarship cutoff graduate on time from private and out-of-state colleges, compared to the 46.0% (0.163+0.297) who enroll. If the marginal students would also have completed at this 64% rate if they had attended such colleges, then a 26 percentage point drop is equivalent to a 40% decrease in completion rates. More likely, these marginal students would have had lower completion rates even had they attended such colleges, so that the percentage impact is likely even larger. umn (3) implies that, for these marginal students, attending a college with a four-year graduation rate one percentage point higher would translate into a 1.3 percentage point increase in the probability of graduating on time. Differences in college-level graduation rates translate slightly more than one-for-one into individual-level graduation rates for this subset of students.
Columns (4)-(6) yield similar results with different scales. Attending a college with one point higher SAT scores raises the probability of graduating on time by slightly less than one percentage point. Attending a college with a one percent lower admissions rate raises the probability of graduating on time by more than two percentage points. Attending a college with one less student per faculty member raises the probability of graduating on time by 11 percentage points. Or, combining these three measures, attending a college of one standard deviation higher quality raises the probability of graduating on time by 37 percentage points. This is roughly three times larger than the effect estimated in Long (2008) by OLS and by instrumenting college quality by the average quality of nearby colleges. 21 All of these IV estimates are substantially larger than their OLS counterparts, suggesting either that omitted variable bias is driving the latter toward zero or that the marginal student induced to switch college due to scholarship eligibility is more sensitive to college quality than the average student. We should also note that these estimates actually confound a change in college quality with the availability of additional financial aid. Given that the latter should make students more likely to complete college on time, we believe these coefficients are actually slight underestimates of the impact of a change only in college quality.
In Table 6, we take advantage of a unique feature of this policy design to provide further evidence that a reduction in college quality is responsible for students' decreased four-year completion rates. In typical applications, regression discontinuity estimates measure local average treatment for effects for students near the treatment threshold. In particular, this treatment threshold is generally a fixed number along a single dimension, such as age 65 for Medicare eligibility.
Even in recent applications with multiple thresholds, the authors can at best measure LATEs along two different dimensions, such English and math thresholds. Our setting is the first regression dis-21 See the first row of Table 6 in that paper. continuity design of which we are aware that allows for measurement of multiple LATEs along a single dimension. Because part of scholarship eligibility is determined by school district-level 75th percentile cutoffs, the total MCAS score determining one component of the eligibility threshold varies by district. In other words, students at the threshold in low-scoring districts have lower test scores than students at the threshold in high-scoring districts. The result is that, unlike any regression discontinuity paper we have previously seen, we can measure treatment effects at multiple points in the distribution of overall academic skill.
To do so, we split the sample in the following way. For each graduating high school class, we divide students into two groups by the median 11th grade district-level cutoff score. Table 6 shows the results of this analysis. 22 We label students in the lower half of districts as "low skill" and students in the top half as "'high skill." In panel A, the first few columns show that both sets of students react strongly to the scholarship by increasing enrollment at in-state public colleges.
Low-skilled students' enrollment in U. Mass. campuses reacts twice as strongly as that of highskilled students. This is partly due to the fact that the scholarship raises the four-year college enrollment rate by 3.3 percentage points for low-skilled students, though it has no such effect on high-skilled students.
Columns (6)-(8) make clear that only the high-skilled students are foregoing higher quality colleges. As a result of scholarship eligibility, high-skilled students are 3.4 percentage points less likely to enroll in most or highly competitive colleges, attend college with graduation rates 2.3 percentage points lower, and forego 0.1 standard deviations of college quality. Low-skilled students do not forego any college quality as a result of the scholarship. This parallels closely the estimates in column (5) of panel B, which show that scholarship eligibility decreases the on-time graduation rates only of high-skilled students and not of low-skilled students. The overall decrease in on-time graduation previously estimated is driven entirely by high-skilled students.
This heterogeneity analysis provides further evidence that college quality is the key channel through which graduation rates are affected. High-skilled students who would otherwise have 22 Because of the extraordinarily high correlation between district-level income and test scores, the results that follow are nearly unchanged if we instead divide students by district-level poverty rates. For this reason, we omit separate analysis by income.
gone to higher quality colleges are induced to enroll in the lower quality U. Mass. campuses and state colleges. The result is no impact on overall enrollment rates but a decrease in on-time graduation rates due to the decrease in college quality. Low-skilled students would, in the absence of the scholarship, have enrolled in colleges of similar quality to Massachusetts public colleges or in no college at all. The scholarship does not induce a loss of college quality for them and thus has no impact on their on-time graduation rates.

Robustness
We explore the robustness of our main enrollment and completion results to alternative specifications in Table 7. Panel A is our preferred specification that we have been using throughout the paper, which employs a bandwidth of |GAP | ≤ 10, fits linear functions on either side of the threshold, and includes students' demographic controls. Panel B uses the same specification but excludes demographic controls. Panel C replicates panel A but reduces the bandwidth to |GAP | ≤ 6. Panel D replicates panel A but expands the bandwidth to |GAP | ≤ 20 and fits quadratic functions on either side of the threshold. Finally, to account for the few cases in which our assignment rule does not match DESE's assignment rule, Panel E replicates panel A with a fuzzy regression discontinuity in which we instrument official Adams eligibility with our designation of Adams eligibility. 23 Because our algorithm closely follows DESE's procedure, there is little difference between our designation of Adams eligibility and official Adams notification, with a first stage coefficient of 0.98 (t=303). Across the panels, all five models show nearly identical results, suggesting that our central estimates are robust to inclusion of demographic controls, choice of bandwidth, choice of functional form, and measurement error in the eligibility variable.
We also show, in table 8, that our results are not driven by one particular class of students. This is a concern given the possibility that the decrease in on-time completion rates is driven by the first eligible class' misunderstanding of the scholarship letter's promise of "four years of free tuition." If such students did not know the fee structure of the public colleges, they may have miscalculated the financial burden attendance would imply, thus interfering with their ability to complete college on time. If that were the case, we would expect enrollment and completion impacts to diminish over time as subsequent classes learned from prior classes' mistakes. Table 8, which estimates these enrollment and graduation impacts separately by high school class, suggests this is not the case. The estimated impacts of scholarship eligibility on enrollment and graduation are relatively similar across the first three classes. The negative impact of the scholarship on four-year graduation rates is actually highest for the class of 2007, suggesting that the above misinformation story is not driving our main results. Also interesting is the fact that the scholarship's impact on in-state public enrollment was notably smaller in 2008 and 2010 than in earlier years. This could relate to deteriorating macroeconomic conditions in those years or to the fact that the scholarship's value diminished over time relative to rising fees.
Finally, Table 9 shows that considering five-and six-year graduation rates leaves the main conclusions of this paper intact. Panel A shows the classes of 2005 and 2006, for whom we can observe four-and five-year graduation rates. Panel B shows just the class of 2005, for whom we can observe four-, five-and six-year graduation rates. Column (1) reveals that the impact of the scholarship is higher on five-year graduation rates than it is on four-year rates, suggesting that some fraction of marginal students do take a fifth year to graduate. There is little difference, however, between impacts on five-and six-year rates. Columns (5) and (6) show that estimated impacts of the scholarship on graduation rates are negative even when students are given an extra year or two to complete their degrees. Overall, these figures suggest that under one-third of the negative impact on four-year graduation rates is due to delayed graduation, while the remaining two-thirds is due to failure to graduate at all (at least within the six year window we observe).

Implications for Massachusetts
A cost-benefit analysis of the Adams Scholarship policy is complicated by the fact that the costs are never listed in budget appropriations. Because the scholarship comes in the form of a tuition waiver, the costs of the program are determined by the revenue foregone by the state of Massachusetts when such waivers are granted to enrolled students. We have found no prior calculation of these costs, but can compute them with our data by multiplying the number of scholarship eligible students at each campus with the tuition at that campus, as in Table A.3. We limit the calculations to students within their first four years of college, given the scholarship's restrictions.
Our calculations suggest that, once the scholarship was financing a full four classes of students a year, the annual cost to the state was on the order of $25 million. To date, the state has spent upwards of $100 million on the program. 24 These expenditures have led to an eight percentage point increases in enrollment in Massachusetts' four-year public colleges, as hoped for by Governor Romney and other proponents of the policy. Such enrollment increases have not, however, translated into substantial increases in in-state production of college degrees. Our estimates suggest at most a two percentage point increase in in-state graduation rates, suggesting that three quarters of the induced enrollment did not translate into in-state completion. This gap is due in small part to the scholarship's shifting of students from in-state private colleges but in larger part to the fact that students induced into Massachusetts public colleges do not finish college at all.
If we assume that two percentage point estimate applies to the entire distribution of student eligible for the scholarship, or about 15,000 students per year, then the scholarship is producing roughly 300 more four-year college graduates in Massachusetts per year for $25,000,000. 25 This translates to a cost per additional in-state graduate of over $80,000. One reason for this high number is that, according to our estimates in Table 3, about three-fourths of Adams Scholarship users are infra-marginal, meaning that they would have attended in-state public colleges even without the scholarship. Similarly, according to Table 4, three-fourths of the scholarship funds flow to students whose college choices are unaffected by those funds. This calculation also likely underestimates the true cost per in-state graduate given that some fraction of these students will leave Massachusetts soon after graduating from college. A full cost-benefit analysis would also account for the two percent of, or roughly 300, students who failed to complete a college degree as a result of the scholarship. All in all, these considerations suggest the state is spending large amounts of money for relatively little net benefit or even net harm to its students.

Conclusion
We draw three broad conclusions from our findings. First, our estimates suggest that students have a poor understanding of the importance of college quality, particularly as measured by can be worth about $7,000 for students attending the most expensive U. Mass. campuses for a full four years. It is possible that some students were so myopic or financially constrained that switching into scholarship eligible institutions was a rational decision. More likely, the marginal student did not understand that foregoing college quality would lower her own chance of earning a college degree.
Second, this poor understanding of the importance of college quality and on-time graduation rates suggests a possible scope for policy interventions to make certain information about colleges both more readily available and more salient. Students, parents and high schools should take more advantage of tools like the U.S. Department of Education's "College Navigator", which allows students to compare characteristics such as completion rates across institutions. Simultaneously, such tools, which often present users with hundreds of statistics about a given college, should privilege variables that users are known to underweight. On-time graduation rates are one such variable.
Third and finally, the harm done to students by exposure to colleges with relatively little funding and low graduation rates highlights the critical importance of improving such institutions.
Governor Romney attempted to improve Massachusetts' public colleges by changing the composition of the student body. The evidence here suggests that approach did not succeed. Students exposed to those colleges because of the Adams Scholarship had worse long-run outcomes than those not exposed. Whether college quality operates through access to coursework, campus resources, peer effects, or other channels is beyond the scope of this paper. These results do, however, suggest that improving college quality requires steps beyond merely changing the composition of the student body. Deeper exploration of the factors preventing on-time completion is needed, particularly for postsecondary systems with such obvious challenges.   (2) and (3) split the sample by Adams Scholarship eligibility. Column (4) restricts the sample to those who use the Adams Scholarship. Column (5) restricts the sample to students close enough to the threshold to participate in regresssion discontinuity analysis.   Notes: Heteroskedasticity robust standard errors clustered by 10th grade school district are in parentheses (* p<.10 ** p<.05 *** p<.01). All models include student demographic characteristic controls as listed in Panel (A) of Table 3. The sample consists of the high school classes of 2005-2007. In columns (1)- (7), outcomes are defined as one only for four-year colleges. Panel A defines on-time enrollment as enrollment by the fall following high school graduation.
Panel B defines on-time graduation as college graduation within four years of high school graduation. Panel C defines enrollment without graduation as on-time enrollment in a given sector but failure to graduate from any four-year college within four years. Listed below each coefficient is the mean of the outcome for students just below the Adams cutoff.   Notes: Heteroskedasticity robust standard errors clustered by 10th grade school district are in parentheses (* p<.10 ** p<.05 *** p<.01). Each column presents an instrumental variables regression of graduation within four years on the endogenous regressor listed at the top of each column, where the endogenous regressor has been instrumented with Adams eligibility. The first three rows show the first stage, reduced form and instrumental variables estimates respectively. The final row shows the OLS estimate of the same regressions without using the instrument. Notes: Heteroskedasticity robust standard errors clustered by 10th grade school district are in parentheses (* p<.10 ** p<.05 *** p<.01). High-and lowskilled students are those whose 11th grade district cutoff scores are respectively above and below the median such score in that high school class. Sample includes the classes of 2005-2007.In columns (1)-(6), outcomes are defined as one only for four-year colleges. In columns (7) and (8), the outcome is defined only for four-year colleges.  Notes: Heteroskedasticity robust standard errors clustered by 10th grade school district are in parentheses (* p<.10 ** p<.05 *** p<.01). All models include student demographic characteristic controls as listed in Panel (A) of Table 3. In columns (1)- (3) and (6)- (8), outcomes are defined as one only for four-year colleges. In columns (4) and (5), the outcome is defined only for four-year colleges. In columns (6)- (8), graduation is defined as within four years. College quality is the standardized first principal component of the student-faculty ratio, 75th percentile SAT math score, and percent admitted. Listed below each coefficient is the mean of the outcome for students just below the Adams cutoff. Dear [Name to be mail-merged]: Congratulations! You are one of the first recipients of the John and Abigail Adams Scholarship. The Adams scholarship is good for four years of free tuition at any University of Massachusetts campus, or any state or community college. Your outstanding MCAS results automatically qualify you to receive this award.
We created this merit scholarship program to reward your hard work and achievement, and to encourage you to go to college at one of our top-notch public higher education institutions.
With the support of the Board of Higher Education, the Class of 2005 is now the first to be awarded this opportunity. It is the strongest expression we can make of our commitment to attracting students like youthe best and brightest in the state-to our Commonwealth's public higher education system.
I encourage you to apply to any of the campuses on the attached list. Congratulations again, and best wishes for your continued success. Dear <Student Name>:

Congratulations!
You have qualified to receive a John and Abigail Adams Scholarship, which entitles you to four years of free tuition upon your acceptance to a participating Massachusetts public institution of higher education, including a University of Massachusetts campus, a Massachusetts state university, or a community college. Now in its eighth year, the Adams Scholarship rewards high academic achievement on MCAS tests, and provides families of college-bound students with financial assistance. Please note that the Adams Scholarship covers tuition only, and does not include college fees.
Please review the enclosed guidelines carefully to determine whether you meet the eligibility requirements. If you do, I encourage you to apply to one of the campuses on the attached list.
It is extremely important that you make a copy of this letter and keep the letter and copy in a safe place. In order to receive the scholarship, you must submit this letter to the financial aid office of the Massachusetts public college or university to which you have been accepted and complete the online Free Application for Federal Student Aid (FASFA).
Congratulations again, and best wishes for your continued success in college and beyond.