This Request for Proposals—a joint effort of AV’s Higher Education and Evidence-Based Policy initiatives— seeks grant applications to conduct rigorous impact evaluations of programs and practices (“interventions”) to promote college success in the United States that fall into one of three tiers: (i) The intervention is backed by promising prior evidence suggesting it could produce sizable impacts on important student success outcomes (e.g., student learning, persistence, degree or certificate completion, job placement, post-college earnings, and debt burden); (ii) The intervention is widely adopted in practice, but has not yet been rigorously evaluated and its impacts on key student success outcomes are thus largely unknown; or (iii) The intervention is growing in use and likely to become widely adopted, but has not yet been rigorously evaluated.
Arnold Ventures (AV) is a nonpartisan philanthropy whose core mission is to invest in evidence-based solutions that maximize opportunity and minimize injustice. The Higher Education initiative seeks to identify and scale effective practices that improve student success and address equity gaps in higher education. Even as access to higher education has significantly expanded, we still struggle to help students complete their credentials and secure a strong return on their investments. Colleges need sound evidence to identify ways to support students’ financial, social, and academic needs. We support research to uncover the most effective programs and practices that will pave the way for success among all students, especially those underserved by the current system. In recent years, a number of highquality, randomized controlled trial (RCT) evaluations have identified several programs with the potential to substantially increase student success and address equity gaps. Arnold Ventures works to continue building the evidence base and to secure policies and investments to scale up what works.
Deadline: Rolling Basis
Eligibility: We ask applicants to address the following four criteria in both the letter of interest and the full proposal. The full proposal should provide more detail (e.g., on the study design) than the letter of interest, and address any questions or issues identified by AV in its invitation to submit a full proposal.
- PROMISING OR WIDELY-ADOPTED: Is the applicant proposing to evaluate an intervention that falls into one of the following three tiers?
- (i) The intervention is backed by promising prior evidence suggesting it could produce sizable impacts on student success outcomes of clear policy importance, including, but not limited to, student learning, persistence, degree completion, job placement, post-college earnings, and debt burden. For example, we specifically encourage applications seeking to replicate findings from prior impact evaluations that are especially promising but not yet conclusive due to study limitations—e.g., short follow-up period, single-site study design, or well matched comparison groups but not randomization. (Please provide full citations to the relevant prior studies as an attachment to the letter of interest.) As a threshold condition for “promising” evidence, applicants should show that the intervention can be or (preferably) has been successfully delivered under real-world implementation conditions.
- (ii) The intervention is widely adopted in practice with significant taxpayer investment, but has not yet been rigorously evaluated and its impacts on key student success outcomes are thus largely unknown.
- (iii) The intervention is growing in use and likely to become widely adopted with significant taxpayer investment, but has not yet been rigorously evaluated.
- Appendix A contains illustrative examples of four interventions that we believe meet the “promising” or “widely adopted” criteria and would be excellent candidates for funding.
- STUDY DESIGN: Is the applicant’s proposed research design valid? In other words, does the proposed study have a sufficiently large sample (as shown through a power analysis) and other elements needed to generate credible evidence about the intervention’s impact on one or more targeted outcomes of high policy importance as outlined above? We strongly encourage designs 3 that measure outcomes in both the short and longer term to determine whether the intervention produces effects on outcomes which constitute meaningful improvement in people’s lives such as degree or certificate completion and post-college earnings. Reviewers, in assessing an applicant’s proposed design, will use Key Items to Get Right When Conducting RCTs of Social Programs as a reference.
- Applicants, as part of their discussion of this criterion, should specify the study’s primary outcome(s) of interest; how they will measure the outcome(s) and over what length of time; and what analyses they plan to conduct (e.g., any subgroups to be examined, regression methods to be used).
- While we generally seek to fund RCT evaluations, we will also consider submissions for rigorous quasi-experimental evaluations if the applicant can make a convincing case that an RCT is not feasible, and—per the “Experienced Researcher” criterion below—can demonstrate their experience carrying out a well-conducted quasi-experimental evaluation by providing at least one, and not more than two, reports from prior comparable quasi-experiments. In such cases, we ask applicants—in addition to addressing the design elements above (e.g., sizable sample, important outcomes, extended follow-up)—to show that the proposed study adheres closely to the design features empirically shown to increase the likelihood of a valid quasi-experimental finding, as described in this document.
- EXPERIENCED RESEARCHER: Does the applicant’s team include at least one researcher in a key substantive role who has previously carried out a well-conducted RCT or quasi-experimental study (as applicable)?1 A well-conducted RCT is characterized, for example, by low sample attrition, sufficient sample size, close adherence to random assignment, and valid outcome measures and statistical analyses. To address this criterion, we request that applicants submit at least one or two, reports from prior rigorous impact evaluation studies of comparable design to the proposed study that the researcher played a substantive role in conducting. (Please send the full study report(s) as an email attachment to the letter of interest.) Reviewers will rely primarily on these reports in assessing this selection criterion, using Key Items to Get Right When Conducting RCTs of Social Programs as a reference.
- We recognize the need to expand and diversify the pool of researchers with RCT and quasi experimental design experience, and are committed to reducing barriers to achieving this goal. Thus we strongly encourage researchers who are new to rigorous impact evaluations, including those from groups historically underrepresented in the research community—such as researchers of color and women—to participate in this funding opportunity. We especially encourage researchers representing Minority Serving Institutions (MSIs), Historically Black Colleges and Universities (HBCUs), Hispanic-Serving Institutions (HSIs), and Tribal Colleges and Universities (TCUs) to participate. Such individuals who do not meet the “experienced researcher” criterion themselves may still serve as a study’s lead researcher as long as they partner with a colleague who does meet the criterion and will play at least a key advisory role in the study. (Prospective applicants are welcome to contact us for assistance in addressing this criterion; see contact information in section IV below.)
- FUNDING AND OTHER PARTNERS: Do funders of the intervention and any other essential parties agree to the study? To verify such agreement(s), the reviewers will look for attached letters or other communications showing that the necessary parties (e.g., funder and/or provider of the intervention) assent to the study, including random assignment. Such agreement(s) may be tentative at the time the letter of interest is submitted, but should be finalized before submission of the full proposal. We especially encourage agreements in which the necessary parties not only assent to the study, but also provide a credible description of how they or others would use the study findings to inform program or policy decisions.
- In the grants awarded under this Request for Proposals, we will generally fund the cost of the study, which may include research related costs to the implementing partner, but typically expect other parties to pay the cost of delivering the intervention to the treatment group. However, we may help support costs of intervention delivery in a limited number of grant awards where the case for an evaluation is particularly compelling—based, for example, on especially promising prior evidence. In these situations, AV funding support for intervention delivery would be limited to the amount needed to enable a sufficiently-sized evaluation to go forward. Applicants seeking such funding support should so indicate in their submission, and provide the compelling reason(s) for the request.
Full RFP: https://craftmediabucket.s3.amazonaws.com/uploads/Student-Success-RFP_Final.pdf