Arnold Ventures’ (AV) Evidence-Based Policy team invites grant applications to conduct randomized controlled trials (RCTs) of social programs in any area of U.S. policy in which: (i) AV will fund the RCT, and government or another entity will fund the program’s delivery; and (ii) The RCT meets the additional selection criteria set out below.
Our main goal in funding such RCTs is to build the body of programs rigorously shown to produce sizable, sustained effects on important life outcomes. We recognize how challenging this is: Surprisingly few programs are found to produce the hoped-for improvements in participants’ lives when evaluated in a well-conducted RCT. This Request for Proposals seeks to optimize the chances of success by prioritizing RCTs of programs with highly-promising prior evidence or other compelling policy reasons for a rigorous evaluation.
Deadline: Rolling Basis
Dollar Amount: Varies
Eligibility:
We ask applicants to address the following four criteria in both the letter of interest and the full proposal. The full proposal should provide more detail (e.g., on the study design) than the letter of interest, and also address any questions or issues identified by AV in its invitation to submit a full proposal.
- PROGRAM FUNDER: Will the proposed RCT evaluate a program whose delivery is paid for by another funder, and does that funder, or do other essential parties, agree to the study? To verify such agreement(s), the reviewers will look for attached letters or other communications showing that the necessary parties (e.g., program funder and/or program provider) assent to the study, including random assignment. Such agreement(s) may be tentative at the time the letter of interest is submitted, but should be finalized before submission of the full proposal. We especially encourage agreements in which the necessary parties not only assent to the study, but also provide a credible description of how they or others would use the study findings to inform program or policy decisions.
- IMPORTANCE: Is the applicant proposing to evaluate a program –
- That is backed by highly-promising prior evidence, suggesting it could produce sizable impacts on outcomes of recognized policy importance – such as educational achievement, workforce earnings, criminal arrests, hospitalizations, child maltreatment, and government spending. For example, we specifically encourage applications seeking to replicate findings from prior rigorous evaluations that are especially promising but not yet conclusive—e.g., due to only short-term follow-up, a single-site study design, or well-matched comparison groups but not randomization. (Please provide full citations to the relevant prior studies as an attachment to the letter of interest.) As a threshold condition for “highly promising” evidence, applicants should show that the program can be or (preferably) has been successfully delivered under real-world implementation conditions, since effective implementation is usually necessary for a program to produce meaningful impacts - or - For which there are other compelling policy reasons to evaluate its effectiveness – e.g., it is, or soon will be, widely implemented with significant taxpayer investment, and its impact on its targeted outcomes is currently unknown.
- Please note that, to meet this criterion, it is not sufficient to establish that the program seeks to address an important problem, or that the study will fill a gap in the research or test a theory. Applicants must also present a compelling policy reason, as described above, to evaluate the specific program.
- STUDY DESIGN: Is the applicant’s proposed RCT design valid? In other words, does it have a sufficiently large sample (as shown through a power analysis) and other elements needed to generate credible evidence about the program’s impact on one or more targeted outcomes of high policy importance? We strongly encourage designs that measure such outcomes in both the short and longer term, as appropriate for the type of program and study, to determine whether the effects endure long enough to constitute meaningful improvement in people’s lives. Reviewers, in assessing an applicant’s proposed design, will use Key Items to Get Right When Conducting RCTs of Social Programs as a reference.
- Applicants, in discussing this criterion, should specify the study’s primary outcome(s) of interest; how they will measure the outcome(s) and over what length of time; and what analyses they plan to conduct (e.g., any subgroups to be examined, regression methods to be used).
- EXPERIENCED RESEARCHER: Does the applicant’s team include at least one researcher in a key substantive role who has led or played a key role in a prior well-conducted RCT? A well conducted RCT is characterized, for example, by low sample attrition, sufficient sample size, close adherence to random assignment, and valid outcome measures and statistical analyses. To address this criterion, we request that applicants submit at least one, and not more than two, reports from such prior RCTs. (Please send the full study reports as email attachments to the letter of interest.) Reviewers will rely primarily on these reports in assessing this selection criterion.
- We recognize the need to expand and diversify the pool of researchers with RCT experience. Thus we strongly encourage researchers who are new to RCTs, including those from groups historically underrepresented in the research community – such as researchers of color and women – to participate in this funding opportunity. We therefore want to clarify that such individuals who do 3 not meet the “experienced researcher” criterion themselves may still serve as a study’s lead researcher as long as they partner with a colleague who does meet the criterion and will play a key substantive role in the study.