Jul 30, 2019

ORS Impact; Fund for Shared Insight;

Past evaluations of Shared Insight's Listen4Good grantees have reported that through L4G they increased their technical ability to perform high-quality feedback loops, gained insights that informed data-driven changes to programming and internal operations, and improved both their programs' effectiveness and their overall ability to serve clients. Evaluations have also found that L4G helped foster a culture of openness and listening in participating organizations and advanced their equity, diversity, and inclusion efforts. Until now, however, these studies have examined L4G participants only during their grant periods. In this report, ORS Impact takes on a central learning question for the sustainability of Shared Insight's feedback work: To what extent do grantees continue collecting high-quality perceptual feedback from clients after the L4G grant ends? Here, ORS looks at 46 nonprofits that were the first to receive L4G grants in 2016, conducting interviews with 35 of them about a year after their grant ended.

Jul 30, 2019

W.K. Kellogg Foundation;

This report shows how equity-based family engagement helps parents and caretakers in underserved communities become effective advocates and culture-bearers in schools, which boosts educational quality and relevance.

Jul 29, 2019

MDRC; Bill & Melinda Gates Foundation; Community College Research Center (CCRC) at Teachers College, Columbia University;

Integrated Planning and Advising for Student Success (iPASS) is an initiative funded by the Bill & Melinda Gates Foundation to support colleges that seek to incorporate technology into their advising and student services. In iPASS, such technology is intended to increase advising's emphasis on a student's entire college experience, enabling advisers to more easily (1) intervene when students show early warning signs of academic and nonacademic challenges, (2) regularly follow up as students progress through college, (3) refer students to tutoring and other support services when needed, and (4) provide personalized guidance that reflects students' unique needs. To study how technology can support advising redesign, MDRC and the Community College Research Center partnered with three institutions already implementing iPASS: California State University, Fresno; Montgomery County Community College; and the University of North Carolina at Charlotte. The three institutions increased the emphasis on providing timely support, boosted their use of advising technologies, and used administrative and communication strategies to increase student contact with advisers. The enhancements at all three institutions are being evaluated using a randomized controlled trial research design. This report shows that the enhancements generally produced only a modestly different experience for students in the program group compared with students in the control group, although at one college, the enhancements did substantially increase the number of students who had contact with an adviser. Consequently, it is not surprising that the enhancements have so far had no discernible positive effects on students' academic performance. The findings also highlight the potential for unintended consequences. Before the study, each of the institutions had required that certain groups of students see an adviser before registering for classes in the next semester. Each institution expanded this preregistration requirement to include all students in the study's program groups, but at one institution, the requirement appears to have contributed to a small reduction in earned credits.

Jul 26, 2019

Oxfam GB;

Oxfam GB's Global Performance Framework is part of the organization's effort to better understand and communicate its effectiveness, as well as to enhance learning for staff and partners. Under this Framework, a small number of completed or mature projects are selected at random each year for an evaluation of their impact; this exercise is known as an 'Effectiveness Review'. One key focus is on the extent to which the projects have promoted change in relation to relevant Oxfam GB global outcome indicators. The global outcome indicator for the livelihoods thematic area is defined as 'total household consumption per adult equivalent per day'. This indicator is explained in more detail in section 5 of this report. Niger's 'Community-Based Integrated Water Resource Management' project was one of those selected for an Effectiveness Review in the 2016/17 financial year. The project activities were implemented by Oxfam GB in conjunction with the partner organization Karkara and the Department of Agriculture of the Republic of Niger. The project was started in April 2013 and was completed in March 2015. It was evaluated one year after closure.

Jul 12, 2019

John D. and Catherine T. MacArthur Foundation; EnCompass LLC;

Since 2015, the MacArthur Foundation's Big Bet On Nigeria is investing in efforts to reduce corruption in Nigeria by supporting Nigerian-led endeavors that strengthen transparency, accountability, and participation. Corruption, impunity, and lack of accountability in Nigeria have far-reaching impacts on access to and quality of public services, the well-being of Nigerians, and overall development. The On Nigeria strategy builds on Jonathan Fox's "sandwich" theory,1 which recognizes the importance of the combination of a push from below and a squeeze from above to effect change and sustain momentum. The push from below is the "voice"— representing citizens' actions to demand change and develop local solutions to corruption, while the squeeze from above is the "teeth"—representing the efforts of government and other high-level actors to develop and enforce laws and regulations, using incentives to discourage corruption and sanctions to punish it. The On Nigeria theory of change harnesses the "voice" of Nigerian citizens and the "teeth" of Nigerian public and private institutions, and combined with capacity building and collaboration, intends to address the problem of corruption in Nigeria. The On Nigeria evaluation and learning framework seeks to answer three overarching evaluation questions: (1) How is the MacArthur Foundation's strategy contributing to changing transparency and accountability of government and private-sector actors? (2) How is the MacArthur Foundation's strategy contributing to changing social norms and citizens' behaviors related to corruption? and (3) What kinds of adaptation or changes are needed in the theory of change and/or strategy to achieve better results? The framework is designed to provide specific information related to On Nigeria's landscape, outcomes, impacts, and feedback on the strategy to assess progress and adapt the strategy as needed.

• Clearly States Methodology

Jul 02, 2019

W.K. Kellogg Foundation; KKS Advisors;

The Mission Aligned Framework for Investing, introduced in this report, is a management and governance tool created by KKS Advisors on behalf of the Kellogg Foundation. It offers a set of determinants related to our programmatic strategy and desired outcomes. As we evaluate our investments, this analysis is contributing to our thinking and ongoing measurement of sustained impact. We offer this report as a resource for the field and a tool for other organizations exploring mechanisms to evaluate the social effect of investments in pursuit of deep impact.

Apr 02, 2019

Stars Foundation; With And For Girls;

Funding for adolescent girls has been gaining traction in recent years. While feminist funders have traditionally focused on women and young people, there has been a drive to put more flexible funding in the hands of girl-led and girl-centered organisations.  This evaluation reviews and assesses the With and For Girls Collective, the With and For Girls Award and the awards journey with a view to drawing out lessons from the Collective's experience to help encourage funders to increase flexible funding and other resources to girl-led and girl-centered organisations globally.

Mar 12, 2019

MDRC;

The study, funded by Robin Hood, is the most rigorous, independent, third-party evaluation of group microfinance in the United States, assessing Grameen America's program, a microfinance model that provides small loans to low-income women entrepreneurs in the United States seeking to launch or expand small businesses.

Mar 01, 2019

Calgary Foundation;

Calgary Foundation supports strategies that strengthen relationships between Indigenous and non-Indigenous people, that are built on mutual respect and traditional knowledge. In order to strategically grant for the most impact, it is necessary to study each of the priority areas to learn how to best support the work that needs to be done. This Impact Report provides insight as to how we might best affect change in this area.

Feb 01, 2019

Mathematica Policy Research, Inc.; John D. and Catherine T. MacArthur Foundation;

For more than 20 years, we have supported work to improve population and reproductive health in India. After making significant progress in this field, particularly in the areas of maternal health and rights, we are preparing to exit the population and reproductive health field in India and are supporting a concluding round of grantmaking focused on maternal health quality of care. Through this four-year strategy, we aim to advance maternal health by supporting a shift in the field's focus from access to quality of maternal health care. To accomplish this goal, the strategy backs three main areas of work or sub strategies: strengthening the supply of quality maternal health services, building the demand for quality services through accountability mechanisms, and building an evidence base and support for maternal health quality of care. The strategy officially launched in June 2015. Our evaluation partner, Mathematica Policy Research, documented early progress of the strategy through March 2017. Building on earlier evaluations of the strategy, this document provides findings from the midline evaluation covering April 2017 to March 2018.

Jan 31, 2019

NORC at the University of Chicago; John D. and Catherine T. MacArthur Foundation;

The International Connections Fund (ICF) was established in 2008 with the goal of helping Chicago nonprofit organizations advance their work by collaborating with peer organizations abroad. While eligibility criteria for ICF grants have shifted over the program's lifespan, this core mission has remained unchanged. During the life of the ICF program, MacArthur has administered 14 grant cycles, making 141 grants totaling more than $5.8 million. The majority of these grants—133 in all, totaling $5.4 million—have been awarded to support arts and culture projects. These projects have enabled Chicago artists, audiences, and arts and culture organizations to participate in international exchanges with counterparts from 63 different countries on six continents. Ten years into the program, MacArthur commissioned NORC at the University of Chicago to take stock of how the program has operated; learn what impact it has made on ICF grantees, their collaborators, and audiences; and consider how the program can best serve future grantees as ICF enters its second decade. The evaluation reviewed 12 ICF grant cycles that took place between 2008 and 2016, during which 114 grants were made to 91 different arts and culture organizations.

• Clearly States Methodology
• Provides access to data

Jan 09, 2019

Loyola University Chicago Center for Urban Research and Learning;

The Domestic Violence Outcome Project had a two-fold purpose: first, to identify the long-term outcomes and needs of those who receive services from domestic violence agencies, and second, to establish procedures for on-going evaluation within agencies. Working closely with 15 agencies that are members of the Chicago Metropolitan Battered Women's Network, the researchers developed a survey to evaluate services and identify client needs. The services evaluated included court advocacy (e.g., assistance from an advocate in obtaining an order of protection), legal services (assistance from a licensed attorney with divorce or other court proceedings), emergency shelter, and counseling. Development of the survey benefited greatly from extensive feedback from service providers and clients and from previous evaluation research. The Chicago Metropolitan Battered Women's Network and the participating agencies administered the survey, which had both an on-line and paper option. Agency staff recruited participants, maintained contact with them over about 6 months, and then had them complete the survey. Here we present findings from analysis of data provided by 450 participants. We also include a discussion of the challenges encountered in sustaining ongoing evaluation in agencies. One of the key findings of this report is that emergency safety needs (i.e., emergency shelter and getting an order of protection) are no longer the most prominent issues of concern for participants. Fewer than 5% of the sample reported currently needing shelter and fewer than 10% reported needing help getting an order of protection. In contrast, counseling/therapy is now the primary need reported by about 46% of participants. In addition, about a quarter of participants reported a need for help with those things that enable one to sustain a stable and independent household, which is critical to maintaining safety: economic assistance, either in the form of emergency cash, help with credit history, financial planning/literacy, food/clothing, health care, or work. Also, a sizeable minority of participants reported needs (both new and continuing from when they initially sought services) regarding divorce, child support, and visitation. These legal issues are likely to be related to the one outstanding safety concern reported by a substantial minority of survivors, managing contact with the abuser. Few differences among reported needs existed by race/ethnicity, parenting status, or level of socioeconomic resources. This report begins with a brief introduction to how the project came about and a description of our research methods. Next we present the current needs reported by participants and consider whether there are differences in needs among participants by race/ethnicity, education and income resources, and whether or not they have children. We then examine the relationship of past services to current needs and satisfaction with past services. After that, we consider outcomes of receiving services (e.g., "As a result of receiving services, I feel safe from violence in my home"). Finally, we describe difficulties encountered in sustaining ongoing evaluation in agencies, such as high staff 7 turnover rates and the need for a program coordinator to maintain staff motivation. We conclude with a summary of the findings.