About Us
Child Abuse Training
International Activities
Rossi Award for Program Evaluation
UMD Capstone Courses
Mailing List
Contact Us

Student Reports

Spring 2014

Professors Douglas J. Besharov, Douglas Call, Christopher Mihm, and Kevin Jones

In Spring 2014, students worked with clients at Catholic Relief Services, CGIAR, Creative Associates International, Education for Employment, Freedom House, Future’s Group, International Association for Community Development, Poverty Action Lab, U.S. Agency for International Development, the U.S. State Department, and the World Bank. Students performed a wide variety of analyses, including implementation evaluation, policy analysis, performance measurement development, summative evaluation, and research synthesis.


The following are the final projects prepared by the students. They are listed in order of clients.

Note: The associated files are best viewed using Adobe Reader. To get the software, go to http://get.adobe.com/reader/

Federal Government

U.S. Agency for International Development

1. A Multi-Country Analysis of Bottlenecks and Opportunities for Scaling Legume Seed Systems. USAID’s Feed the Future initiative is aimed at improving food security for individuals in developing countries. The basis for ensuring better food security is increasing agricultural production in these countries. Therefore, USAID seeks to scale technologies and production systems in order to bring the best practices to these countries. One of the ways USAID seeks to do this is through the improvement of grain legume seed systems, more specifically focusing on beans. Most developing countries, specifically in Africa, have weak legume seed systems, and therefore USAID seeks to develop the systems for these vital crops by scaling these based on the best practices of efficient legume systems like that of the US. However, it is important to look at the practices of the US in detail and identify the key bottlenecks in the systems to provide USAID with realistic expectations for what an efficient seed system will be able to produce in an African country. Therefore, the purpose of this project is to not only examine a seed system in the United States (specifically Michigan) but to also look at other seed systems in Africa to better understand the infrastructure, governmental regulation and cultural norms of agriculture in developing countries. This project will examine seed systems in Kenya, Uganda and Ghana and identify the key actors in each stage of the system as well as the significant causes of bottlenecks in getting a variety of seed from the researchers to the farmers. This will then be applied to the seed system in Mozambique, and a recommendation will be made about how to best scale a system in Mozambique based on the lessons learned from the aforementioned case studies. The conclusion is that basing Mozambique’s legume seed system on the US is not wholly realistic. There are opportunities in both the formal and informal aspects of seed dissemination that can be utilized to develop a seed system that is feasible and more efficient given the existing capacities of Mozambique’s agricultural sector.

Paper Presentation

2. A Policy Analysis of Tobacco Control in Sub-Saharan Africa. Tobacco use is the leading preventable cause of death and disability worldwide. It is a rising problem for Sub-Saharan Africa because: 1) prevalence levels among minors is alarming; 2) the tobacco industry is aggressively pursuing a market in this region, specifically targeting youth; and 3) the public is poorly aware of the danger caused by tobacco use. Without policy intervention, Sub-Saharan Africa will become a future epicenter of tobacco use.

Six policy options are identified for this region based on the best practices of tobacco control and the Framework Convention of Tobacco Control by the World Health Organizations: 1) price and taxation; 2) smoke-free policies; 3) marketing bans; 4) packaging and label warning; 5) public awareness campaign; and 6) regional cooperation against illicit trade. The options are identified from a business perspective aiming to reduce both the supply and demand of tobacco products in Sub-Saharan Africa. Three general recommendations are made. First, price methods are the most effective methods in reducing consumption of tobacco products, and smuggling should be curbed in order to secure the effectiveness of utilizing the price mechanism. Second, a public awareness campaign should be launched both in order to educate the danger of tobacco use and to enhance the political foundation by gaining broad public support. Finally, a multifaceted approach involving the civil society, research institutions, governments, media, and external donors should be taken.
Paper Presentation

3. Feed the Future: Natural Resource Management and Global Climate Change Indicator Review. Feed the Future (FTF) is the United States Government’s five-year-old global hunger and food security initiative. FTF is an initiative that aims to end global hunger and poverty by increasing agricultural productivity in rural and underdeveloped areas across the world. This project seeks to examine the current suite of Feed the Future’s Global Climate Change and Natural Resource Management indicators as well as indicators for similar organizations and suggests alterations to the current indicator set with the hope of providing a more robust and useful suite of indicators. The findings suggest that of the ten indicators Feed the Future uses for monitoring progress on Global Climate Change and Natural Resource Management, four should be eliminated, six should be altered, and four new indicators should be added.

Paper Presentation

4. On Measuring Democracy and Governance: Needs Assessment of Democracy and Governance Indices to Inform Asia Trend Analysis Program. The objective of this project is to inform the Democracy, Governance, Peace and Security (DGPS) team of the USAID Asia Bureau on existing democracy and governance trend analyses in order for the team to develop an Asia Barometer for operational and strategic decision-making. Benchmark indices are analyzed based on two characteristics: the type of source data utilized (including surveys, in-house expert assessments, and composite scores) and the dependent variables measured. Recommendations on variables to measure and data sources to use are made.

Paper Presentation

5. Remittances and Development: Systematic Review for USAID. Although remittances constitute a significant portion of capital flows to developing countries, U.S. government agencies struggle to harness these resources for development. A lack of understanding of remittances, their impacts on development, and effective interventions deters U.S. government strategy development on how to manage these flows. Due to confounding variables, reverse causality, and generalizability issues, researchers find conflicting development impacts of remittances. To overcome these challenges, this systematic review researches and compares the process and impacts of remittances in three developing countries in which these flows amount to a high percentage of GDP. Findings suggest that remittances reduce poverty yet disproportionately advantage wealthier countries and increase household income inequality. This paper identifies various activities within a remittance policy lever framework which encourage positive impacts and mitigate adverse effects. To develop a remittance strategy, USAID should conduct further research to confirm these findings, target initiatives and countries in which these findings hold true, and develop country-level activities most appropriate in local contexts.

Paper Presentation

6. USAID Initiative for Long Term Training and Capacity Building Program Tracking and Monitoring Study. This paper presents the findings of the initial stages of a tracking and monitoring study of the USAID Initiative for Long Term Training and Capacity Building Program (UILTCB). The survey was designed and implemented by the UILTCB management team at Michigan State University. Data analysis and recommendations are made with the purpose of informing the management team at MSU. This report concludes that the UILTCB program has been successful in increasing individual capacity, returning trainees to their sponsoring institutions, and increasing trainee responsibility within their sponsoring institutions. Trainees also report a number of intangible benefits such as increased knowledge, skills, and attitudes as well as being more culturally aware and more motivated and confident in their abilities and careers. Additionally, the majority of trainees received promotions and higher salaries upon completion of the program. Trainee feedback regarding program implementation is overwhelmingly positive with the exception of a few areas. Trainees felt that they did not have enough time to prepare for the GREs and some of the first cohort of trainees had issues related to stipends and obtaining housing in the United States.

Paper Presentation
U.S. State Department

7. Building a Stronger Western Hemisphere: A Case Study on Incentives to Participating in the Small Business Network of the Americas. This case study is focused on examining the domestic challenges of connecting the small business development centers within the United States to the Small Business Network of the Americas (SBNA) program. In particular, this case study aims to identify the incentives of getting domestic centers to participate in the SBNA program through increasing the use of SBDCGlobal.com and encouraging the formalization of center partnership agreements via the Sister Center program. Doing so will aid the Department of State and SBNA partners in realizing the White House’s goal of formalizing at least 20 Sister Center partnerships by the end of 2014.

Paper Presentation

World Bank

8. Designing an Assessment Framework for the E-reader Project in Djibouti. The “E-reader Project” is an educational pilot project to be implemented in rural Djibouti high schools. Its principal aim is to test whether the e-readers (such as the Amazon Kindle) cause attitudinal and behavioral changes in students’ daily reading and learning. Currently the pilot planning and procurement have not been concluded. This project assisted the project officer in designing the identification strategy as well as developing a cost-effective method to evaluate the effectiveness of the intervention.

In order to select the most appropriate identification strategy, the author took into account the sample size required and the project’s budget constraint, laying out several feasible options and compared the advantages and disadvantages of each option and recommended that the project adopt a clustered randomization strategy to identify the treatment and control groups. The author also reviewed the evaluation results of previous e-reader projects and other ICT projects, finding that the evaluators should put more emphasis on students’ attitudinal and behavioral changes rather than solely focusing on students’ attrition and attendance. The author adapted the Knowledge, Attitude, and Practice (KAP) survey as a cost-effective assessment method for the e-reader project.
Paper Presentation

9. Impact Evaluation Frameworks for Gender-Based Violence Community Mobilization Interventions in South Asia. To move from collective knowledge on gender-based violence issues to viable solutions that achieve desired gender outcomes, it is necessary to build an operational chain that links promising intervention identification with evidence-based opportunities to either replicate or scale up those identified as successful. This can only be achieved through more comprehensive and rigorous evaluation of gender based violence (GBV) interventions to establish what is and is not working in the field. This paper seeks to: (1) identify evaluation methods and tools used in CMIs and explain each method’s condition for use (e.g. pros and cons); (2) identify and discuss challenges associated with the evaluation methods being used, as well as challenges inherent to CMI design and implementation; (3) highlight innovative evaluation methods from three relevant CMI case studies; and (4) recommend an impact evaluation framework for gender-based violence community mobilization interventions in the region.

Paper Presentation

10. Using Poverty and Social Analysis for Policy. It is uncertain to what extent technical assistance (TA) generated by the World Bank’s Poverty and Social Impact Analysis (PSIA) is being used for policy. The following paper aims to analyze and track the use of these TAs in the energy and water sector in South Asia. The purpose of the study is two-fold: First, it aims to create an opportunity for better use of PSIA recommendations in the future. Analysis of this would be based on four indicators: if past recommendations have been implemented in the client country, if PSIAs have influenced the World Bank programs, if client countries have created legislations or new programs, and if PSIAs were requested from client governments. Second, it identifies possible themes for the future design of PSIA in Pakistan.

Paper Presentation

Catholic Relief Services

11. Global Alliance for Vaccinations and Immunization Civil Society Organization Project: A Review of Phase 1 and Phase 2 for Catholic Relief Services. Following two years of perceived success in its GAVI Civil Society Organization Project, Catholic Relief Services wants to analyze the organizational and procedural factors contributing to increased Civil Society Organization participation and representation in national health strategies. The GAVI CSO Project seeks to support the establishment of national civil society platforms in GAVI priority countries that advocate for health systems strengthening. As the GAVI project enters Phase Three, it is important for CRS to understand what constitutes success and what factors led to such a success. The following evaluation seeks to better understand the progress the program has made, and what the next steps should be in order to measure such progress.

Paper Presentation

12. Nutrition Driven Agricultural Development. Nutrition is a cross cutting issue that is intertwined with almost every aspect of development, but has been largely ignored by the international development agendas of this new millennium. The predicted failure of many countries to reach the UN Millennium Development Goals by 2015 has turned the attention of the developing world to nutrition. The Catholic Relief Services (CRS), like many other development practitioners, does not currently have a thematic focus for programming specifically aimed at improving nutrition, but the organization desires to explore methods of creating programs targeting nutrition outcomes. Instead of making new themes within the organization model, CRS can use one of its current thematic focuses to target nutrition. This project seeks to find a pathway to target nutrition through the CRS thematic focus of agriculture and answer questions on how to create a link between agricultural activities and nutrition impacts.

A case study of three nutrition driven agricultural framework from Feed the Future, the Global Alliance for Integrated Nutrition, and the UN Food and Agriculture Organization are analyzed and compared to each other to find reoccurring trends that will be asserted as best practices. Based on analysis from the frameworks, it is recommended that the existing CRS agricultural programs be reviewed and opportunities for integrating nutrition objectives be identified. Two CRS programs, The Great Lakes Cassava Imitative and the Wellness and Agriculture for Life Advancement in Malawi, are both analyzed and used to model of how to carry out the recommended steps of integrating nutrition into a CRS program.

Paper Presentation

13. Recommendations to Catholic Relief Services for Engaging in Climate Change Advocacy. This project is a policy analysis of current advocacy and program initiatives related to fighting climate change. It was undertaken with the idea that developing an understanding of climate change policy practices and initiatives will enable Catholic Relief Services’ (CRS) Public Policy team to engage with the US government in order to bring about needed changes and reinforce decisions in program design, funding, and policy development.

In order to synthesize the information available on this subject, an extensive literature review and expert interviews were conducted. The literature review consisted of an examination of CRS’ internal policies and documents as well as available publications from a variety of actors who address climate change. Expert interviews were conducted with CRS staff as well as external organizations to see how programming areas compared and to gain insights on current advocacy priorities. This information was then compiled into case studies in order to present findings in a comparable format.

Paper Presentation

14. Identifying Immediate Outcomes for Agriculture for Nutrition and Health (A4NH). Despite rapid progress in agriculture development, little evidence has been established connecting agriculture development with nutrition and health outcomes. The mission of Agriculture for Nutrition and Health (A4NH) is to unite the various research programs currently active in agriculture, health and nutrition to achieve the desired impact of good health and nutrition outcomes for the poor, especially women and children. A standard results framework that can easily be adopted by the different research centers to link the outputs to the established intermediate development outcomes, and eventually the impact, is essential. The analysis of the current framework indicates a need to define some immediate outcomes that directly measure the immediate changes that result from the research outputs.

This paper identifies immediate outcomes that will bridge the gap between research outputs and long term intermediate outcomes for A4NH related projects. Given the nature of the research outputs, the immediate changes that occur pertain to the ways research is used by the many stakeholders involved. Therefore, the immediate outcomes focus on research uptake and are categorized as: Knowledge Development, Capacity Development, and Relationship Development. These immediate outcomes will demonstrate how research is making a difference and influencing the achievement of the intermediate development outcomes. The ultimate success of the program relies on the functions of the internal program management unit where A4NH must initiate certain internal process pertaining to capacity development, establishing partnership with various stakeholders and developing effective communication strategies that will lead to the outflow of information.
Paper Presentation
Creative Associates International

15. Summative Evaluation of Creative Associates International’s Health Portfolio Expansion. Creative Associates International has reached a key point in their efforts for expansion into a key sector of international development work. They would like to gauge their progress thus far and their future potential in pursuing health work. So far, most emphasis has been given to pursuing HIV/AIDS prevention and assistance to orphans and vulnerable children. This project includes a summative evaluation of the process undertaken by Creative Associates to build a portfolio in the health sector and a set of recommendations for further progress and action.

Paper Presentation
Education for Employment

16. Comparing Outputs and Outcomes for Palestinian Education for Employment Job Training and Placement Programs. Education For Employment (EFE) is a non-profit organization whose mission is to provide economic opportunities for youth unemployed in the MENA (Middle East and North Africa) region by providing an array of professional training services. EFE works with locally run affiliates that administer these services across the MENA region. EFE’s regional affiliates then partner with business and universities to develop a curriculum that match the skills participants learn with the skills desired in the labor market. This particular evaluation assesses the outcomes for EFE’s Job Training and Placement (JT&P) Programs that were conducted in Palestine in 2013. There were 46 programs administered by the EFE’s regional affiliate, Palestine Education For Employment (PEFE). Of the 46 programs completed in 2013, the sample for this study consists of 35 JT&P programs administered by PEFE in conjunction with various institutional partners that provided funding and support for the programs.

Paper Presentation

17. Reviewing EFE-Tunisie’s Performance Measurement System: Opportunities and Methods for Improvement. Education for Employment sought to expand its M & E Performance Measurement System by adding a Results-Based Management Framework to track the performance of an individual project. The Moustakbali Project, funded by the Drosos foundation and implemented by the EFE-Tunisie affiliate, was selected as the pilot project to construct and use this tool for performance tracking and reporting to the funding organization. There is a unique opportunity afforded to EFE’s performance measurement system in constructing this tool, which includes not only Key Performance Indicators, but also other indicators required by the donor. By constructing a more exhaustive, although still not comprehensive, set of indicators that focus on one project in particular, the RBM Framework can also serve as a vehicle to evaluate the operation of EFE-Tunisie’s performance measurement system, on the ground.

The overarching question this project hopes to answer is whether EFE’s performance measurement system can effectively measure performance of particular donor-funded projects in real time. In evaluating this question, opportunities for improvement and methods for achieving this improvement can be diagnosed.
Paper Presentation
Freedom House

18. Constraining Civil Society in the Eurasian Region: An Examination of Legislative Trends. Authoritarian and semi-authoritarian regimes in the Eurasian region have initiated and constantly imposed various legislative restrictions on civil society organizations’ activities over the last two decades. Although the trends of constraining civil society organizations from doing human rights and democracy advocacy are becoming too clear around the world, so far there is not enough research to provide needed support and groundwork for designing a specific program to address this issue. The paper aims to examine whether trends of “copycatting” of restricting legislation exist, as well as to provide an assessment of possible options to address the problem from the perspective of the client (Freedom House).

Paper Presentation
Future's Group

19. A Policy Analysis on the Exclusion of Orphans and Vulnerable Children from Mozambique’s Basic Social Subsidy Cash Transfer Program. The Basic Social Subsidy Programme (PSSB) is the largest cash transfer system in Mozambique, providing vital support to the country’s most vulnerable populations. This paper analyzes gaps in the PSSB through the lens of the US Government’s President’s Emergency Plan for AIDS Relief (PEPFAR), focusing on the exclusion of Orphans and Vulnerable Children (OVC) from the program. Though eligible to receive benefits, OVC are systematically under-covered. This paper identifies the problem and its causes, ultimately recommending that PEPFAR conduct an implementation evaluation to shed insight into the operational disconnects that may be taking place within the PSSB.

Paper Presentation
International Association for Community Development

20. International Association for Community Development (IACD): A Needs Assessment of E-bulletin Subscribers. International Association for Community Development (IACD) is an international NGO working to promote social justice through community development. It has more than 3,500 subscribers for its free monthly E-bulletin, but only 331 are paid members. IACD seeks to find out how to reduce this free-riding issue and get more paid members. Based on this background, an online needs assessment survey was conducted. In this report, recommendations are provided for IACD according to the survey findings. IACD could provide online training, international tours, and more networking opportunities for paid members. For members whose membership is expiring, IACD could send out reminder messages twice to remind them to renew their membership. In addition, more convenient membership fee payment method can also be explored to increase the number of paid members.

Paper Presentation
Poverty Action Lab (JPAL)

21. Implementation Evaluation Design Mi Parque Program. This project consists of an implementation evaluation design for the program developed by the Mi Parque Foundation in vulnerable neighborhoods in Santiago, Chile. The evaluation design responds to a requirement made by the Latin America & Caribbean office of Poverty Action Lab (J-PAL), and seeks to complement the impact assessment of the current Mi Parque program this organization is developing today. This report presents the problem that motivates the intervention of Mi Parque, and then briefly describes the foundation and the program. It then describes in detail the design proposal and provides recommendations for proper implementation of the evaluation.

Paper Presentation

Back to top