About Us
Child Abuse Training
International Activities
Rossi Award for Program Evaluation
UMD Capstone Courses
Mailing List
Contact Us

Student Reports

Spring 2015

Professors Douglas J. Besharov and Douglas Call


In Spring 2015, students worked with clients at U.S. Agency for International Development, United States Department of Agriculture, NASA, World Bank, Catholic Relief Services, and the Education for Employment. Students performed a wide variety of analyses, including implementation evaluation, policy analysis, performance measurement development, summative evaluation, and research synthesis.


The following are the final projects prepared by the students. They are listed in order of clients.

Note: The associated files are best viewed using Adobe Reader. To get the software, go to http://get.adobe.com/reader/

Federal Government

U.S. Agency for International Development

1. Women's Empowerment in Agriculture Index: Assessing the Utility of Autonomy Indicator for Research and Practice. The Women�s Empowerment in Agriculture Index (WEAI) is an innovative new survey tool that directly measures components of women�s empowerment and inclusion in the agricultural sector. Developed to track gender outcomes of the US Government Feed the Future (FTF) Initiative programs, the WEAI is currently undergoing revision for length, clarity, and ease of use to increase its utilization beyond FTF implementing partners. Some challenges arose during piloting surrounding the Autonomy indicator, including lack of respondent comprehension, translation of questions, and time necessary to administer the module questions. A literature review identified several common themes commonly arose surrounding the concept of women�s autonomy, including its potential context dependence and threats to validity. Benefits to the measurement of autonomy were also identified. The possibility of using the autonomy indicator as an additional and optional module is found to be sensible as it keeps the benefits offered by the indicator available for those interested, while reducing the overall time of the core WEAI by the target amount.


2. The Ukrainian Energy Quandary and How Biomass May Be the Answer. The Ukraine is in a time of political, economic, and social uncertainty. Foreign energy reliance heightens the uncertainty and this policy analysis aims to highlight options for the Ukrainian energy sector that reduce foreign energy reliance, increase environmental sustainability, and aid in development of one of the most promising renewable energy sources of the nation, biomass.


3. Evaluating Performance Measures While Exploring Direct and Indirect Beneficiaries within USAID�s Feed the Future Initiative. I worked with the Monitoring, Evaluation and Learning (MEL) team of the Bureau for Food Security (BFS) at USAID on the United States Government�s (USG) Feed the Future (FTF) initiative, to identify and develop methods to monitor, track, and count indirect beneficiaries of FTF activities, including analyzing performance metrics that track direct beneficiaries. While current FTF metrics count and track the reach of FTF activities to direct beneficiaries e.g. target farmers at demonstration events, and people receiving goods, FTF metrics do not track the reach of FTF activities to indirect beneficiaries via spillover effects.

My specific tasks were to: a) survey Implementing Partners (IPs) receiving FTF funds and identify: i. how they currently define beneficiaries (direct and indirect) within their monitoring systems, ii. how they monitor, track, and count them, and iii. review the robustness of these IPs monitoring practices; b) analyze the larger development field (i.e. other donors, other sectors, etc.) and investigate whether development practitioners are using methods to monitor, track and count indirect beneficiaries, and if such practices exist, review their robustness. c) Finally, I present this report and that will detail my survey and reviews in parts (a) and (b), and to the extent possible, provide recommendations on how IPs working with FTF funds may monitor and measure the complete reach of their activities to direct and indirect beneficiaries.

U.S. Department of Agriculture

4. A Research Synthesis & Policy Analysis of Current Opportunities in Youth Agricultural Development Programs. Rural youth in Serbia face significant challenges to employment. Underperforming education systems, small informal agriculture production, and rural-urban flight have continued to decrease prospects for economic prosperity. At the same time youth interest in agricultural employment has declined. USDA has invested in increased youth engagement in agriculture with 4-H pilot programs with some success though evaluation of program outcomes and impacts is limited. A review of youth agricultural programs implemented globally indicate that sustainable agriculture training has the potential to increase youth interest in agricultural production and employment at a lower cost than other models.


5. Improving Social Impact Performance Measures for NASA�s Regional Visualization and Monitoring System (SERVIR) Program. This study aims to analyze and propose improvements to the existing performance measures used by NASA�s Regional Visualization and Monitoring System (SERVIR) program, focusing especially on improving the measurement of long-term social outcomes and impacts. This was done by reviewing the program�s literature, constructing a logic model for the program, proposing new indicators and improvements to the existing ones, evaluating the proposed indicators, and interviewing program staff to gather additional insight. The findings are a list of recommended indicators, possible improvements to the program�s results framework and possible improvements to the M&E operations in the field. I recommend that SERVIR expand its results framework to include the activities and objectives that have been left, improve some existing performance measures to make them more clear and operational for the hub�s staff, and add new custom performance measures to account for the social impacts that the program is attempting to achieve.


6. Impact Evaluation Design for The Nasa Servir Applied Sciences Team. In order to design a framework for an impact evaluation of the NASA Applied Sciences Team, the author conducted interviews with 11 scientists on the Team and performed a literature review of indicators from other development organizations. The author found that physical scientists on the Applied Sciences Team had great interest in learning the impacts of their work, and that they would willingly participate in evaluation data collection efforts. These efforts must, according to the scientists, be supported through the Team�s Coordination Office in order to provide the social science knowledge needed to conduct such an evaluation. Principal Investigators indicated that they would be willingly to use their network of beneficiaries to solicit information for the impact evaluation, such as through surveys delivered via email. The Principal Investigators also indicated that they would prefer early engagement in the evaluation process and a clear statement of data collection expectations in upcoming calls for new Applied Sciences Team cohorts in order to properly allocate resources to the task. The author recommends early engagement with Principal Investigators to establish indicators for an evaluation that accurately capture the theory of change behind the work done by individual researchers.


World Bank

7. Performance Measures for In-Service Teacher Training Programs in Developing Countries. This report seeks to identify and develop performance measures for in-service teacher training programs funded by the World Bank in developing countries. Teacher training programs often constitute an important component of education programs funded by the World Bank in developing countries. The rationale is that training programs in developing countries are often inadequate and of unsatisfactory quality. Several World Bank projects therefore include teacher training as a component to increase the overall success of the program. The following study seeks to identify performance measures for teacher training programs funded by the Bank which may be used to improve the quality of the program.


Catholic Relief Services

8. East Africa Coffee Livelihoods Diversification Project. The Coffee Livelihoods Diversification Project implemented by Catholic Relief Services (CRS) seeks to improve the livelihoods of smallholder coffee farmers in Ethiopia, Kenya, and Rwanda by implementing activities that seek to (1) increase food security and (2) increase access to financial services. This lessons learned document provides insight into key findings regarding the project�s successes and limitations. It additionally offers insight into current issues related to smallholder productivity and costs, as well as environmental concerns associated with the coffee value chain. Based on these findings, the document offers suggestions for ways in which a future CRS program can work with smallholder coffee farmers in East Africa to improve the coffee value chain.

Education for Employment

9. The Middle East and North Africa (MENA) region has the highest youth unemployment rate in the world. The regional youth unemployment rate is 25%, and reached 38.9% in Egypt in 2013. Youth unemployment rates are even higher for women, a rate that reaches 71% in Egypt. The mismatch between the skills of the population and the needs of the labor market is one of the major problems contributing to high youth unemployment rates in MENA. Unemployment rates are high even among those who have completed tertiary education, especially among women. In Jordan, the unemployment rates of female and males who have completed tertiary education are 69.7% and 22.9%, respectively.

Education for Employment (EFE) seeks to address this problem by providing MENA youth with training programs that are aimed at developing the skills that cater to the labor market. EFE operates in Egypt, Jordan, Morocco, Tunisia, Palestine, and Yemen, offering three programs for participants.

I conducted a literature review to gather more information on women�s unemployment and to identify knowledge gaps on the subject. Based on my literature review, I conducted two surveys that were distributed to EFE employer partners as well as EFE alumni in order to gather background information relating to women�s employment as well as to assess their attitudes and preferences regarding women in the workplace. Overall, many women stated that full-time work with flexible working hours, parttime work, a child-care center within the company�s facilities, and a work-from home option would help them stay in the workplace after having a child. Employers had mixed responses to the questions, depending on which country they answered from. For example, Yemen had more negative/neutral attitudes regarding women�s capabilities than other countries. The survey had some limitations, including respondent error, unit non-response error, survey tool difficulties, limited number of respondents, as well as limited range of answers. The next step would be to expand the survey past the six countries to all of the MENA region, gathering responses from women who are not necessarily alumni of the EFE program in order to target a more diverse population.


Back to top