About Us
Child Abuse Training
International Activities
Rossi Award for Program Evaluation
UMD Capstone Courses
Mailing List
Contact Us

Student Reports

Spring 2018

Professors Douglas Besharov, Douglas Call, and Carl DeLorenzo

In Spring 2017, students worked with the Government Accountability Office, the Montgomery County Board of Elections, the Montgomery County Police Department, the Maryland Governor's Office for Children, the Maryland Interagency Council on Homelessness, the National Endowment for the Arts, the South Baltimore Gateway Partnership, the U.S. Department of Education (Office for Civil Rights, Office of English Language Acquisition, Office of State Support), and the U.S. Department of Health and Human Services (Administration for Children and Families, National Center for Health Statistics, Office of Rural Health Policy). Students performed a wide variety of analyses, including needs assessment, performance measure development, policy analysis, process evaluation, and program design assessment.

The following are projects prepared by students. They are listed in order of clients.

Note: The associated files are best viewed using Adobe Reader. To get the software, go to http://get.adobe.com/reader/

County Government

Montgomery County Board of Elections

1. Absentee Ballot Canvassing in Montgomery County: A Process Evaluation. The Montgomery County Board of Elections (MCBOE) administers voting within the borders of Montgomery County. In 2016, there were almost 720,000 registered voters within the county. Of these voters, 70,578 of them voted absentee. These absentee ballots are the subject of this report. Tabulating absentee ballots is more involved than in-person ballots. Absentee ballots must first be canvassed: a process of sorting and preparing the ballots to be counted. The absentee ballot canvassing process is an involved one. In 2016, 48 employees spent a collective total of 3,826 hours canvassing each ballot that was returned. The MCBOE aims to complete the canvass as quickly and accurately as possible, allowing for final results to be tabulated earlier and giving Montgomery County residents faith in the integrity of their elections. This study is a process evaluation of the absentee ballot canvassing process. The outcomes of the cavass will be compared against its goals to determine the efficacy of the process design and any improvements that can be made.

Paper Presentation

Montgomery County Police Department

2. A Process Evaluation: The Montgomery County Police Department Montgomery County, Maryland. In a partnership with the Montgomery County Police Department, I have conducted a process evaluation. The purpose of this process evaluation is to determine if the Montgomery County Police Department has been serving residents as intended by collecting the residents� views and perceptions of the police department. For this process evaluation, opinions and perceptions of Montgomery County residents were collected through a community-based survey. Based on the findings of the community-based survey, recommendations were made to the Montgomery County Police Department suggesting ways to improve their engagement with the community and deliver services in ways that will meet the expectations of residents.

Paper Presentation

State Government

Maryland Governor�s Office for Children

3. A Performance Measurement Analysis on the Maryland Governor�s Office for Children�s Child Well-Being Indicators. I have conducted a performance measurement analysis of the Maryland Governor�s Office for Children�s thirty performance measures1 for child well-being. The analysis was conducted through the use of both quantitative and qualitative data. I used current and historical data, current performance measurement definitions and qualities that indicate good performance measures to rate each indicator on a scale between 1 and 3. The overall purpose of this analysis was to determine whether these indicators are accurate enough for use within the state of Maryland. The following research questions were used in making the final analysis and recommendations:
1. What are the current reasons for using the existing performance measures?
2. Do the performance measures follow Theodore Poister�s criteria for good performance measures?
3. Do the performance measures adhere to the definition of an indicator established by Glennerster and Tavarasha?
4. Is it feasible to replace or remove performance measures that do not the fit criteria and/or definition laid out for this report?

Paper Presentation

Maryland Interagency Council on Homelessness

4. Engaging Consumers of Homlessness Services in Maryland: Recommendations to Increase Consumer Participation in the Policy-Making Process Across the State. This research seeks to assist the Interagency Council on Homelessness (ICH) in increasing consumers� engagement and incorporating their experience into the policy-making process of homeless services across the state of Maryland by providing recommendations on the development of Consumer Advisory Boards (CABs), forums where a representative committee from the community of consumers participate as partners with governing bodies to advice on policy. Currently, the community of individuals experiencing homelessness consists of different subpopulations, but the state only receives direct input from one consumer member at the Maryland Interagency Council on Homelessness, whom is meant to represent the experience from numerous homeless subpopulations. Hence, there is a lack of consumer participation in decisions affecting their lives. This study assessed how local Continuums of Care (CoCs) are engaging with consumers in the policy-making process, the feasibility and effectiveness of developing a statewide CAB and provided recommendations to increase consumer engagement.

Paper Presentation

Federal Government

Government Accountability Office

5. Barriers to employer participation in programs under the Workforce Innovation and Opportunity Act (WIOA). The Government Accountability Office would like to learn more about barriers to employer participation in programs under the Workforce Innovation and Opportunity Act (WIOA), which reauthorized the Workforce Investment Act of 1998 (WIA). I am conducting a policy analysis to explore this topic. This analysis will rely on publicly available data, current literature, and interviews with local Workforce Development Board (WDB) members.

Paper Presentation

Government Accountability Office, Education, Workforce, and Income Security Team

6. Mandatory Arbitration in Employment Contracts. In partnership with the Government Accountability Office�s (GAO) Education, Workforce, and Income Security (EWIS) team, I have assessed the extent and consequences of mandatory arbitration clauses in employment contracts (�mandatory arbitration�). Mandatory arbitration causes have become increasingly prevalent in employment contracts over the past three decades. To better understand mandatory arbitration, I worked with the team at GAO to develop four research questions that are the focus of this paper:
1) What is the extent of mandatory arbitration in employment contracts? How many employees and employers are affected?
2) What are the consequences of arbitration agreements for employers? What are the consequences for employees?
3) What policy responses are being considered in response to the use of arbitration by employers?
4) What is the predicted outcome of the upcoming Epic Systems case, and what are the likely consequences of a ruling in favor of either party?

Paper Presentation

National Endowment for the Arts

7. A Qualitative Analysis of Folk and Traditional Arts Art Work Grants. The National Endowment for the Arts (NEA) has embarked on a multi-phase study to examine the infrastructure for Folk & Traditional Arts (F&TA) in the United States and the agency�s role in supporting the discipline. In particular, the NEA seeks to understand the nature and impact of its F&TA Art Works grant program, which provides project-based support to organizations working in the folk and traditional arts. To gain a broader understanding of these grants, this content analysis explores the connections between project objectives, inputs, activities, outputs, and outcomes as reported by grantees through five interrelated research questions: What are the objectives of grantees? What was the underlying context? What are the inputs to F&TA projects? What are the activities supporting F&TA in NEA-supported projects? What are the outputs of NEA-supported F&TA projects? Finally, what are the outcomes of NEA-supported F&TA projects?

Paper Presentation

United States Department of Education, Office for Civil Rights

8. Policy Analysis of the Civil Rights Data Collection (CRDC). This project was prepared for the Department of Education (Department) through the Office for Civil Rights (OCR), which administers the Civil Rights Data Collection (CRDC). The CRDC is a data collection that has been a key part of OCR�s strategy for administering and enforcing civil rights laws since 1968. OCR is concerned that certain state and local education policies are impacting how school districts are responding to the survey, ultimately affecting the data collection process. This report provides a policy analysis of the CRDC to identify relevant policies and analyze their effects on the collection and reporting of data. The project also consists of an examination of similar large Department of Education data sets. The research and analysis is then aggregated into options to OCR for improvement of the data collection process, and recommendations are then offered based on the feasibility of each option.

Paper Presentation

United States Department of Education, Office of English Language Acquisition

9. The School-Level Relationship Between EL Incidence and EL Achievement: CO, FL, IL and NM. Since the No Child Left Behind Act (NCLB) made English learners (ELs) a priority student sub-group, additional resources and attention have been focused on EL achievement across America. The Every Student Succeeds Act (ESSA), the successor to NCLB, has in turn maintained the prioritization of EL achievement and the federal reporting requirements of EL achievement data. But, has this focus on EL academic achievement benefitted all ELs equally? Do low-incidence EL schools produce the same results for ELs as high-incidence EL schools? This line of questioning has been largely absent from education production function literature. The purpose of this paper is to determine if there is a relationship between the number of ELs in a school and EL math and reading proficiencies on state standardized tests. This is the question my client, the US Department of Education�s Office of English Language Acquisition(OELA), would like to answer. After 15 years of federal funding for and prioritization of EL achievement, there should be enough evidence to answer this important question. If disparities are found, then it is up to educators and policymakers to work together to correct ESSA�s EL-related shortcomings.

Paper Presentation

United States Department of Education, Office of State Support

10. Advantages and Challenges to Approaches State Educational Agencies Use to Monitor Subgrantees of Selected Elementary and Secondary Education Act Programs (Title I, Part A; Title II, Part A; and Title III, Part A). My Policy Engagement Project focuses on the monitoring practices of State Educational Agencies (SEAs) for large federal formula grant programs administered by the Office of State Support (OSS) in the Office of Elementary and Secondary Education (OESE) at the U.S. Department of Education (ED). My main research question and subquestions are:
1. What are the advantages and challenges to various approaches SEAs use to monitor subgrantees of selected large federal formula grant programs including the Elementary and Secondary Education Act (ESEA) Title I, Part A; Title II, Part A; and Title III, Part A?
2. What approaches do SEAs use to monitor subgrantees of selected large federal formula grant programs?
3. What are the components and processes of the approaches SEAs use to monitor subgrantees of selected large federal formula grant programs?
4. What are the strengths and limitations of each approach that SEAs use to monitor selected large federal formula grant programs?

Paper Presentation

United States Department of Health and Human Services, Administration for Children and Families

11. An Evaluation of Three Early Childhood Workforce Reports: Current Processes and Future Research. Over the past ten years, renewed attention to the early childhood workforce has highlighted the crisis of low compensation afflicting the group responsible for caring for and educating children in their most formative years. Therefore, the Administration for Children and Families (ACF) asked me to complete an evaluation of three early childhood education and care workforce reports published in the past three years, particularly regarding the wage data presented. The three reports, High Quality Early Learning Settings Depend on a High-Quality Workforce: Low Compensation Undermines Quality1; 2016 State of Preschool Yearbook (and the 2015 Workforce Supplement); and the Early Childhood Workforce Index, take different approaches to reporting on early childhood education and the early childhood workforce. I conducted an evaluation that incorporates elements of literature reviews/cross study-analyses and process evaluations. This project will inform my client about the current state of the early childhood workforce, and research related to the workforce, as ACF moves forward with future policies and activities.

Paper Presentation

12. States� Investments in Pre-K Educators Under Federal Race to the Top Early Learning Challenge and Preschool Development Grants. This process evaluation was prepared for the U.S. Department for Health and Human Services Administration for Children and Families. In eight states which received either Race to the Top Early Learning Challenge (ELC) or Preschool Development Grant (PDG), this examines what kinds of initiatives have state grantees undertaken to increase workforce compensation, to what degree have they been/will they be sustained after the grant; and what common factors have affected those decisions. Methods included reviewing most recently published 2016 Annual Performance Reports of each state, interviewing state Grant Directors or other personnel who had worked on the grants, and reviewing online materials on specific programs and state policies. By increasing ACF�s understanding of how and why states invest in their preschool workforce during and after federal grants, this project is intended to inform future grant design to maximize the sustainability of state compensation initiatives.

Paper Presentation

United States Department of Health and Human Services, Federal Office of Rural Health Policy

13. Rural Health Clinics, Quality Improvement, and Value-Based Care: A Needs Assessment for the Federal Office of Rural Health Policy. In order for the Hospital State Division to develop an appropriate response to the challenges facing Rural Health Clinics (RHCs), certain fundamental questions will have to be answered. The overarching question that is at the core of this needs assessment is: How can the Flex and SHIP programs be used to prepare provider-based RHCs for quality improvement and value-based care? Before that question can be adequately answered, we must develop a more comprehensive understanding of the RHC situation. For that purpose, this needs assessment responds to the following questions: How have the pertinent organizations developed an understanding of RHC conditions and needs, and how does the situation differ from state to state? What current programs are being run to help RHCs build their capacity for quality improvement activities and value-based care? What data has been collected from RHCs? How do State Offices of Rural Health use the Flex and SHIP programs to assist provider-based RHCs, and what are their thoughts on the use of these programs for RHCs in the future?


United States Department of Health and Human Services, National Center for Health Statistics

14. Policy Analysis of Health, United States� Website Format and Whether the Format Meets User Needs. I conducted a policy analysis for the National Center for Health Statistics (NCHS) to assess whether Health, United States� (HUS) website features currently meet user needs. NCHS is interested in improving the format of their HUS website features to better meet the needs of their presumed primary users: policymakers and researchers. In order to complete this policy analysis, I had to answer three questions: who is using HUS?; what are users� needs and opinions of the current web format of HUS?; and how do HUS�s website features compare to other federal agencies and publications that compile and release large amounts of data?

Paper Presentation


South Baltimore Gateway Partnership

15. Impact Performance Indicators for the South Baltimore Gateway Partnership. The South Baltimore Gateway Partnership (SBGP, Gateway Partnership) focuses on improving sixteen neighborhoods (full list can be found in the Appendix) in the area surrounding the Horseshoe Casino in Baltimore City. By state law, a portion of casino revenues must be used as Local Impact Grants to improve the community in proximity to the casino. The City of Baltimore originally developed a South Baltimore Gateway Master Plan to understand the needs of the community that could benefit from the additional funding. The high and complex needs led the city to establish the Gateway Partnership in 2016 to serve as a more flexible and entrepreneurial organization to distribute funds. As a result, SBGP created a strategic plan focusing on three strategic areas stemming from the South Baltimore Gateway Master Plan to create depth in their efforts. The SBGP strategic areas are: Community Development and Revitalization, Environmental Sustainability, and Health and Wellness. Thus, my research questions are as follows: What are best-in-practice indicators per strategic area? What realistic and useful impact performance indicators should SBGP collect for impacts? What are the available indicators from Baltimore City data sources relating to the SBGP�s performance measures?

Paper Presentation