About Us
Child Abuse Training
International Activities
Rossi Award for Program Evaluation
UMD Capstone Courses
Mailing List
Contact Us

Student Reports

Spring 2014

Professors Douglas J. Besharov, Douglas Call, Christopher Mihm, and Kevin Jones

In Spring 2014, students worked with clients at Baltimore City Public Schools, Capital Partners for Education, Folger Shakespeare Library, the Hilltop Institute, Magnets Schools of America, Maryland Department of Health and Mental Hygiene, Maryland Department of Human Resources, Montgomery County CountyStat, Montgomery County Department of Corrections and Rehabilitation, Montgomery County Department of Parks and Recreation, National Endowment for the Arts, U.S. Department of Health and Human Services (Agency for Healthcare Quality and Research, Office of the Assistant Secretary for Planning and Evaluation, and the Administration for Children and Families), U.S. Department of Housing and Urban Development, U.S. Government Accountability Office, and the U.S. Social Security Administration. Students performed a wide variety of analyses, including cost-benefit analysis, cost-effectiveness analysis, policy analysis, performance measure development, process evaluation, and research synthesis.

The following are the final projects prepared by the students. They are listed in order of clients.

Note: The associated files are best viewed using Adobe Reader. To get the software, go to http://get.adobe.com/reader/

Municipal Government

Baltimore City Public Schools

1. Evaluation of BCPS School Survey Questions on Parent Involvement. Parent involvement is an important indicator of student achievement. The Baltimore City Public Schools (BCPS) system is interested in both understanding the current level of parent involvement in their students� academic lives and ways to improve the BCPS Annual School Survey questions that attempt to measure parent involvement. Using the results from the 2012-2013 School Survey for Parents, the purpose of this evaluation is threefold: 1) to describe the BCPS School Survey for Parents, its purpose, and delivery; 2) to analyze the effectiveness of the three Parent Involvement questions on the survey as valid indicators of Parent Involvement; and 3) to provide options for both better capturing the current level of parent involvement in BCPS students� education and possible alternative methods to improve survey response rates.

Paper Presentation
County Government

Montgomery County Department of Corrections and Rehabilitation

2. Recidivism in Montgomery County, Maryland. This project examines recidivism as a correctional systems metric in Montgomery County, Maryland. The first component serves as a resource and springboard for CountyStat�s investigation into a recidivism measure. It explores the magnitude, causes and patterns of recidivism, generally defined as the return of an ex-offender to the criminal justice system. It explains the methodological and theoretical problems in regarding recidivism as an evaluative measure of program performance and comparing recidivism rates across jurisdictions. After identifying great diversity amongst jurisdictions in the various elements of a recidivism definition (measure type, time period, triggering act, and informing databases) this paper concludes that Montgomery County Pre-Release Center (PRC) should maintain its current definition of recidivism. The second component of this paper consists of a quantitative analysis of a sample of releases by the PRC in 2010 and 2012. The highest rates appeared among males, young adults, those without college education, higher LSIR, African-Americans, and Drug Court offenders. Applying regression analysis reveals that the apparent differences by race and gender are attributable to correlation with the true predictors of recidivism: age and LSIR risk. Interventions and resources should be targeted to these populations.

Paper Presentation

Montgomery County Department of Parks and Recreation

3. Design of Performance Indicators for Recreation Program & Facilities in Montgomery County. This project proposes a comprehensive set of indicators/measures for reporting and assessing the performance of a wide-range of recreation activities and facilities in Montgomery County, including Aquatics, Regions, County-wide Services, Seniors and Teens. Types of outputs and outcomes that are worth tracking and reporting on a consistent basis are denoted. The appropriate frequency of reporting will also be addressed. The measures will be able to communicate to Department Management, CountyStat, and other stakeholders the progress each team is making towards their individual goals, while supporting the overall mission of the department. At the same time, to adapt to technical advancement, the indicators are designed to be compatible with a new system, ActiveNet, that is replacing the current primary operating software, CLASS.

Paper Presentation
State Government

Maryland Department of Health and Mental Hygiene

4. Performance Measure Evaluation of the Developmental Disabilities Administration�s Residential Facilities: Steps Toward Institutional Unification of Performance Measurement Tools and Accreditation. The Developmental Disabilities Administration (DDA) has four facilities devoted to community integration of the Intellectually Disabled (ID) community. The DDA is continuously working to fulfil the program goal of community integration for as many residents as possible and as quickly as possible based on the resident�s preparedness for reintegration. Currently, all four facilities operate using different sets of performance measures. The purpose of this project was to develop a set of unified performance measures for all DDA facilities. If all four of the DDA facilities can be unified under a single set of performance measures that achieve the long-term goal of rehabilitating and reintegrating the ID client base back into the community, they will be able to treat and rehabilitate the greater ID community that currently go without treatment/services and satisfy both state and federal legislative requirements. Additionally, unifying performance measures for all DDA facilities will assist the DDA in providing long-term solutions for reaching program goals of community reintegration and national accreditation.

Paper Presentation

5. Restructuring the Oversight of the Maryland Department of Health and Mental Hygiene�s Police Force: A Policy Analysis. The Maryland Department of Health and Mental Hygiene (DHMH) currently lacks a centralized system for its police force. This has resulted in two distinct challenges. First, DHMH has no standardized policies related to the daily activities and conduct of their officers. The lack of universal policies makes the standardization and evaluation of performance among all DHMH officers difficult, particularly when assessing comparable duties and responsibilities across all DHMH facilities. It also makes it more challenging to maintain consistent disciplinary standards should officers exhibit certain discouraged behaviors or fall below expected performance standards. In this analysis, I evaluate the options available to DHMH and provide a recommendation based on those options.

Paper Presentation

Maryland Department of Human Resources

6. Enhanced Earned Income Disregard as a Path to Self Sufficiency among Maryland Welfare Recipients. This report is an evaluability assessment of an enhanced income disregard pilot program that will be put in place in Maryland�s welfare program in October 2014. The Maryland Department of Human Resources is looking for insight into how such initiatives have performed, and under what circumstances, as it prepares to implement the pilot program. This report finds that evaluations of these disregards, and of similar programs that provide cash incentives for employment, have shown results that are mixed, yet largely insignificant. In some cases, employment and/or earnings have shown to increase for single parents or for long-term welfare recipients, for example. On the other hand, one broader examination found that overall, impacts of earnings disregards are minor and not statistically significant. Based on past examples, results of the pilot program should be evaluated with consideration to the fact that enhanced earned income disregards have traditionally been used in moderation and in tandem with other job-related requirements and incentives.


7. Maryland�s Payment Accuracy Plans for the Supplemental Nutrition Assistance Program: An Implementation Evaluation to Improve Effectiveness of PAPs at the Local Level. This study evaluates the requirements of current Food Supplement Management Evaluation (FSME) and Payment Accuracy Plan (PAP) process according to federal regulations; determines the data elements that will be used in measuring the effectiveness of the PAPs established by the local departments; and develops an implementation strategy for the new PAP for 2014. To complete this implementation evaluation, a literature review, key informant interviews, logic models, and data from Maryland�s Bureau of Program Evaluation are utilized.

This implementation evaluation uncovers several challenges and successes during the PAP process. Communication between the central and local office staff appears excellent. The central staff is knowledgeable with both local and state level experience. Data discrepancies, resource limitations, and lack of technology trainings limit continuous monitoring and evaluation by the local jurisdictions. These issues also minimize local jurisdictions� abilities to improve current PAPs and error rates. Formal training and reviews of best practices should be implemented to enhance current PAP processes. In addition, an evaluation of current error rate calculations, as well as education and training of local jurisdictions regarding the assumptions used to conduct calculations, should be established. Alternatively, specific staff should be assigned to assist local jurisdictions in conducting monthly calculations to enhance continuous monitoring and evaluation.

Paper Presentation
Federal Government

National Endowment for the Arts

8. National Endowment for the Arts: Arts Data Profile Page. The National Endowment for the Arts (NEA) is an organization that seeks to generate more arts activities throughout the country and to bring more awareness of the arts already present in the U.S. Without the spread of knowledge on the arts, the growth of cultural activities and the impact of NEA grant making would be diminished. Documentation and promotion of arts funding activities will help continue to demonstrate the need for the NEA�s grant programs. Moreover, the diffusion of research and knowledge about the arts comes at the same time as the the Open Government Initiative is attempting to increase transparency in government practices. As an effort to encompass all of these needs, one initiative of the NEA�s research and analysis division has been to create the Arts Data Profile Pages for their website. The objective of these pages is to spotlight specific datasets from various sources related to art. By presenting these datasets in a clear and digestible fashion, more people can understand artistic contributions to the country and use the information to further examine related topics. To date, the NEA does not have a profile page for their data on grants. The purpose of this project is to create a data profile page for the NEA�s website showcasing NEA grant data from both direct and state level grants.

U.S. Department of Housing and Urban Development

9. Food Access, Location Efficiency and Public Health Outcomes for HUD assisted Residents Living in Food Deserts in Baltimore County. Given the increasing evidence that food deserts may be affecting the health of low income Americans, efforts by the federal government to increase access to healthy and affordable foods are underway. As part of their efforts to address food deserts the Department of Housing and Urban Development (HUD) commissioned a needs assessment on the process of combining food desert data from another federal agency, the Department of Agriculture, and point level housing data to explore the incidence of HUD assisted families living in food deserts. The study was conducted by a University of Maryland School of Public Policy student in conjunction with HUD staff from January to May 2012. This report builds on the research done by the federal government and other research entities to inform the results of the maps created, analyzes the area of Baltimore County to report the incidence of HUD assisted residents and food deserts and identifies research gaps and methods for improving the mapping process.

Paper Presentation
U.S. Department of Health and Human Services

10. Court-ordered Treatment for Substance Abuse: A Policy Analysis. Within the broader justice-involved substance abuse population there is a subpopulation of individuals that are ordered by the court to receive treatment. This population faces additional barriers and constraints to treatment and there is limited research specifically focusing on this population. Therefore, this project examines historical and current issues facing the justice involved-substance abuse population, with special attention to the court-ordered to treatment population. Additionally, this project explores opportunities for increasing health insurance coverage for this population provided through the Affordable Care Act (ACA), state legislation, and Essential Health Benefits. Lastly, this project provides policy options for consideration in improving access to coverage for this population.

Paper Presentation

11. Drug Testing for TANF Benefits: State Policies and Proposals. The debate over substance abuse among welfare recipients has been an issue of large concern. Legislators believe that substance abuse hinders recipients� employability and ability to retain a job. Furthermore, parents� abuse of controlled substances can play a role in child abuse and neglect. The solution to reduce substance abuse among welfare applicants and recipients has been to initiate drug tests. Consequences for a positive drug test include participation in a substance abuse treatment program and denial of TANF benefits. The goal of substance abuse treatment programs is to decrease individuals� drug dependence, which in turn allows individuals to become more self-sufficient and reduces child abuse and neglect. Denial of welfare benefits for individuals with positive drug tests is the policy option for states that want to reduce program costs and ensure substance abusers are not receiving welfare benefits. This analysis on drug testing for TANF benefits will discuss the background of the policy, findings from state policies and proposals, policy options for consideration, and recommendations.

Paper Presentation

12. Father Involvement in Home Visiting Programs. While the stated aim of home visiting programs is to provide resources and support for at risk families, in practice most home visiting programs focus on the relationship between the mother and child. Fathers have been largely left out of home visiting models despite research indicating that a father�s presence during a child�s early years can lead to improved developmental outcomes. This exploratory case study sought to examine the challenges around engaging fathers in home visiting services. This study focused on one organization that offers home visiting services. In-depth interviews were conducted with staff, including the executive director and three home visitors. During the interviews, staff shared their experiences working with fathers, challenges in working with fathers, and the potential benefits of doing so.

Paper Presentation

13. Pediatric Health Quality Measures Program: A Process-Case Study. Increased spending on children�s healthcare caused increasing demand for measures of pediatric healthcare quality. However, several issues were identified within the plethora of measures that had been developed by the early 2000s. In response, the Pediatric Quality Measures Program (PQMP) looked to improve upon the child core set of measures developed by CMS�both were products of the Children�s Health Insurance Program Reauthorization Act of 2009. This project utilizes a process evaluation to synthesize commentary from 12 federal officials and Centers of Excellence stakeholders on the successes, accomplishments, needs, and challenges of the program as well as the process of measure development and implementation. The groups agreed on a number of topics related to the inputs, activities, and outputs of the program. However, there was variance in their responses related to program design, theory and outputs. As such, there is an opportunity for greater learning and clarification once program administrators re-examine the program�s logic model.

Paper Presentation

14. Process Evaluation of Pediatric Quality Measures Program: The Experiences of Grantees. This report describes a process evaluation of the Pediatric Quality Measures Program (PQMP), focusing exclusively on the operations of Centers of Excellence (COEs) and the experiences of Principle Investigators (PIs) in leading them. Since the grantees are closest to the process of quality measures development and improvement, they can provide unique perspectives and valuable information that can constitute an important part of the overall evaluation of the program. The final product includes a description of the operations (inputs, activities, and outputs) of the program, and, where appropriate, an analysis of the extent to which they match the original program design. This process evaluation identifies successes of PQMP to date according to the PIs, and it also discusses methodological and administrative issues that the COEs have faced throughout the duration of PQMP. As PQMP is still ongoing, it aims to generate information and feedback that ensures program implementation is proceeding as planned and suggests improvements in program operations. It also discusses future challenges that are of significant concern to the grantees as the program nears its completion and enters the final year of funding. This timely feedback will be helpful in improving both the theoretical and operational design of PQMP in an increasingly dynamic and unpredictable political environment. Because PQMP is the largest and the only grant-funded national effort that focuses solely on pediatric quality measures improvement and development, the process evaluation also provides lessons and recommendations that could be useful in future quality measurement opportunities.


15. Quality Assurance Evaluation of Domestic Violence Sites� Data Collection Methods. This project was conducted under the Department of Health and Human Services (DHHS) through the Family Violence Prevention Services Program (FVPSP), which administers the Family Violence Prevention and Services Act (FVPSA). FVPSA provides funding for emergency shelter and related services for domestic violence victims and their children by administering grants to state domestic violence coalitions. Each coalition then distributes a specified amount of funding to each of their qualifying domestic violence program sites. This project sought to evaluate the accuracy of data collection and reporting methods at program sites that receive these funds throughout the nation through interviews with state administrators and program site employees in five states. The project also investigated how entities at both the state and local level reported their numbers on their performance progress report (PPR), a document that synthesizes each state�s program�s functions into one report.

Paper Presentation

16. State Advisory Councils on Early Childhood Education and Care Grant Implementation Evaluation. The creation of the State Advisory Councils on Early Childhood Education and Care (SAC) grant began with the Improving Head Start Readiness Act of 2007, and was funded for 100 million dollars with the passage of the American Recovery and Reinvestment Act (ARRA) of 2009. These stimulus funds gave states the resources for three years to assess early childhood education and care needs in their state and to make recommendations on how to improve the coordination, quality, and access to their early care and education system. This evaluation examined the implementation of this grant to determine how successful states were at meeting its requirements, first on the national level and then on the state level with closer examinations of the three Councils in Connecticut, Georgia, and Arizona.

Paper Presentation
U.S. Government Accountability Office

17. Pathways to Prosperity: How Career Pathways Approaches are Affecting Educational & Economic Opportunities. This analysis was a research synthesis for the Government Accountability Office on the career pathways approach. The career pathways approach is a response to employer demand for a more skilled and educated workforce and an opportunity to help lower-skilled workers gain economic security and advancement. Most of the current research focuses on how to develop quality career pathways services and programs. Many states are driving work to develop frameworks, toolkits, and metrics that fulfill this goal as part of initiatives like CLASP�s Alliance for Quality Career Pathways and the Joyce Foundation�s Shifting Gears project. Preliminary results from such work in states demonstrate promise for improving the educational foundation and skills of the U.S. workforce. Washington state�s bridge program called I-BEST is already being replicated around the country.

Paper Presentation

Capital Partners for Education

18. Strategies for Increasing Alumni Engagement. The purpose of this project was to develop a plan for Capital Partners for Education (CPE) to increase alumni engagement in order to increase alumni donations to and involvement with CPE. The project involved conducting interviews with people at other educational organizations who are experienced with alumni engagement, and through reviewing academic research on alumni engagement.

Paper Presentation
Folger Shakespeare Library

19. Reaching Teachers: Training Educators to Teach Shakespeare. This report begins by describing the Folger Shakespeare Library�s current structure, including administration and funding. Then, it provides a literature review that provides basic context on the landscape of Shakespeare education, the importance of Shakespeare, the history of Shakespeare education, the effectiveness of performance-based Shakespeare education, and the Folger Library�s philosophy of education. This is followed by a summary of the needs assessment process and results. Finally, the report recommends a logic model for a basic new program design based on the needs assessment results.

Hilltop Institute

20. Preventing Functional Decline in the Hospitalized Elderly: A Cost-Benefit Analysis. The goal of this project is to determine what investments hospitals themselves can make to improve their own financial stability in light of changing incentives from the State of Maryland. The project focused specifically on how hospitals can prevent functional decline. Based on my analysis, it appears that investing in preserving the personal autonomy of the elderly while they are in the hospital can be financially beneficial for hospitals. That being said, if providers are not careful, they could wind up making an unwise investment that worsens their financial well being. The two programs that offered the greatest return were the ACE Units and the Delirium Education program. The two programs unworthy of investment were the HELP program and the Multi-Component Falls prevention program. The remaining programs showed moderate returns that were still well above the break-even point.

Paper Presentation
Magnet Schools of America

21. An Analysis on Magnet Schools in LAUSD. As an important public school choice option, magnet schools are perceived not only as an effort on public school desegregation, but also as high quality education institutions. High student academic achievement is one of the key factors that attract students from various backgrounds. An essential question to ask is whether magnet schools have higher academic performance than regular public schools. This study tries to answer the question with a focus on the Los Angeles Unified School District. The study finds that magnet schools educate a more socioeconomically diverse student body than regular public schools do; the student body in magnet schools is also more racially diverse; at elementary and middle school levels, magnet schools have significantly higher academic achievement when controlling for other school characteristics; at the high school level, magnet schools on average have higher academic achievement than regular public schools, but the difference is not statistically significant.


Back to top