Evaluating Federal Research Programs

The Government Performance and Results Act (GPRA), passed by Congress in 1993, requires that federal agencies write five-year strategic plans with annual performance goals and produce an annual report that demonstrates whether the goals have been met. The first performance reports are due in March 2000. Measuring the performance of basic research is particularly challenging because major breakthroughs can be unpredictable and difficult to assess in the short term. This book recommends that federal agencies use an "expert review" method to examine the quality of research they support, the relevance of that research to their mission, and whether the research is at the international forefront of scientific and technological knowledge. It also addresses the issues of matching evaluation measurements to the character of the research performed, improving coordination among agencies when research is in the same field, and including a human resource development component in GPRA strategic and performance plans.

The Government Performance and Results Act (GPRA), passed by Congress in 1993, requires that federal agencies write five-year strategic plans with annual performance goals and produce an annual report that demonstrates whether the goals have been met. The first performance reports are due in March 2000. Measuring the performance of basic research is particularly challenging because major breakthroughs can be unpredictable and difficult to assess in the short term. This book recommends that federal agencies use an "expert review" method to examine the quality of research they support, the relevance of that research to their mission, and whether the research is at the international forefront of scientific and technological knowledge. It also addresses the issues of matching evaluation measurements to the character of the research performed, improving coordination among agencies when research is in the same field, and including a human resource development component in GPRA strategic and performance plans.

As requested by Congress and the White House Office of Science and Technology Policy (OSTP), this report assists federal agencies in crafting plans and reports that are responsive to the Government Performance and Results Act (GPRA), OMB Guidance, and agency missions. Using a case study approach, the report identifies best practices used by individual agencies to evaluate the performance and results of their science and technology programs. The report takes into account individual agencies' missions and how science and technology programs and human resource needs are factored into agency GPRA plans. Specific applications of recommendations are included from COSEPUP's earlier report entitled Evaluating Federal Research Programs: Research and the Government Performance and Results Act.

The medical research landscape in the United States is supported by a variety of organizations that spend billions of dollars in government and private funds each year to seek answers to complex medical and public health problems. The largest government funder is the National Institutes of Health (NIH), followed by the Department of Defense (DoD). Almost half of DoD's medical research funding is administered by the Congressionally Directed Medical Research Programs (CDMRP). The mission of CDMRP is to foster innovative approaches to medical research in response to the needs of its stakeholdersâ€"the U.S. military, their families, the American public, and Congress. CDMRP funds medical research to be performed by other government and nongovernmental organizations, but it does not conduct research itself. The major focus of CDMRP funded research is the improved prevention, diagnosis, and treatment of diseases, injuries, or conditions that affect service members and their families, and the general public. The hallmarks of CDMRP include reviewing applications for research funding using a two-tiered review process, and involving consumers throughout the process. Evaluation of the Congressionally Directed Medical Research Programs Review Process evaluates the CDMRP two-tiered peer review process, its coordination of research priorities with NIH and the Department of Veterans Affairs, and provides recommendations on how the process for reviewing and selecting studies can be improved.

Each year, approximately 5,000 fatal work-related injuries and 4 million non-fatal injuries and illnesses occur in the United States. This number represents both unnecessary human suffering and high economic costs. In order to assist in better evaluating workplace safety and create safer work environments, the Institute of Medicine conducted a series of evaluations of the National Institute for Occupational Safety and Health (NIOSH) research programs, assessing the relevance and impact of NIOSH's work on improving worker safety and health.

As social action programs in health, education, and welfare have expanded, interest has grown in evaluating their implementation and effectiveness. Policymakers and social planners--at all levels of government and in the private sector--are currently confronted with the problem of evaluating the large number of human service programs that compete for available resources. Academic and Entrepreneurial Research presents a systematic study of the expenditure of federal funds for evaluation research. It reviews federally-supported evaluations of programs, including evaluations of social change experiments and research-demonstration programs funded by the various executive departments of the federal government. Evaluation studies of these large-scale programs vary in scope, quality, and potential utility. Bernstein and Freeman examine all projects initiated during fiscal year 1970 in order to understand better the methods employed, the types of persons engaged in such research, and expectations regarding the utilization of findings. The book provides data about "high" and "low" quality evaluation research and contains recommendations for restructuring the entire evaluation research enterprise in light of the findings.

USA. Research results of a study of the federal system for the evaluation of social policy programmes - distinguishes four types of evaluation, viz. Programme impact, programme strategy, project evaluation and project rating, covers administrative aspects, organizational relationships between national level and local level, financial aspects and personneling, evaluation techniques, etc., and includes recommendations. Bibliography pp. 121 to 134.