Informing Research Choices Indicators And Judgment

"The Council of Canadian Academies (the Council) has, since 2006, completed seven expert panel assessments analyzing in great depth Canada's performance in science and technology (S&T) and innovation. This document synthesizes the main findings of that work, from which two main conclusions emerge: Canadian academic research, overall, is strong and well regarded internationally; Canadian business innovation, by contrast, is weak by international standards, and this is the primary cause of Canada's poor productivity growth"--Executive summary.

Computing and information and communications technology (ICT) has dramatically changed how we work and live, has had profound effects on nearly every sector of society, has transformed whole industries, and is a key component of U.S. global leadership. A fundamental driver of advances in computing and ICT has been the fact that the single-processor performance has, until recently, been steadily and dramatically increasing year over years, based on a combination of architectural techniques, semiconductor advances, and software improvements. Users, developers, and innovators were able to depend on those increases, translating that performance into numerous technological innovations and creating successive generations of ever more rich and diverse products, software services, and applications that had profound effects across all sectors of society. However, we can no longer depend on those extraordinary advances in single-processor performance continuing. This slowdown in the growth of single-processor computing performance has its roots in fundamental physics and engineering constraints-multiple technological barriers have converged to pose deep research challenges, and the consequences of this shift are deep and profound for computing and for the sectors of the economy that depend on and assume, implicitly or explicitly, ever-increasing performance. From a technology standpoint, these challenges have led to heterogeneous multicore chips and a shift to alternate innovation axes that include, but are not limited to, improving chip performance, mobile devices, and cloud services. As these technical shifts reshape the computing industry, with global consequences, the United States must be prepared to exploit new opportunities and to deal with technical challenges. The New Global Ecosystem in Advanced Computing: Implications for U.S. Competitiveness and National Security outlines the technical challenges, describe the global research landscape, and explore implications for competition and national security.

‘Represents the culmination of an 18-month-long project that aims to be the definitive review of this important topic. Accompanied by a scholarly literature review, some new analysis, and a wealth of evidence and insight... the report is a tour de force; a once-in-a-generation opportunity to take stock.’ – Dr Steven Hill, Head of Policy, HEFCE, LSE Impact of Social Sciences Blog ‘A must-read if you are interested in having a deeper understanding of research culture, management issues and the range of information we have on this field. It should be disseminated and discussed within institutions, disciplines and other sites of research collaboration.’ – Dr Meera Sabaratnam, Lecturer in International Relations at the School of Oriental and African Studies, University of London, LSE Impact of Social Sciences Blog Metrics evoke a mixed reaction from the research community. A commitment to using data and evidence to inform decisions makes many of us sympathetic, even enthusiastic, about the prospect of granular, real-time analysis of our own activities. Yet we only have to look around us at the blunt use of metrics to be reminded of the pitfalls. Metrics hold real power: they are constitutive of values, identities and livelihoods. How to exercise that power to positive ends is the focus of this book. Using extensive evidence-gathering, analysis and consultation, the authors take a thorough look at potential uses and limitations of research metrics and indicators. They explore the use of metrics across different disciplines, assess their potential contribution to the development of research excellence and impact and consider the changing ways in which universities are using quantitative indicators in their management systems. Finally, they consider the negative or unintended effects of metrics on various aspects of research culture. Including an updated introduction from James Wilsdon, the book proposes a framework for responsible metrics and makes a series of targeted recommendations to show how responsible metrics can be applied in research management, by funders, and in the next cycle of the Research Excellence Framework. The metric tide is certainly rising. Unlike King Canute, we have the agency and opportunity – and in this book, a serious body of evidence – to influence how it washes through higher education and research.

Once treated as the absence of knowledge, ignorance today has become a highly influential topic in its own right, commanding growing attention across the natural and social sciences where a wide range of scholars have begun to explore the social life and political issues involved in the distribution and strategic use of not knowing. The field is growing fast and this handbook reflects this interdisciplinary field of study by drawing contributions from economics, sociology, history, philosophy, cultural studies, anthropology, feminist studies, and related fields in order to serve as a seminal guide to the political, legal and social uses of ignorance in social and political life. Chapter 33 of this book is freely available as a downloadable Open Access PDF under a Creative Commons Attribution-Non Commercial-No Derivatives 4.0 license available here: https://tandfbis.s3-us-west-2.amazonaws.com/rt-files/docs/Open+Access+Chapters/9780415718967_oachapter33.pdf

Why bibliometrics is useful for understanding the global dynamics of science but generate perverse effects when applied inappropriately in research evaluation and university rankings. The research evaluation market is booming. “Ranking,” “metrics,” “h-index,” and “impact factors” are reigning buzzwords. Government and research administrators want to evaluate everything—teachers, professors, training programs, universities—using quantitative indicators. Among the tools used to measure “research excellence,” bibliometrics—aggregate data on publications and citations—has become dominant. Bibliometrics is hailed as an “objective” measure of research quality, a quantitative measure more useful than “subjective” and intuitive evaluation methods such as peer review that have been used since scientific papers were first published in the seventeenth century. In this book, Yves Gingras offers a spirited argument against an unquestioning reliance on bibliometrics as an indicator of research quality. Gingras shows that bibliometric rankings have no real scientific validity, rarely measuring what they pretend to. Although the study of publication and citation patterns, at the proper scales, can yield insights on the global dynamics of science over time, ill-defined quantitative indicators often generate perverse and unintended effects on the direction of research. Moreover, abuse of bibliometrics occurs when data is manipulated to boost rankings. Gingras looks at the politics of evaluation and argues that using numbers can be a way to control scientists and diminish their autonomy in the evaluation process. Proposing precise criteria for establishing the validity of indicators at a given scale of analysis, Gingras questions why universities are so eager to let invalid indicators influence their research strategy.

A comprehensive, state-of-the-art examination of the changing ways we measure scholarly performance and research impact. Bibliometrics has moved well beyond the mere tracking of bibliographic citations. The web enables new ways to measure scholarly productivity and impact, making available tools and data that can reveal patterns of intellectual activity and impact that were previously invisible: mentions, acknowledgments, endorsements, downloads, recommendations, blog posts, tweets. This book describes recent theoretical and practical advances in metrics-based research, examining a variety of alternative metrics—or “altmetrics”—while also considering the ethical and cultural consequences of relying on metrics to assess the quality of scholarship. Once the domain of information scientists and mathematicians, bibliometrics is now a fast-growing, multidisciplinary field that ranges from webometrics to scientometrics to influmetrics. The contributors to Beyond Bibliometrics discuss the changing environment of scholarly publishing, the effects of open access and Web 2.0 on genres of discourse, novel analytic methods, and the emergence of next-generation metrics in a performance-conscious age. Contributors Mayur Amin, Judit Bar-Ilan, Johann Bauer, Lutz Bornmann, Benjamin F. Bowman, Kevin W. Boyack, Blaise Cronin, Ronald Day, Nicola De Bellis, Jonathan Furner, Yves Gingras, Stefanie Haustein, Edwin Henneken, Peter A. Hook, Judith Kamalski, Richard Klavans, Kayvan Kousha, Michael Kurtz, Mark Largent, Julia Lane, Vincent Larivière, Loet Leydesdorff, Werner Marx, Katherine W. McCain, Margit Palzenberger, Andrew Plume, Jason Priem, Rebecca Rosen, Hermann Schier, Hadas Shema, Cassidy R. Sugimoto, Mike Thelwall, Daril Vilhena, Jevin West, Paul Wouters