Bad Data

Highlights the pitfalls of data analysis and emphasizes the importance of using the appropriate metrics before making key decisions.Big data is often touted as the key to understanding almost every aspect of contemporary life. This critique of "information hubris" shows that even more important than data is finding the right metrics to evaluate it.The author, an expert in environmental design and city planning, examines the many ways in which we measure ourselves and our world. He dissects the metrics we apply to health, worker productivity, our children's education, the quality of our environment, the effectiveness of leaders, the dynamics of the economy, and the overall well-being of the planet. Among the areas where the wrong metrics have led to poor outcomes, he cites the fee-for-service model of health care, corporate cultures that emphasize time spent on the job while overlooking key productivity measures, overreliance on standardized testing in education to the detriment of authentic learning, and a blinkered focus on carbon emissions, which underestimates the impact of industrial damage to our natural world. He also examines various communities and systems that have achieved better outcomes by adjusting the ways in which they measure data. The best results are attained by those that have learned not only what to measure and how to measure it, but what it all means. By highlighting the pitfalls inherent in data analysis, this illuminating book reminds us that not everything that can be counted really counts.

What is bad data? Some people consider it a technical phenomenon, like missing values or malformed records, but bad data includes a lot more. In this handbook, data expert Q. Ethan McCallum has gathered 19 colleagues from every corner of the data arena to reveal how they’ve recovered from nasty data problems. From cranky storage to poor representation to misguided policy, there are many paths to bad data. Bottom line? Bad data is data that gets in the way. This book explains effective ways to get around it. Among the many topics covered, you’ll discover how to: Test drive your data to see if it’s ready for analysis Work spreadsheet data into a usable form Handle encoding problems that lurk in text data Develop a successful web-scraping effort Use NLP tools to reveal the real sentiment of online reviews Address cloud computing issues that can impact your analysis effort Avoid policies that create data analysis roadblocks Take a systematic approach to data quality analysis

This monograph is a contribution to the study of the identification problem: the problem of identifying an item from a known class us ing positive and negative examples. This problem is considered to be an important component of the process of inductive learning, and as such has been studied extensively. In the overview we shall explain the objectives of this work and its place in the overall fabric of learning research. Context. Learning occurs in many forms; the only form we are treat ing here is inductive learning, roughly characterized as the process of forming general concepts from specific examples. Computer Science has found three basic approaches to this problem: • Select a specific learning task, possibly part of a larger task, and construct a computer program to solve that task . • Study cognitive models of learning in humans and extrapolate from them general principles to explain learning behavior. Then construct machine programs to test and illustrate these models. xi Xll PREFACE • Formulate a mathematical theory to capture key features of the induction process. This work belongs to the third category. The various studies of learning utilize training examples (data) in different ways. The three principal ones are: • Similarity-based (or empirical) learning, in which a collection of examples is used to select an explanation from a class of possible rules.

Scientific progress depends on good research, and good research needs good statistics. But statistical analysis is tricky to get right, even for the best and brightest of us. You'd be surprised how many scientists are doing it wrong. Statistics Done Wrong is a pithy, essential guide to statistical blunders in modern science that will show you how to keep your research blunder-free. You'll examine embarrassing errors and omissions in recent research, learn about the misconceptions and scientific politics that allow these mistakes to happen, and begin your quest to reform the way you and your peers do statistics. You'll find advice on: –Asking the right question, designing the right experiment, choosing the right statistical analysis, and sticking to the plan –How to think about p values, significance, insignificance, confidence intervals, and regression –Choosing the right sample size and avoiding false positives –Reporting your analysis and publishing your data and source code –Procedures to follow, precautions to take, and analytical software that can help Scientists: Read this concise, powerful guide to help you produce statistically sound research. Statisticians: Give this book to everyone you know. The first step toward statistics done right is Statistics Done Wrong.

Proceedings of the Ninth Power Systems Computation Conference

This book presents innovative ideas, cutting-edge findings, and novel techniques, methods, and applications in a broad range of cybersecurity and cyberthreat intelligence areas. As our society becomes smarter, there is a corresponding need to secure our cyberfuture. The book describes approaches and findings that are of interest to business professionals and governments seeking to secure our data and underpin infrastructures, as well as to individual users.

Smart grid technologies include sensing and measurement technologies, advanced components aided with communications and control methods along with improved interfaces and decision support systems. Smart grid techniques support the extensive inclusion of clean renewable generation in power systems. Smart grid use also promotes energy saving in power systems. Cyber security objectives for the smart grid are availability, integrity and confidentiality. Five salient features of this book are as follows: AI and IoT in improving resilience of smart energy infrastructure IoT, smart grids and renewable energy: an economic approach AI and ML towards sustainable solar energy Electrical vehicles and smart grid Intelligent condition monitoring for solar and wind energy systems