Big Data Analytics

The main purpose of this book is to investigate, explore and describe approaches and methods to facilitate data understanding through analytics solutions based on its principles, concepts and applications. But analyzing data is also about involving the use of software. For this, and in order to cover some aspect of data analytics, this book uses software (Excel, SPSS, Python, etc) which can help readers to better understand the analytics process in simple terms and supporting useful methods in its application.

This comprehensive edited 2-volume handbook provides a unique platform for researchers, engineers, developers, educators and advanced students in the field of Big Data analytics. The first volume presents methodologies that support Big Data analytics, while the second volume offers a wide range of Big Data analytics applications.

A handy reference guide for data analysts and data scientists to help to obtain value from big data analytics using Spark on Hadoop clusters About This Book This book is based on the latest 2.0 version of Apache Spark and 2.7 version of Hadoop integrated with most commonly used tools. Learn all Spark stack components including latest topics such as DataFrames, DataSets, GraphFrames, Structured Streaming, DataFrame based ML Pipelines and SparkR. Integrations with frameworks such as HDFS, YARN and tools such as Jupyter, Zeppelin, NiFi, Mahout, HBase Spark Connector, GraphFrames, H2O and Hivemall. Who This Book Is For Though this book is primarily aimed at data analysts and data scientists, it will also help architects, programmers, and practitioners. Knowledge of either Spark or Hadoop would be beneficial. It is assumed that you have basic programming background in Scala, Python, SQL, or R programming with basic Linux experience. Working experience within big data environments is not mandatory. What You Will Learn Find out and implement the tools and techniques of big data analytics using Spark on Hadoop clusters with wide variety of tools used with Spark and Hadoop Understand all the Hadoop and Spark ecosystem components Get to know all the Spark components: Spark Core, Spark SQL, DataFrames, DataSets, Conventional and Structured Streaming, MLLib, ML Pipelines and Graphx See batch and real-time data analytics using Spark Core, Spark SQL, and Conventional and Structured Streaming Get to grips with data science and machine learning using MLLib, ML Pipelines, H2O, Hivemall, Graphx, SparkR and Hivemall. In Detail Big Data Analytics book aims at providing the fundamentals of Apache Spark and Hadoop. All Spark components – Spark Core, Spark SQL, DataFrames, Data sets, Conventional Streaming, Structured Streaming, MLlib, Graphx and Hadoop core components – HDFS, MapReduce and Yarn are explored in greater depth with implementation examples on Spark + Hadoop clusters. It is moving away from MapReduce to Spark. So, advantages of Spark over MapReduce are explained at great depth to reap benefits of in-memory speeds. DataFrames API, Data Sources API and new Data set API are explained for building Big Data analytical applications. Real-time data analytics using Spark Streaming with Apache Kafka and HBase is covered to help building streaming applications. New Structured streaming concept is explained with an IOT (Internet of Things) use case. Machine learning techniques are covered using MLLib, ML Pipelines and SparkR and Graph Analytics are covered with GraphX and GraphFrames components of Spark. Readers will also get an opportunity to get started with web based notebooks such as Jupyter, Apache Zeppelin and data flow tool Apache NiFi to analyze and visualize data. Style and approach This step-by-step pragmatic guide will make life easy no matter what your level of experience. You will deep dive into Apache Spark on Hadoop clusters through ample exciting real-life examples. Practical tutorial explains data science in simple terms to help programmers and data analysts get started with Data Science

Power systems are increasingly collecting large amounts of data due to the expansion of the Internet of Things into power grids. In a smart grids scenario, a huge number of intelligent devices will be connected with almost no human intervention characterizing a machine-to-machine scenario, which is one of the pillars of the Internet of Things. The book characterizes and evaluates how the emerging growth of data in communications networks applied to smart grids will impact the grid efficiency and reliability. Additionally, this book discusses the various security concerns that become manifest with Big Data and expanded communications in power grids. Provide a general description and definition of big data, which has been gaining significant attention in the research community. Introduces a comprehensive overview of big data optimization methods in power system. Reviews the communication devices used in critical infrastructure, especially power systems; security methods available to vet the identity of devices; and general security threats in CI networks. Presents applications in power systems, such as power flow and protection. Reviews electricity theft concerns and the wide variety of data-driven techniques and applications developed for electricity theft detection.

"Big Data Analytics is a field that dissects, efficiently extricates data from, or in any case manages informational indexes that are excessively huge or complex to be managed by customary information preparing application programming. Information with numerous cases (lines) offers more noteworthy factual force, while information with higher multifaceted nature may prompt a higher bogus disclosure rate. Enormous information challenges incorporate catching information, information stockpiling, information investigation, search, sharing, move, representation, and questioning, refreshing, data security and data source. Large information was initially connected with three key ideas: volume, variety and velocity. Consequently, huge information regularly incorporates information with sizes that surpass the limit of conventional programming to measure inside a satisfactory time and worth. Current utilization of the term enormous information will in general allude to the utilization of predictive analytics, user behavior analytics, or certain other progressed information investigation techniques that concentrate an incentive from information, and sometimes to a specific size of informational index. There is little uncertainty that the amounts of information now accessible are undoubtedly enormous, however that is not the most important quality of this new information biological system. Investigation of informational indexes can discover new relationships to spot business patterns or models. Researchers, business persons, clinical specialists, promoting and governments consistently meet challenges with huge informational collections in territories including Internet look, fintech, metropolitan informatics, and business informatics. Researchers experience constraints in e-Science work, including meteorology, genomics, connectomics, complex material science reproductions, science and ecological exploration. The main objective of this book is to write about issues, challenges, opportunities, and solutions in novel research projects about big data in various domains. The topics of interest include, but are not limited to: efficient storage, management and sharing large scale of data; novel approaches for analyzing data using big data technologies; implementation of high performance and/or scalable and/or real-time computation algorithms for analyzing big data; usage of various data sources like historical data, social networking media, machine data and crowd-sourcing data; using machine learning, visual analytics, data mining, spatio-temporal data analysis and statistical inference in different domains (with large scale datasets); Legal and ethical issues and solutions for using, sharing and publishing large datasets; and the results of data analytics, security and privacy issues"--

Data Science and Big Data Analytics is about harnessing the power of data for new insights. The book covers the breadth of activities and methods and tools that Data Scientists use. The content focuses on concepts, principles and practical applications that are applicable to any industry and technology environment, and the learning is supported and explained with examples that you can replicate using open-source software. This book will help you: Become a contributor on a data science team Deploy a structured lifecycle approach to data analytics problems Apply appropriate analytic techniques and tools to analyzing big data Learn how to tell a compelling story with data to drive business action Prepare for EMC Proven Professional Data Science Certification Get started discovering, analyzing, visualizing, and presenting data in a meaningful way today!

This timely text/reference reviews the state of the art of big data analytics, with a particular focus on practical applications. An authoritative selection of leading international researchers present detailed analyses of existing trends for storing and analyzing big data, together with valuable insights into the challenges inherent in current approaches and systems. This is further supported by real-world examples drawn from a broad range of application areas, including healthcare, education, and disaster management. The text also covers, typically from an application-oriented perspective, advances in data science in such areas as big data collection, searching, analysis, and knowledge discovery. Topics and features: Discusses a model for data traffic aggregation in 5G cellular networks, and a novel scheme for resource allocation in 5G networks with network slicing Explores methods that use big data in the assessment of flood risks, and apply neural networks techniques to monitor the safety of nuclear power plants Describes a system which leverages big data analytics and the Internet of Things in the application of drones to aid victims in disaster scenarios Proposes a novel deep learning-based health data analytics application for sleep apnea detection, and a novel pathway for diagnostic models of headache disorders Reviews techniques for educational data mining and learning analytics, and introduces a scalable MapReduce graph partitioning approach for high degree vertices Presents a multivariate and dynamic data representation model for the visualization of healthcare data, and big data analytics methods for software reliability assessment This practically-focused volume is an invaluable resource for all researchers, academics, data scientists and business professionals involved in the planning, designing, and implementation of big data analytics projects. Dr. Mohammed M. Alani is an Associate Professor in Computer Engineering and currently is the Provost at Al Khawarizmi International College, Abu Dhabi, UAE. Dr. Hissam Tawfik is a Professor of Computer Science in the School of Computing, Creative Technologies & Engineering at Leeds Beckett University, UK. Dr. Mohammed Saeed is a Professor in Computing and currently is the Vice President for Academic Affairs and Research at the University of Modern Sciences, Dubai, UAE. Dr. Obinna Anya is a Research Staff Member at IBM Research – Almaden, San Jose, CA, USA.

This book has a collection of articles written by Big Data experts to describe some of the cutting-edge methods and applications from their respective areas of interest, and provides the reader with a detailed overview of the field of Big Data Analytics as it is practiced today. The chapters cover technical aspects of key areas that generate and use Big Data such as management and finance; medicine and healthcare; genome, cytome and microbiome; graphs and networks; Internet of Things; Big Data standards; bench-marking of systems; and others. In addition to different applications, key algorithmic approaches such as graph partitioning, clustering and finite mixture modelling of high-dimensional data are also covered. The varied collection of themes in this volume introduces the reader to the richness of the emerging field of Big Data Analytics.