The Art Of Feature Engineering

A practical guide for data scientists who want to improve the performance of any machine learning solution with feature engineering.

The process of developing predictive models includes many stages. Most resources focus on the modeling algorithms but neglect other critical aspects of the modeling process. This book describes techniques for finding the best representations of predictors for modeling and for nding the best subset of predictors for improving model performance. A variety of example data sets are used to illustrate the techniques along with R programs for reproducing the results.

Deliver huge improvements to your machine learning pipelines without spending hours fine-tuning parameters! This book’s practical case-studies reveal feature engineering techniques that upgrade your data wrangling—and your ML results. In Feature Engineering Bookcamp you will learn how to: Identify and implement feature transformations for your data Build powerful machine learning pipelines with unstructured data like text and images Quantify and minimize bias in machine learning pipelines at the data level Use feature stores to build real-time feature engineering pipelines Enhance existing machine learning pipelines by manipulating the input data Use state-of-the-art deep learning models to extract hidden patterns in data Feature Engineering Bookcamp guides you through a collection of projects that give you hands-on practice with core feature engineering techniques. You’ll work with feature engineering practices that speed up the time it takes to process data and deliver real improvements in your model’s performance. This instantly-useful book skips the abstract mathematical theory and minutely-detailed formulas; instead you’ll learn through interesting code-driven case studies, including tweet classification, COVID detection, recidivism prediction, stock price movement detection, and more. About the technology Get better output from machine learning pipelines by improving your training data! Use feature engineering, a machine learning technique for designing relevant input variables based on your existing data, to simplify training and enhance model performance. While fine-tuning hyperparameters or tweaking models may give you a minor performance bump, feature engineering delivers dramatic improvements by transforming your data pipeline. About the book Feature Engineering Bookcamp walks you through six hands-on projects where you’ll learn to upgrade your training data using feature engineering. Each chapter explores a new code-driven case study, taken from real-world industries like finance and healthcare. You’ll practice cleaning and transforming data, mitigating bias, and more. The book is full of performance-enhancing tips for all major ML subdomains—from natural language processing to time-series analysis. What's inside Identify and implement feature transformations Build machine learning pipelines with unstructured data Quantify and minimize bias in ML pipelines Use feature stores to build real-time feature engineering pipelines Enhance existing pipelines by manipulating input data About the reader For experienced machine learning engineers familiar with Python. About the author Sinan Ozdemir is the founder and CTO of Shiba, a former lecturer of Data Science at Johns Hopkins University, and the author of multiple textbooks on data science and machine learning. Table of Contents 1 Introduction to feature engineering 2 The basics of feature engineering 3 Healthcare: Diagnosing COVID-19 4 Bias and fairness: Modeling recidivism 5 Natural language processing: Classifying social media sentiment 6 Computer vision: Object recognition 7 Time series analysis: Day trading with machine learning 8 Feature stores 9 Putting it all together

Feature engineering is a crucial step in the machine-learning pipeline, yet this topic is rarely examined on its own. With this practical book, you’ll learn techniques for extracting and transforming features—the numeric representations of raw data—into formats for machine-learning models. Each chapter guides you through a single data problem, such as how to represent text or image data. Together, these examples illustrate the main principles of feature engineering. Rather than simply teach these principles, authors Alice Zheng and Amanda Casari focus on practical application with exercises throughout the book. The closing chapter brings everything together by tackling a real-world, structured dataset with several feature-engineering techniques. Python packages including numpy, Pandas, Scikit-learn, and Matplotlib are used in code examples. You’ll examine: Feature engineering for numeric data: filtering, binning, scaling, log transforms, and power transforms Natural text techniques: bag-of-words, n-grams, and phrase detection Frequency-based filtering and feature scaling for eliminating uninformative features Encoding techniques of categorical variables, including feature hashing and bin-counting Model-based feature engineering with principal component analysis The concept of model stacking, using k-means as a featurization technique Image feature extraction with manual and deep-learning techniques

Extract accurate information from data to train and improve machine learning models using NumPy, SciPy, pandas, and scikit-learn libraries Key FeaturesDiscover solutions for feature generation, feature extraction, and feature selectionUncover the end-to-end feature engineering process across continuous, discrete, and unstructured datasetsImplement modern feature extraction techniques using Python's pandas, scikit-learn, SciPy and NumPy librariesBook Description Feature engineering is invaluable for developing and enriching your machine learning models. In this cookbook, you will work with the best tools to streamline your feature engineering pipelines and techniques and simplify and improve the quality of your code. Using Python libraries such as pandas, scikit-learn, Featuretools, and Feature-engine, you’ll learn how to work with both continuous and discrete datasets and be able to transform features from unstructured datasets. You will develop the skills necessary to select the best features as well as the most suitable extraction techniques. This book will cover Python recipes that will help you automate feature engineering to simplify complex processes. You’ll also get to grips with different feature engineering strategies, such as the box-cox transform, power transform, and log transform across machine learning, reinforcement learning, and natural language processing (NLP) domains. By the end of this book, you’ll have discovered tips and practical solutions to all of your feature engineering problems. What you will learnSimplify your feature engineering pipelines with powerful Python packagesGet to grips with imputing missing valuesEncode categorical variables with a wide set of techniquesExtract insights from text quickly and effortlesslyDevelop features from transactional data and time series dataDerive new features by combining existing variablesUnderstand how to transform, discretize, and scale your variablesCreate informative variables from date and timeWho this book is for This book is for machine learning professionals, AI engineers, data scientists, and NLP and reinforcement learning engineers who want to optimize and enrich their machine learning models with the best features. Knowledge of machine learning and Python coding will assist you with understanding the concepts covered in this book.

There is broad interest in feature extraction, construction, and selection among practitioners from statistics, pattern recognition, and data mining to machine learning. Data preprocessing is an essential step in the knowledge discovery process for real-world applications. This book compiles contributions from many leading and active researchers in this growing field and paints a picture of the state-of-art techniques that can boost the capabilities of many existing data mining tools. The objective of this collection is to increase the awareness of the data mining community about the research of feature extraction, construction and selection, which are currently conducted mainly in isolation. This book is part of our endeavor to produce a contemporary overview of modern solutions, to create synergy among these seemingly different branches, and to pave the way for developing meta-systems and novel approaches. Even with today's advanced computer technologies, discovering knowledge from data can still be fiendishly hard due to the characteristics of the computer generated data. Feature extraction, construction and selection are a set of techniques that transform and simplify data so as to make data mining tasks easier. Feature construction and selection can be viewed as two sides of the representation problem.

Machine learning has become an integral part of many commercial applications and research projects, but this field is not exclusive to large companies with extensive research teams. If you use Python, even as a beginner, this book will teach you practical ways to build your own machine learning solutions. With all the data available today, machine learning applications are limited only by your imagination. You’ll learn the steps necessary to create a successful machine-learning application with Python and the scikit-learn library. Authors Andreas Müller and Sarah Guido focus on the practical aspects of using machine learning algorithms, rather than the math behind them. Familiarity with the NumPy and matplotlib libraries will help you get even more from this book. With this book, you’ll learn: Fundamental concepts and applications of machine learning Advantages and shortcomings of widely used machine learning algorithms How to represent data processed by machine learning, including which data aspects to focus on Advanced methods for model evaluation and parameter tuning The concept of pipelines for chaining models and encapsulating your workflow Methods for working with text data, including text-specific processing techniques Suggestions for improving your machine learning and data science skills

Access to large data sets has led to a paradigm shift in the tourism research landscape. Big data is enabling a new form of knowledge gain, while at the same time shaking the epistemological foundations and requiring new methods and analysis approaches. It allows for interdisciplinary cooperation between computer sciences and social and economic sciences, and complements the traditional research approaches. This book provides a broad basis for the practical application of data science approaches such as machine learning, text mining, social network analysis, and many more, which are essential for interdisciplinary tourism research. Each method is presented in principle, viewed analytically, and its advantages and disadvantages are weighed up and typical fields of application are presented. The correct methodical application is presented with a "how-to" approach, together with code examples, allowing a wider reader base including researchers, practitioners, and students entering the field. The book is a very well-structured introduction to data science – not only in tourism – and its methodological foundations, accompanied by well-chosen practical cases. It underlines an important insight: data are only representations of reality, you need methodological skills and domain background to derive knowledge from them - Hannes Werthner, Vienna University of Technology Roman Egger has accomplished a difficult but necessary task: make clear how data science can practically support and foster travel and tourism research and applications. The book offers a well-taught collection of chapters giving a comprehensive and deep account of AI and data science for tourism - Francesco Ricci, Free University of Bozen-Bolzano This well-structured and easy-to-read book provides a comprehensive overview of data science in tourism. It contributes largely to the methodological repository beyond traditional methods. - Rob Law, University of Macau