Building The Agile Database

Is fast development the enemy of good development? Not necessarily. Agile development requires that databases are designed and built quickly enough to meet fast-based delivery schedules — but in a way that also delivers maximum business value and reuse. How can these requirements both be satisfied? This book, suitable for practitioners at all levels, will explain how to design and build enterprise-quality high-value databases within the constraints of an Agile project. Starting with an overview of the business case for good data management practices, the book defines the various stakeholder groups involved in the software development process, explains the economics of software development (including “time to market” vs. “time to money”), and describes an approach to Agile database development based on the five PRISM principles. This book explains how to work with application developers and other stakeholders, examines critical issues in Agile Development and Data Management, and describes how developers and data professionals can work together to make Agile projects successful while delivering maximum value data to the enterprise. Building the Agile Database will serve as an excellent reference for application developers, data managers, DBAs, project managers, Scrum Masters and IT managers looking to get more value from their development efforts. Among the topics covered: 1. Why Agile is more than just the latest development fad 2. The critical distinction between the logical and physical views of data 3. The importance of data virtualization, and how to achieve it 4. How to eliminate the “object-relational impedance mismatch” 5. The difference between logical modeling and physical design 6. Why databases are more than “persistence engines” 7. When and how to do logical modeling and physical design 8. Use of the logical data model in model-driven development 9. Refactoring made easier 10. Developing an “Agile Attitude”

Describes Agile Modeling Driven Design (AMDD) and Test-Driven Design (TDD) approaches, database refactoring, database encapsulation strategies, and tools that support evolutionary techniques Agile software developers often use object and relational database (RDB) technology together and as a result must overcome the impedance mismatch The author covers techniques for mapping objects to RDBs and for implementing concurrency control, referential integrity, shared business logic, security access control, reports, and XML An agile foundation describes fundamental skills that all agile software developers require, particularly Agile DBAs Includes object modeling, UML data modeling, data normalization, class normalization, and how to deal with legacy databases Scott W. Ambler is author of Agile Modeling (0471202827), a contributing editor with Software Development (www.sdmagazine.com), and a featured speaker at software conferences worldwide

There are many database books; what's special about this one? This book doesn't patronize. There are some technical things to learn about relational databases if you want to use them effectively and this book does not skimp on those. We cover them in detail with practical examples that you can work through. You are not a dummy; you are smart and you are keen to learn. This book doesn't blind you with science. The theory of relational databases can be described in mathematical terms. This is good, but it is not the way people talk about relational databases in projects out there in the business world, in the public sector and in charities. This book uses plain English to describe the concepts and why they are important. This book will help you to communicate with users and colleagues to build great database applications. This book isn't tied to any particular vendor. it discusses all the leading databases, highlighting their similarities and their differences so that you can implement your applications whatever relational database you are using. This book isn't bogged down in bureaucracy. We are enthusiastic about real agile development and we know how to build database applications in an agile team because we have done it, successfully, several times. We do care about doing the job well, testing thoroughly, documenting what needs to be documented, managing frequent releases, but we do not care for the three-inch-thick up-front specification. We have learned some things that make applications successful and some that lead to failure. We share these with you so that you get the best possible opportunity to succeed. We enjoy building database applications and we hope you will too.

Data science teams looking to turn research into useful analytics applications require not only the right tools, but also the right approach if they’re to succeed. With the revised second edition of this hands-on guide, up-and-coming data scientists will learn how to use the Agile Data Science development methodology to build data applications with Python, Apache Spark, Kafka, and other tools. Author Russell Jurney demonstrates how to compose a data platform for building, deploying, and refining analytics applications with Apache Kafka, MongoDB, ElasticSearch, d3.js, scikit-learn, and Apache Airflow. You’ll learn an iterative approach that lets you quickly change the kind of analysis you’re doing, depending on what the data is telling you. Publish data science work as a web application, and affect meaningful change in your organization. Build value from your data in a series of agile sprints, using the data-value pyramid Extract features for statistical models from a single dataset Visualize data with charts, and expose different aspects through interactive reports Use historical data to predict the future via classification and regression Translate predictions into actions Get feedback from users after each sprint to keep your project on track

Agile Data Warehouse Design is a step-by-step guide for capturing data warehousing/business intelligence (DW/BI) requirements and turning them into high performance dimensional models in the most direct way: by modelstorming (data modeling + brainstorming) with BI stakeholders. This book describes BEAM✲, an agile approach to dimensional modeling, for improving communication between data warehouse designers, BI stakeholders and the whole DW/BI development team. BEAM✲ provides tools and techniques that will encourage DW/BI designers and developers to move away from their keyboards and entity relationship based tools and model interactively with their colleagues. The result is everyone thinks dimensionally from the outset! Developers understand how to efficiently implement dimensional modeling solutions. Business stakeholders feel ownership of the data warehouse they have created, and can already imagine how they will use it to answer their business questions. Within this book, you will learn: ✲ Agile dimensional modeling using Business Event Analysis & Modeling (BEAM✲) ✲ Modelstorming: data modeling that is quicker, more inclusive, more productive, and frankly more fun! ✲ Telling dimensional data stories using the 7Ws (who, what, when, where, how many, why and how) ✲ Modeling by example not abstraction; using data story themes, not crow's feet, to describe detail ✲ Storyboarding the data warehouse to discover conformed dimensions and plan iterative development ✲ Visual modeling: sketching timelines, charts and grids to model complex process measurement - simply ✲ Agile design documentation: enhancing star schemas with BEAM✲ dimensional shorthand notation ✲ Solving difficult DW/BI performance and usability problems with proven dimensional design patterns Lawrence Corr is a data warehouse designer and educator. As Principal of DecisionOne Consulting, he helps clients to review and simplify their data warehouse designs, and advises vendors on visual data modeling techniques. He regularly teaches agile dimensional modeling courses worldwide and has taught dimensional DW/BI skills to thousands of students. Jim Stagnitto is a data warehouse and master data management architect specializing in the healthcare, financial services, and information service industries. He is the founder of the data warehousing and data mining consulting firm Llumino.

Mining big data requires a deep investment in people and time. How can you be sure you’re building the right models? With this hands-on book, you’ll learn a flexible toolset and methodology for building effective analytics applications with Hadoop. Using lightweight tools such as Python, Apache Pig, and the D3.js library, your team will create an agile environment for exploring data, starting with an example application to mine your own email inboxes. You’ll learn an iterative approach that enables you to quickly change the kind of analysis you’re doing, depending on what the data is telling you. All example code in this book is available as working Heroku apps. Create analytics applications by using the agile big data development methodology Build value from your data in a series of agile sprints, using the data-value stack Gain insight by using several data structures to extract multiple features from a single dataset Visualize data with charts, and expose different aspects through interactive reports Use historical data to predict the future, and translate predictions into action Get feedback from users after each sprint to keep your project on track

You have to make sense of enormous amounts of data, and while the notion of “agile data warehousing might sound tricky, it can yield as much as a 3-to-1 speed advantage while cutting project costs in half. Bring this highly effective technique to your organization with the wisdom of agile data warehousing expert Ralph Hughes. Agile Data Warehousing Project Management will give you a thorough introduction to the method as you would practice it in the project room to build a serious “data mart. Regardless of where you are today, this step-by-step implementation guide will prepare you to join or even lead a team in visualizing, building, and validating a single component to an enterprise data warehouse. Provides a thorough grounding on the mechanics of Scrum as well as practical advice on keeping your team on track Includes strategies for getting accurate and actionable requirements from a team’s business partner Revolutionary estimating techniques that make forecasting labor far more understandable and accurate Demonstrates a blends of Agile methods to simplify team management and synchronize inputs across IT specialties Enables you and your teams to start simple and progress steadily to world-class performance levels

Learn how to manage a modern data stack and get the most out of data in your organization! Thanks to the emergence of new technologies and the explosion of data in recent years, we need new practices for managing and getting value out of data. In the modern, data driven competitive landscape the "best guess" approach—reading blog posts here and there and patching together data practices without any real visibility—is no longer going to hack it. The Informed Company provides definitive direction on how best to leverage the modern data stack, including cloud computing, columnar storage, cloud ETL tools, and cloud BI tools. You'll learn how to work with Agile methods and set up processes that's right for your company to use your data as a key weapon for your success . . . You'll discover best practices for every stage, from querying production databases at a small startup all the way to setting up data marts for different business lines of an enterprise. In their work at Chartio, authors Fowler and David have learned that most businesspeople are almost completely self-taught when it comes to data. If they are using resources, those resources are outdated, so they're missing out on the latest cloud technologies and advances in data analytics. This book will firm up your understanding of data and bring you into the present with knowledge around what works and what doesn't. Discover the data stack strategies that are working for today's successful small, medium, and enterprise companies Learn the different Agile stages of data organization, and the right one for your team Learn how to maintain Data Lakes and Data Warehouses for effective, accessible data storage Gain the knowledge you need to architect Data Warehouses and Data Marts Understand your business's level of data sophistication and the steps you can take to get to "level up" your data The Informed Company is the definitive data book for anyone who wants to work faster and more nimbly, armed with actionable decision-making data.