Concurrent Data Processing In Elixir

Learn different ways of writing concurrent code in Elixir and increase your application's performance, without sacrificing scalability or fault-tolerance. Most projects benefit from running background tasks and processing data concurrently, but the world of OTP and various libraries can be challenging. Which Supervisor and what strategy to use? What about GenServer? Maybe you need back-pressure, but is GenStage, Flow, or Broadway a better choice? You will learn everything you need to know to answer these questions, start building highly concurrent applications in no time, and write code that's not only fast, but also resilient to errors and easy to scale. Whether you are building a high-frequency stock trading application or a consumer web app, you need to know how to leverage concurrency to build applications that are fast and efficient. Elixir and the OTP offer a range of powerful tools, and this guide will show you how to choose the best tool for each job, and use it effectively to quickly start building highly concurrent applications. Learn about Tasks, supervision trees, and the different types of Supervisors available to you. Understand why processes and process linking are the building blocks of concurrency in Elixir. Get comfortable with the OTP and use the GenServer behaviour to maintain process state for long-running jobs. Easily scale the number of running processes using the Registry. Handle large volumes of data and traffic spikes with GenStage, using back-pressure to your advantage. Create your first multi-stage data processing pipeline using producer, consumer, and producer-consumer stages. Process large collections with Flow, using MapReduce and more in parallel. Thanks to Broadway, you will see how easy it is to integrate with popular message broker systems, or even existing GenStage producers. Start building the high-performance and fault-tolerant applications Elixir is famous for today. What You Need: You'll need Elixir 1.9+ and Erlang/OTP 22+ installed on a Mac OS X, Linux, or Windows machine.

Learn different ways of writing concurrent code in Elixir and increase your application's performance, without sacrificing scalability or fault-tolerance. Most projects benefit from running background tasks and processing data concurrently, but the world of OTP and various libraries can be challenging. Which Supervisor and what strategy to use? What about GenServer? Maybe you need back-pressure, but is GenStage, Flow, or Broadway a better choice? You will learn everything you need to know to answer these questions, start building highly concurrent applications in no time, and write code that's not only fast, but also resilient to errors and easy to scale. Whether you are building a high-frequency stock trading application or a consumer web app, you need to know how to leverage concurrency to build applications that are fast and efficient. Elixir and the OTP offer a range of powerful tools, and this guide will show you how to choose the best tool for each job, and use it effectively to quickly start building highly concurrent applications. Learn about Tasks, supervision trees, and the different types of Supervisors available to you. Understand why processes and process linking are the building blocks of concurrency in Elixir. Get comfortable with the OTP and use the GenServer behaviour to maintain process state for long-running jobs. Easily scale the number of running processes using the Registry. Handle large volumes of data and traffic spikes with GenStage, using back-pressure to your advantage. Create your first multi-stage data processing pipeline using producer, consumer, and producer-consumer stages. Process large collections with Flow, using MapReduce and more in parallel. Thanks to Broadway, you will see how easy it is to integrate with popular message broker systems, or even existing GenStage producers. Start building the high-performance and fault-tolerant applications Elixir is famous for today. What You Need: You'll need Elixir 1.9+ and Erlang/OTP 22+ installed on a Mac OS X, Linux, or Windows machine.

Learn different ways of writing concurrent code in Elixir and increase your application's performance, without sacrificing scalability or fault-tolerance. Most projects benefit from running background tasks and processing data concurrently, but the world of OTP and various libraries can be challenging. Which Supervisor and what strategy to use? What about GenServer? Maybe you need back-pressure, but is GenStage, Flow, or Broadway a better choice? You will learn everything you need to know to answer these questions, start building highly concurrent applications in no time, and write code that's not only fast, but also resilient to errors and easy to scale. Whether you are building a high-frequency stock trading application or a consumer web app, you need to know how to leverage concurrency to build applications that are fast and efficient. Elixir and the OTP offer a range of powerful tools, and this guide will show you how to choose the best tool for each job, and use it effectively to quickly start building highly concurrent applications. Learn about Tasks, supervision trees, and the different types of Supervisors available to you. Understand why processes and process linking are the building blocks of concurrency in Elixir. Get comfortable with the OTP and use the GenServer behaviour to maintain process state for long-running jobs. Easily scale the number of running processes using the Registry. Handle large volumes of data and traffic spikes with GenStage, using back-pressure to your advantage. Create your first multi-stage data processing pipeline using producer, consumer, and producer-consumer stages. Process large collections with Flow, using MapReduce and more in parallel. Thanks to Broadway, you will see how easy it is to integrate with popular message broker systems, or even existing GenStage producers. Start building the high-performance and fault-tolerant applications Elixir is famous for today. What You Need: You'll need Elixir 1.9+ and Erlang/OTP 22+ installed on a Mac OS X, Linux, or Windows machine.

Leverage the power of Elixir programming language to solve practical problems associated with scalability, concurrency, fault tolerance, and high availability. Key Features Enhance your Elixir programming skills using its powerful tools and abstractions Discover how to develop a full-fledged file server Understand how to use Phoenix to create a web interface for your application. Book Description Running concurrent, fault-tolerant applications that scale is a very demanding responsibility. After learning the abstractions that Elixir gives us, developers are able to build such applications with inconceivable low effort. There is a big gap between playing around with Elixir and running it in production, serving live requests. This book will help you fll this gap by going into detail on several aspects of how Elixir works and showing concrete examples of how to apply the concepts learned to a fully fledged application. In this book, you will learn how to build a rock-solid application, beginning by using Mix to create a new project. Then you will learn how the use of Erlang's OTP, along with the Elixir abstractions that run on top of it (such as GenServer and GenStage), that allow you to build applications that are easy to parallelize and distribute. You will also master supervisors (and supervision trees), and comprehend how they are the basis for building fault-tolerant applications. Then you will use Phoenix to create a web interface for your application. Upon fnishing implementation, you will learn how to take your application to the cloud, using Kubernetes to automatically deploy, scale, and manage it. Last, but not least, you will keep your peace of mind by learning how to thoroughly test and then monitor your application. What you will learn Use Elixir tools, including IEx and Mix Find out how an Elixir project is structured and how to create umbrella applications Discover the power of supervision trees, the basis for fault-tolerance Create a Domain-Specifc Language (DSL) that abstracts complexity Create a blazing-fast web interface for your application with Phoenix Set up an automatic deployment process for the cloud Monitor your application and be warned if anything unexpected happens Who this book is for Mastering Elixir is for you if you have experience in Elixir programming and want to take it to the next level. This Elixir book shows you how to build, deploy, and maintain robust applications, allowing you to go from tinkering with Elixir on side projects to using it in a live environment. However, no prior knowledge of Elixir is required to enjoy the complex topics covered in the book.

Summary Revised and updated for Elixir 1.7, Elixir in Action, Second Edition teaches you how to apply Elixir to practical problems associated with scalability, fault tolerance, and high availability. Along the way, you'll develop an appreciation for, and considerable skill in, a functional and concurrent style of programming. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the Technology When you're building mission-critical software, fault tolerance matters. The Elixir programming language delivers fast, reliable applications, whether you're building a large-scale distributed system, a set of backend services, or a simple web app. And Elixir's elegant syntax and functional programming mindset make your software easy to write, read, and maintain. About the Book Elixir in Action, Second Edition teaches you how to build production-quality distributed applications using the Elixir programming language. Author Saša Jurić introduces this powerful language using examples that highlight the benefits of Elixir's functional and concurrent programming. You'll discover how the OTP framework can radically reduce tedious low-level coding tasks. You'll also explore practical approaches to concurrency as you learn to distribute a production system over multiple machines. What's inside Updated for Elixir 1.7 Functional and concurrent programming Introduction to distributed system design Creating deployable releases About the Reader You'll need intermediate skills with client/server applications and a language like Java, C#, or Ruby. No previous experience with Elixir required. About the Author Saša Jurić is a developer with extensive experience using Elixir and Erlang in complex server-side systems. Table of Contents First steps Building blocks Control flow Data abstractions Concurrency primitives Generic server processes Building a concurrent system Fault-tolerance basics Isolating error effects Beyond GenServer Working with components Building a distributed system Running the system

The Elixir programming language has become a go-to tool for creating reliable, fault-tolerant, and robust server-side applications. Thanks to Nerves, those same exact benefits can be realized in embedded applications. This book will teach you how to structure, build, and deploy production grade Nerves applications to network-enabled devices. The weather station sensor hub project that you will be embarking upon will show you how to create a full stack IoT solution in record time. You will build everything from the embedded Nerves device to the Phoenix backend and even the Grafana time-series data visualizations. Elixir as a programming language has found its way into many different software domains, largely in part to the rock-solid foundation of the Erlang virtual machine. Thanks to the Nerves framework, Elixir has also found success in the world of embedded systems and IoT. Having access to all of the Elixir and OTP constructs such as concurrency, supervision, and immutability makes for a powerful IoT recipe. Find out how to create fault-tolerant, reliable, and robust embedded applications using the Nerves framework. Build and deploy a production-grade weather station sensor hub using Elixir and Nerves, all while leveraging the best practices established by the Nerves community for structuring and organizing Nerves applications. Capture all of your weather station sensor data using Phoenix and Ecto in a lightweight server-side application. Efficiently store and retrieve the time-series weather data collected by your device using TimescaleDB (the Postgres extension for time-series data). Finally, complete the full stack IoT solution by using Grafana to visualize all of your time-series weather station data. Discover how to create software solutions where the underlying technologies and techniques are applicable to all layers of the project. Take your project from idea to production ready in record time with Elixir and Nerves. What You Need: To complete the Nerves weather station project in this book, you will need the following: A Linux, MacOS, or Windows computer to build and deploy Nerves firmware images A Raspberry Pi Zero W or any other Nerves supported target (https://hexdocs.pm/nerves/targets.html#supported-targets-and-systems) A VEML6030 light sensor An BME680 environmental sensor An SGP30 air quality sensor Qwiic connect cables for weather sensors

"Functional programming techniques help you manage the complexities of today’s real-world, concurrent systems; maximize uptime; and manage security. Enter Elixir, with its modern, Ruby-like, extendable syntax, compile and runtime evaluation, hygienic macro system, and more. But, just as importantly, Elixir brings a sense of enjoyment to parallel, functional programming. Your applications become fun to work with, and the language encourages you to experiment."--Publisher's website.

Summary The Little Elixir & OTP Guidebook gets you started programming applications with Elixir and OTP. You begin with a quick overview of the Elixir language syntax, along with just enough functional programming to use it effectively. Then, you'll dive straight into OTP and learn how it helps you build scalable, fault-tolerant and distributed applications through several fun examples. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the Technology Elixir is an elegant programming language that combines the expressiveness of Ruby with the concurrency and fault-tolerance of Erlang. It makes full use of Erlang's BEAM VM and OTP library, so you get two decades' worth of maturity and reliability right out of the gate. Elixir's support for functional programming makes it perfect for modern event-driven applications. About the Book The Little Elixir & OTP Guidebook gets you started writing applications with Elixir and OTP. You'll begin with the immediately comfortable Elixir language syntax, along with just enough functional programming to use it effectively. Then, you'll dive straight into several lighthearted examples that teach you to take advantage of the incredible functionality built into the OTP library. What's Inside Covers Elixir 1.2 and 1.3 Introduction to functional concurrency with actors Experience the awesome power of Erlang and OTP About the Reader Written for readers comfortable with a standard programming language like Ruby, Java, or Python. FP experience is helpful but not required. About the Author Benjamin Tan Wei Hao is a software engineer at Pivotal Labs, Singapore. He is also an author, a speaker, and an early adopter of Elixir. Table of Contents GETTING STARTED WITH ELIXIR AND OTP Introduction A whirlwind tour Processes 101 Writing server applications with GenServer FAULT TOLERANCE, SUPERVISION, AND DISTRIBUTION Concurrent error-handling and fault tolerance with links, monitors, and processes Fault tolerance with Supervisors Completing the worker-pool application Distribution and load balancing Distribution and fault tolerance Dialyzer and type specifications Property-based and concurrency testing