Performance Benchmarking Of Application Monitoring Frameworks

Application-level monitoring of continuously operating software systems provides insights into their dynamic behavior, helping to maintain their performance and availability during runtime. Such monitoring may cause a significant runtime overhead to the monitored system, depending on the number and location of used instrumentation probes. In order to improve a system’s instrumentation and to reduce the caused monitoring overhead, it is necessary to know the performance impact of each probe. While many monitoring frameworks are claiming to have minimal impact on the performance, these claims are often not backed up with a detailed performance evaluation determining the actual cost of monitoring. Benchmarks can be used as an effective and affordable way for these evaluations. However, no benchmark specifically targeting the overhead of monitoring itself exists. Furthermore, no established benchmark engineering methodology exists that provides guidelines for the design, execution, and analysis of benchmarks. This thesis introduces a benchmark approach to measure the performance overhead of application-level monitoring frameworks. The core contributions of this approach are 1) a definition of common causes of monitoring overhead, 2) a general benchmark engineering methodology, 3) the MooBench micro-benchmark to measure and quantify causes of monitoring overhead, and 4) detailed performance evaluations of three different application-level monitoring frameworks. Extensive experiments demonstrate the feasibility and practicality of the approach and validate the benchmark results. The developed benchmark is available as open source software and the results of all experiments are available for download to facilitate further validation and replication of the results.

In many enterprises, the number of deployed applications is constantly increasing. Those applications - often several hundreds - form large software landscapes. The comprehension of such landscapes is frequently impeded due to, for instance, architectural erosion, personnel turnover, or changing requirements. Furthermore, events such as performance anomalies can often only be understood in correlation with the states of the applications. Therefore, an efficient and effective way to comprehend such software landscapes in combination with the details of each application is required. In this thesis, we introduce a live trace visualization approach to support system and program comprehension in large software landscapes. It features two perspectives: a landscape-level perspective using UML elements and an application-level perspective following the 3D software city metaphor. Our main contributions are 1) an approach named ExplorViz for enabling live trace visualization of large software landscapes, 2) a monitoring and analysis approach capable of logging and processing the huge amount of conducted method calls in large software landscapes, and 3) display and interaction concepts for the software city metaphor beyond classical 2D displays and 2D pointing devices. Extensive lab experiments show that our monitoring and analysis approach elastically scales to large software landscapes while imposing only a low overhead on the productive systems. Furthermore, several controlled experiments demonstrate an increased efficiency and effectiveness for solving comprehension tasks when using our visualization. ExplorViz is available as open-source software on www.explorviz.net. Additionally, we provide extensive experimental packages of our evaluations to facilitate the verifiability and reproducibility of our results.

This book constitutes the thoroughly refereed post-conference proceedings of the 8th TPC Technology Conference, on Performance Evaluation and Benchmarking, TPCTC 2017, held in conjunction with the43rd International Conference on Very Large Databases (VLDB 2017) in August/September 2017. The 12 papers presented were carefully reviewed and selected from numeroussubmissions. The TPC remains committed to developing new benchmark standards to keep pace with these rapid changes in technology.

Because performance is paramount today, this thoroughly updated guide shows you how to test mission-critical applications for scalability and performance before you deploy them—whether it’s to the cloud or a mobile device. You’ll learn the complete testing process lifecycle step-by-step, along with best practices to plan, coordinate, and conduct performance tests on your applications. Set realistic performance testing goals Implement an effective application performance testing strategy Interpret performance test results Cope with different application technologies and architectures Understand the importance of End User Monitoring (EUM) Use automated performance testing tools Test traditional local applications, web applications, and web services Recognize and resolves issues often overlooked in performance tests Written by a consultant with over 15 years’ experience with performance testing, The Art of Application Performance Testing thoroughly explains the pitfalls of an inadequate testing strategy and offers a robust, structured approach for ensuring that your applications perform well and scale effectively when the need arises.

The overwhelming majority of a software system’s lifespan is spent in use, not in design or implementation. So, why does conventional wisdom insist that software engineers focus primarily on the design and development of large-scale computing systems? In this collection of essays and articles, key members of Google’s Site Reliability Team explain how and why their commitment to the entire lifecycle has enabled the company to successfully build, deploy, monitor, and maintain some of the largest software systems in the world. You’ll learn the principles and practices that enable Google engineers to make systems more scalable, reliable, and efficient—lessons directly applicable to your organization. This book is divided into four sections: Introduction—Learn what site reliability engineering is and why it differs from conventional IT industry practices Principles—Examine the patterns, behaviors, and areas of concern that influence the work of a site reliability engineer (SRE) Practices—Understand the theory and practice of an SRE’s day-to-day work: building and operating large distributed computing systems Management—Explore Google's best practices for training, communication, and meetings that your organization can use

The importance of small towns is gaining increased recognition as a result of two developments. The first development concerns the possible role of small towns in migration flows and urbanization. Development of small towns, in the form of improved basic services and other amenities, has been promoted in order to abate the impact of urbanization on large urban centres and to alleviate service provision pressures in major urban centers, whilst stimulating rural economies and eventually prompting social transformations. The second development concerns the targets set by the SDGs in 2015. SDG6 requires countries to ensure universal and equitable water services by 2030. This inclusive target requires that the water services needs of small towns are considered in the expansion of sustainable and equitable water services. This book aims to contribute to the study of water services in small towns by critically examining different approaches and experiences of water supply in small towns. It brings together empirical testimonies of how the implementation of reductionist models and the perseverance of certain principles underlying these models in the water sector have yielded suboptimal results. Much remains to be done before achieving universal service coverage in small towns is likely. In order to do that, we should start speaking of small towns as a category on their own and continue the work in elaborating further what these are and how they work. In Focus – a book series that showcases the latest accomplishments in water research. Each book focuses on a specialist area with papers from top experts in the field. It aims to be a vehicle for in-depth understanding and inspire further conversations in the sector.

The objective of APM Best Practices: Realizing Application Performance Management is to establish reliable application performance management (APM) practices—to demonstrate value, to do it quickly, and to adapt to the client circumstances. It's important to balance long-term goals with short-term deliverables, but without compromising usefulness or correctness. The successful strategy is to establish a few reasonable goals, achieve them quickly, and then iterate over the same topics two more times, with each successive iteration expanding the skills and capabilities of the APM team. This strategy is referred to as “Good, Better, Best”. The application performance monitoring marketplace is very focused on ease of installation, rapid time to usefulness, and overall ease of use. But these worthy platitudes do not really address the application performance management processes that ensure that you will deploy effectively, synergize on quality assurance test plans, triage accurately, and encourage collaboration across the application life cycle that ultimately lowers overall application cost and ensures a quality user experience. These are also fine platitudes but these are the ones that are of interest to your application sponsors. These are the ones for which you need to show value. This CA Press book employs this iterative approach, adapted pragmatically for the realities of your organizational and operational constraints, to realize a future state that your sponsors will find useful, predictable and manageable—and something that they will want to fund. In the meantime, you will learn the useful techniques needed to set up and maintain a useful performance management system utilizing best practices regardless of the software provider(s).