Scientific Workflows

This is a timely book presenting an overview of the current state-of-the-art within established projects, presenting many different aspects of workflow from users to tool builders. It provides an overview of active research, from a number of different perspectives. It includes theoretical aspects of workflow and deals with workflow for e-Science as opposed to e-Commerce. The topics covered will be of interest to a wide range of practitioners.

Creating scientific workflow applications is a very challenging task due to the complexity of the distributed computing environments involved, the complex control and data flow requirements of scientific applications, and the lack of high-level languages and tools support. Particularly, sophisticated expertise in distributed computing is commonly required to determine the software entities to perform computations of workflow tasks, the computers on which workflow tasks are to be executed, the actual execution order of workflow tasks, and the data transfer between them. Qin and Fahringer present a novel workflow language called Abstract Workflow Description Language (AWDL) and the corresponding standards-based, knowledge-enabled tool support, which simplifies the development of scientific workflow applications. AWDL is an XML-based language for describing scientific workflow applications at a high level of abstraction. It is designed in a way that allows users to concentrate on specifying such workflow applications without dealing with either the complexity of distributed computing environments or any specific implementation technology. This research monograph is organized into five parts: overview, programming, optimization, synthesis, and conclusion, and is complemented by an appendix and an extensive reference list. The topics covered in this book will be of interest to both computer science researchers (e.g. in distributed programming, grid computing, or large-scale scientific applications) and domain scientists who need to apply workflow technologies in their work, as well as engineers who want to develop distributed and high-throughput workflow applications, languages and tools.

Focuses on how to use web service computing and service-based workflow technologies to develop timely, effective workflows for both business and scientific fields Utilizing web computing and Service-Oriented Architecture (SOA), Business and Scientific Workflows: A Web Service Oriented Approach focuses on how to design, analyze, and deploy web service based workflows for both business and scientific applications in many areas of healthcare and biomedicine. It also discusses and presents the recent research and development results. This informative reference features application scenarios that include healthcare and biomedical applications, such as personalized healthcare processing, DNA sequence data processing, and electrocardiogram wave analysis, and presents: Updated research and development results on the composition technologies of web services for ever-sophisticated service requirements from various users and communities Fundamental methods such as Petri nets and social network analysis to advance the theory and applications of workflow design and web service composition Practical and real applications of the developed theory and methods for such platforms as personalized healthcare and Biomedical Informatics Grids The authors' efforts on advancing service composition methods for both business and scientific software systems, with theoretical and empirical contributions With workflow-driven service composition and reuse being a hot topic in both academia and industry, this book is ideal for researchers, engineers, scientists, professionals, and students who work on service computing, software engineering, business and scientific workflow management, the internet, and management information systems (MIS).

Scientific workflows have emerged as a key technology that assists scientists with the design, management, execution, sharing and reuse of in silico experiments. Workflow management systems simplify the management of scientific workflows by providing graphical interfaces for their development, monitoring and analysis. Nowadays, e-Science combines such workflow management systems with large-scale data and computing resources into complex research infrastructures. For instance, e-Science allows the conveyance of best practice research in collaborations by providing workflow repositories, which facilitate the sharing and reuse of scientific workflows. However, scientists are still faced with different limitations while reusing workflows. One of the most common challenges they meet is the need to select appropriate applications and their individual execution parameters. If scientists do not want to rely on default or experience-based parameters, the best-effort option is to test different workflow set-ups using either trial and error approaches or parameter sweeps. Both methods may be inefficient or time consuming respectively, especially when tuning a large number of parameters. Therefore, scientists require an effective and efficient mechanism that automatically tests different workflow set-ups in an intelligent way and will help them to improve their scientific results. This thesis addresses the limitation described above by defining and implementing an approach for the optimization of scientific workflows. In the course of this work, scientists’ needs are investigated and requirements are formulated resulting in an appropriate optimization concept. In a following step, this concept is prototypically implemented by extending a workflow management system with an optimization framework, including general mechanisms required to conduct workflow optimization. As optimization is an ongoing research topic, different algorithms are provided by pluggable extensions (plugins) that can be loosely coupled with the framework, resulting in a generic and quickly extendable system. In this thesis, an exemplary plugin is introduced which applies a Genetic Algorithm for parameter optimization. In order to accelerate and therefore make workflow optimization feasible at all, e-Science infrastructures are utilized for the parallel execution of scientific workflows. This is empowered by additional extensions enabling the execution of applications and workflows on distributed computing resources. The actual implementation and therewith the general approach of workflow optimization is experimentally verified by four use cases in the life science domain. All workflows were significantly improved, which demonstrates the advantage of the proposed workflow optimization. Finally, a new collaboration-based approach is introduced that harnesses optimization provenance to make optimization faster and more robust in the future.

Scientific workflows are one important means in the context of data-intensive science for reliable and efficient scientific data processing in distributed computing infrastructures such as Grids. A common trend is to adapt existing and established business workflow technologies instead of developing own technologies from scratch. This thesis provides a model-driven approach for scientific workflow engineering, in which domain-specific languages (DSLs) tailored for a certain scientific domain are used for scientific workflow modeling, and automated mapping techniques for technical execution are developed and evaluated. The Business Process Model and Notation (BPMN) is thereby used at the domain-specific layer and the Web Services Business Process Execution Language (BPEL) at the technical layer. The implementation uses the Eclipse Modeling Framework (EMf) and is evaluated in three application scenarios.

This book constitutes the thoroughly refereed post-conference proceedings of the 9th International Conference on High Performance Computing for Computational Science, VECPAR 2010, held in Berkeley, CA, USA, in June 2010. The 34 revised full papers presented together with five invited contributions were carefully selected during two rounds of reviewing and revision. The papers are organized in topical sections on linear algebra and solvers on emerging architectures, large-scale simulations, parallel and distributed computing, numerical algorithms.

Identifies Recent Technological Developments Worldwide The field of grid computing has made rapid progress in the past few years, evolving and developing in almost all areas, including concepts, philosophy, methodology, and usages. Grid Computing: Infrastructure, Service, and Applications reflects the recent advances in this field, covering the research aspects that involve infrastructure, middleware, architecture, services, and applications. Grid Systems Across the Globe The first section of the book focuses on infrastructure and middleware and presents several national and international grid systems. The text highlights China Research and Development environment Over Wide-area Network (CROWN), several ongoing cyberinfrastructure efforts in New York State, and Enabling Grids for E-sciencE (EGEE), which is co-funded by the European Commission and the world’s largest multidisciplinary grid infrastructure today. The second part of the book discusses recent grid service advances. The authors examine the UK National Grid Service (NGS), the concept of resource allocation in a grid environment, OMIIBPEL, and the possibility of treating scientific workflow issues using techniques from the data stream community. The book describes an SLA model, reviews portal and workflow technologies, presents an overview of PKIs and their limitations, and introduces PIndex, a peer-to-peer model for grid information services. New Projects and Initiatives The third section includes an analysis of innovative grid applications. Topics covered include the WISDOM initiative, incorporating flow-level networking models into grid simulators, system-level virtualization, grid usage in the high-energy physics environment in the LHC project, and the Service Oriented HLA RTI (SOHR) framework. With a comprehensive summary of past advances, this text is a window into the future of this nascent technology, forging a path for the next generation of cyberinfrastructure developers.

This book presents innovative ideas, cutting-edge findings, and novel techniques, methods, and applications in a broad range of cybersecurity and cyberthreat intelligence areas. As our society becomes smarter, there is a corresponding need to be able to secure our cyberfuture. The approaches and findings described in this book are of interest to businesses and governments seeking to secure our data and underpin infrastructures, as well as to individual users.