A Student S Guide To Coding And Information Theory

This is a concise, easy-to-read guide, introducing beginners to coding theory and information theory.

This easy-to-read guide provides a concise introduction to the engineering background of modern communication systems, from mobile phones to data compression and storage. Background mathematics and specific engineering techniques are kept to a minimum so that only a basic knowledge of high-school mathematics is needed to understand the material covered. The authors begin with many practical applications in coding, including the repetition code, the Hamming code and the Huffman code. They then explain the corresponding information theory, from entropy and mutual information to channel capacity and the information transmission theorem. Finally, they provide insights into the connections between coding theory and other fields. Many worked examples are given throughout the book, using practical applications to illustrate theoretical definitions. Exercises are also included, enabling readers to double-check what they have learned and gain glimpses into more advanced topics, making this perfect for anyone who needs a quick introduction to the subject.

This fundamental monograph introduces both the probabilistic and algebraic aspects of information theory and coding. It has evolved from the authors' years of experience teaching at the undergraduate level, including several Cambridge Maths Tripos courses. The book provides relevant background material, a wide range of worked examples and clear solutions to problems from real exam papers. It is a valuable teaching aid for undergraduate and graduate students, or for researchers and engineers who want to grasp the basic principles.

Striving to explore the subject in as simple a manner as possible, this book helps readers understand the elusive concept of entropy. Innovative aspects of the book include the construction of statistical entropy from desired properties, the derivation of the entropy of classical systems from purely classical assumptions, and a statistical thermodynamics approach to the ideal Fermi and ideal Bose gases. Derivations are worked through step-by-step and important applications are highlighted in over 20 worked examples. Around 50 end-of-chapter exercises test readers' understanding. The book also features a glossary giving definitions for all essential terms, a time line showing important developments, and list of books for further study. It is an ideal supplement to undergraduate courses in physics, engineering, chemistry and mathematics.

Information Theory: Coding Theorems for Discrete Memoryless Systems presents mathematical models that involve independent random variables with finite range. This three-chapter text specifically describes the characteristic phenomena of information theory. Chapter 1 deals with information measures in simple coding problems, with emphasis on some formal properties of Shannon’s information and the non-block source coding. Chapter 2 describes the properties and practical aspects of the two-terminal systems. This chapter also examines the noisy channel coding problem, the computation of channel capacity, and the arbitrarily varying channels. Chapter 3 looks into the theory and practicality of multi-terminal systems. This book is intended primarily for graduate students and research workers in mathematics, electrical engineering, and computer science.

This book is intended to introduce coding theory and information theory to undergraduate students of mathematics and computer science. It begins with a review of probablity theory as applied to finite sample spaces and a general introduction to the nature and types of codes. The two subsequent chapters discuss information theory: efficiency of codes, the entropy of information sources, and Shannon's Noiseless Coding Theorem. The remaining three chapters deal with coding theory: communication channels, decoding in the presence of errors, the general theory of linear codes, and such specific codes as Hamming codes, the simplex codes, and many others.

This book provides an up-to-date introduction to information theory. In addition to the classical topics discussed, it provides the first comprehensive treatment of the theory of I-Measure, network coding theory, Shannon and non-Shannon type information inequalities, and a relation between entropy and group theory. ITIP, a software package for proving information inequalities, is also included. With a large number of examples, illustrations, and original problems, this book is excellent as a textbook or reference book for a senior or graduate level course on the subject, as well as a reference for researchers in related fields.