Outline Introduction Information and coding theory History Applications Suggested Reading
Information and Coding Theory
Introduction
Malay Bhattacharyya
Assistant Professor
Department of Information Technology
Indian Institute of Engineering Science and Technology
Shibpur, Howrah, India
July, 2014
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
1
Introduction
Data and information
The communication channel
2
Information and coding theory
What is information theory?
What is coding theory?
Relation between information and coding theory
Coding versus cryptography
3
History
4
Applications
5
Suggested Reading
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Data and information
Data: Raw or unorganized content (such as alphabets, numbers,
or symbols) that refer to, or represent, conditions, ideas, or
objects. E.g., 11011000001, a, binary, etc.
Information: A meaningful form of the data. E.g., binary
11011000001 is 1729 in decimal, etc.
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
The communication channel
Visualizing a model of communication channel
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Information theory
Information theory is the mathematical study of encoding
information such that maximum amount of information can be
transmitted rapidly through the communication channels.
It deals with the mathematical models that quanties information.
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Coding theory
Coding theory is the study of the properties of codes and their
tness in protecting the data from noise and for the applications
like data compression, error-correction, etc.
It deals with the mathematical models that makes the data open
but safe.
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Categories of coding theory
Coding theory can be subdivided into the following subareas:
Source coding It deals with data compression
(lossy/lossless).
Channel coding It deals with error-correcting codes.
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Relation between information and coding theory
Information theory formalizes the concepts around information
Coding theory formalizes the concepts around the data
Coding theory is one of the most important and direct applications
of information theory.
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Coding theory versus cryptography
Information Theory Information Security
& &
Coding Theory Cryptography
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
History of information and coding theory
When information is cheap, attention becomes expensive
James Gleick.
1948: Mathematical theory of communication by Claude E.
Shannon The father of Information Theory
1950: Hamming codes
1952: Human algorithm
1954: Reed-Muller codes and their decoders
1957: Cyclic codes
1960: BCH codes
1962: LDPC codes by Gallager
1981: Algebraic-geometry codes
1997: Quantum error-correction codes
2008: Polar codes
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Quantifying the information content
Genes are information contents in a genome
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Transmission of data
Economy of satellite transmission can be achieved through
ecient coding
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Error verication
The ISBN and EAN are capable of verifying errors
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Recovery of information
Error correction codes might help in recovering distorted data
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Data compression
Compressed data consumes lesser space on disk
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Coding as a way of communication
Braille codes help the blind people to communicate
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Resources
Books
1
Thomas M. Cover and Joy A. Thomas, Elements of
Information Theory, John Wiley & Sons, Inc., Second Edition,
2006.
2
Claude E. Shannon and Warren Weaver, The Mathematical
Theory of Communication, University of Illinois Press, 1949.
3
Todd K. Moon, Error Correction Coding: Mathematical
Methods and Algorithms, John Wiley & Sons, Inc., New
Jersey, 2005.
4
Ron M. Roth, Introduction to Coding Theory, Cambridge
University Press, 2006.
Journals
1
IEEE Transactions on Information Theory, IEEE.
2
Designs, Codes and Cryptography, Springer.
Malay Bhattacharyya Information and Coding Theory
Outline Introduction Information and coding theory History Applications Suggested Reading
Resources
Other similar courses
1
Venkatesan Guruswami at Carnegie Mellon University
http://www.cs.cmu.edu/ venkatg/teaching/codingtheory.
2
Tsachy Weissman at Stanford
http://www.stanford.edu/class/ee376a.
3
Madhu Sudan at MIT
http://people.csail.mit.edu/madhu/FT02.
4
John Daugman at University of Cambridge
http://www.cl.cam.ac.uk/teaching/1213/InfoTheory.
Wiki: http://ice902-ict.wikispaces.com
Malay Bhattacharyya Information and Coding Theory
