Information theory

High school (ages 12+)
Science & Technology

Name:

Date:

Step 1 · Read the article

Open the article and read it at the level above. Then answer the questions below in your own words.

https://smallwhale.org/articles/information-theory

Write what each word means, using the article.

Do not copy the article. Write two things you learned, the way you would explain them to a friend.

Answers below are taken from the article’s stored facts, so they match what the student read.

1. What is the formalization date?Claude Shannon's foundational paper was published in 1948
2. What is the interdisciplinary nature?Connects mathematics, computer science, electrical engineering, physics, and biology
3. Why do we use bits in information theory?A bit is the basic unit of information, defined as the amount needed to choose between two equally likely possibilities. It lets us count how much information something has.
4. How does entropy help with data compression?Entropy measures how unpredictable a set of data is. The higher the entropy, the more bits we need, but knowing the entropy lets us compress data to the minimum average number of bits.

Entropy (Information Theory): A measure of the average uncertainty or randomness of a random variable.
Mutual Information: Measures the amount of information one random variable contains about another.
Channel Capacity: The maximum rate at which information can be reliably transmitted over a communication channel.