Lab 2 — Shannon Entropy

Purpose

Understand and implement Shannon's entropy formula for measuring information uncertainty.

Theory

Choice under uncertainty (equal probabilities): entropy = lb(N) — the binary logarithm of the number of outcomes.

Choice under certainty (different probabilities): Shannon entropy formula:

H(X) = sum( p(i) * log2(1 / p(i)) )   for i = 1..N

Example: probabilities 50%, 25%, 12.5%, 12.5% gives H(X) = 1.75

Step 1 — Web Form interface

Create 4 TextBox controls pre-filled with 50, 25, 12.5, 12.5, a Button, and a Label for the result:

<asp:TextBox ID="TextBox1" runat="server" Width="84px">50</asp:TextBox>
<asp:TextBox ID="TextBox2" runat="server" Width="84px">25</asp:TextBox>
<asp:TextBox ID="TextBox3" runat="server" Width="84px">12.5</asp:TextBox>
<asp:TextBox ID="TextBox4" runat="server" Width="84px">12.5</asp:TextBox>
<asp:Button ID="Button1" runat="server" Text="H(X)=" />
<asp:Label ID="Label1" runat="server" Text=" "></asp:Label>

Step 2 — Implementation

protected void Button1_Click(object sender, EventArgs e)
{
    float[] ev = new float[4];
    ev[0] = float.Parse(TextBox1.Text);
    ev[1] = float.Parse(TextBox2.Text);
    ev[2] = float.Parse(TextBox3.Text);
    ev[3] = float.Parse(TextBox4.Text);

    double s = 0, p = 0;
    for (int i = 0; i < ev.Length; i++)
    {
        p  = ev[i] / 100;
        s += p * Math.Log(1 / p, 2);
    }
    Label1.Text = s.ToString();
}

Step 3 — Refactor to MVVM (Read / BusinessLogic / Write)

float[] ev = new float[4];
double s;

void Read()
{
    ev[0] = float.Parse(TextBox1.Text);
    ev[1] = float.Parse(TextBox2.Text);
    ev[2] = float.Parse(TextBox3.Text);
    ev[3] = float.Parse(TextBox4.Text);
}

void BusinessLogic()
{
    s = 0;
    double p = 0;
    for (int i = 0; i < ev.Length; i++)
    {
        p  = ev[i] / 100;
        s += p * Math.Log(1 / p, 2);
    }
}

void Write() { Label1.Text = s.ToString(); }

Step 4 — Migrate to Console

Replace TextBox reads with float.Parse(args[i]) and Label write with Console.WriteLine("H=" + s).

Run: _Default.exe 50 25 12.5 12.5 — output: H=1.75

Live Demo

Enter four probabilities (percentages summing to 100) and compute H(X) on the server.

Code-behind (C#)

protected void btnCompute_Click(object sender, EventArgs e)
{
    float[] ev = new float[4];
    ev[0] = float.Parse(txtP1.Text);
    ev[1] = float.Parse(txtP2.Text);
    ev[2] = float.Parse(txtP3.Text);
    ev[3] = float.Parse(txtP4.Text);

    double s = 0, p = 0;
    for (int i = 0; i < ev.Length; i++)
    {
        p = ev[i] / 100;
        if (p > 0)
            s += p * Math.Log(1 / p, 2);
    }

    litSum.Text = string.Join(" + ", ev) + " = " + ev.Sum() + " %";
    litResult.Text = s.ToString("0.######");
    pnlResult.Visible = true;
}

Conclusion

Shannon entropy quantifies the average amount of information in a probability distribution. The MVVM pattern allows the same business logic to run in both web and console environments without changes.