To show \(\sum p_i\log(p_i) = p\)

Consider \(H = -\sum p_i log(p_i) = log(n)\)

Consider weighted AM-GM for \(p_i\):

$$ (\prod \frac{1}{p_i}^p_i)^\frac{1}{\sum p_i} \leq \sum \frac{1}{p_i} \times p_i = n $$

Take log both sides:

$$ \sum p_i \log(\frac{1}{p_i}) \leq \log(n) $$

And hence euqality holds only if \(p_i=p_j=1/n\)

$$ p_i = 1/n $$

Like this post? Share on: TwitterFacebookEmail


Saket Choudhary is a PhD candidate at the University of Southern California

Published

Category

Miscellaneous

Tags

Stay in Touch