logoalt Hacker News

mr_mitm06/16/20252 repliesview on HN

Just look at the definition of entropy. Knowledge about a system never enters the equation.

S := -k_B sum p_i ln (p_i)


Replies

ajkjk06/16/2025

As the other replier said, despite your dismissiveness, the knowledge about the system is in the probabilities, so it's right there in the equation.

Suppose you flip a coin. Before flipping the coin, your knowledge is "heads or tails". After flipping it, your knowledge becomes one of either heads or tails. The amount of information you gained by resolving your imperfect knowledge is the entropy of the distribution.

The same model works for physical entropy without much modification; the imperfect knowledge is the difference between knowing a macrostate versus the exact microstate.

show 1 reply
kgwgk06/16/2025

p_i

Edit to add lots of words:

In the definition of entropy

S := -k_B sum p_i ln (p_i)

knowledge about the system enters the equation in the p_i terms.

The other term is a constant so it’s not like there are many other choices to link the entropy to the system!

show 1 reply