The Mathematical Theory of Communication
# A new type of information theory
this paper is not super well-known but has changed my opinion of how deep learning works more than almost anything else
it says that we should measure the amount of information available in some representation based on how *extractable* it is, given finite... See more
Information Theory is awesome so here is a TL;DR about Shanon's entropy.
This field is about quantifying the amount "of information" contained in a signal and how much can be transmitted under certain conditions.
1/11
François Fleuretx.com