Statistical Distance
Created: 2022-05-19 12:54
#note
Statistical distance is the general idea of calculating the difference between statistical objects like different probability distributions for a random variable. One approach consists in calculating the distance between the two distributions, but this can be very challenging. Instead, divergence it is generally used. Divergence is a scoring of how one distribution differs from another. Divergence is a 'distance' but not a metric since it is asymmetric.
Divergence scores are very important in information theory, for example they provide shortcuts to calculate Mutual Information and cross-entropy.