von Mises–Fisher distribution

The probability density function of the von Mises–Fisher distribution for the random p-dimensional unit vector ${\displaystyle \mathbf {x} }$ is given by:

${\displaystyle f_{p}(\mathbf {x}$ ;{\boldsymbol {\mu }},\kappa )=C_{p}(\kappa )\exp \left({\kappa {\boldsymbol {\mu }}^{\mathsf {T}}\mathbf {x} }\right),}

where ${\displaystyle \kappa \geq 0,\left\Vert {\boldsymbol {\mu }}\right\Vert =1}$ and the normalization constant ${\displaystyle C_{p}(\kappa )}$ is equal to

${\displaystyle C_{p}(\kappa )={\frac {\kappa ^{p/2-1}}{(2\pi )^{p/2}I_{p/2-1}(\kappa )}},}$

where ${\displaystyle I_{v}}$ denotes the modified Bessel function of the first kind at order ${\displaystyle v}$. If ${\displaystyle p=3}$, the normalization constant reduces to

${\displaystyle C_{3}(\kappa )={\frac {\kappa }{4\pi \sinh \kappa }}={\frac {\kappa }{2\pi (e^{\kappa }-e^{-\kappa })}}.}$

The parameters ${\displaystyle {\boldsymbol {\mu }}}$ and ${\displaystyle \kappa }$ are called the mean direction and concentration parameter, respectively. The greater the value of ${\displaystyle \kappa }$, the higher the concentration of the distribution around the mean direction ${\displaystyle {\boldsymbol {\mu }}}$. The distribution is unimodal for ${\displaystyle \kappa >0}$, and is uniform on the sphere for ${\displaystyle \kappa =0}$.

The von Mises–Fisher distribution for ${\displaystyle p=3}$ is also called the Fisher distribution.[1][2] It was first used to model the interaction of electric dipoles in an electric field.[3] Other applications are found in geology, bioinformatics, and text mining.

Starting from a normal distribution, with isotropic covariance, ${\displaystyle \kappa ^{-1}\mathbf {I} }$, and a mean, ${\displaystyle {\boldsymbol {\mu }}}$ of length ${\displaystyle r>0}$, that has the density:

${\displaystyle G_{p}(\mathbf {x}$ ;{\boldsymbol {\mu }},\kappa )=\left({\sqrt {\frac {\kappa }{2\pi }}}\right)^{p}\exp \left(-\kappa {\frac {(\mathbf {x} -{\boldsymbol {\mu }})'(\mathbf {x} -{\boldsymbol {\mu }})}{2}}\right),}

the Von Mises-Fisher distribution is obtained by conditioning on ${\displaystyle \left\|\mathbf {x} \right\|=1}$. By expanding

${\displaystyle (\mathbf {x} -{\boldsymbol {\mu }})'(\mathbf {x} -{\boldsymbol {\mu }})=\mathbf {x} '\mathbf {x} +{\boldsymbol {\mu }}'{\boldsymbol {\mu }}-2{\boldsymbol {\mu }}'\mathbf {x} ,}$

and using the fact that the first two right-hand-side terms are fixed, the Von Mises-Fisher density, ${\displaystyle f_{p}(\mathbf {x} ;r^{-1}{\boldsymbol {\mu }},r\kappa )}$ is recovered by recomputing the normalization constant by integrating ${\displaystyle \mathbf {x} }$ over the unit sphere. If ${\displaystyle r=0}$, we get the uniform distribution, with density ${\displaystyle f_{p}(\mathbf {x}$ ;{\boldsymbol {0}},0)} .

More succinctly, the restriction of any isotropic multivariate normal density to the unit hypershere, gives a Von Mises-Fisher density, up to normalization.

This construction can be generalized by starting with a normal distribution with a general covariance matrix, in which case conditioning on ${\displaystyle \left\|\mathbf {x} \right\|=1}$ gives the Fisher-Bingham distribution.

The expected value of the Von Mises-Fisher distribution is not on the unit hyperpshere, but instead has a length of less than one. This length is given by ${\displaystyle A_{p}(\kappa )}$ as defined above. For a Von Mises-Fisher distribution with mean direction ${\displaystyle {\boldsymbol {\mu }}}$ and concentration ${\displaystyle \kappa >0}$, the expected value is:

${\displaystyle A_{p}(\kappa ){\boldsymbol {\mu }}}$.

For ${\displaystyle \kappa =0}$, the expected value is at the origin. For finite ${\displaystyle \kappa >0}$, the length of the expected value, is strictly between zero and one and is a monotonic rising function of ${\displaystyle \kappa }$.

The empirical mean (arithmetic average) of a collection of points on the unit hypersphere behaves in a similar manner, being close to the origin for widely spread data and close to the sphere for concentrated data. Indeed, for the Von Mises-Fisher distribution, the expected value of the maximum-likelihood estimate based on a collection of points is equal to the empirical mean of those points.

The expected value can be used to compute differential entropy and KL divergence.

The differential entropy of ${\displaystyle f_{p}(\mathbf {x}$ ;{\boldsymbol {\mu }},\kappa )} is:

${\displaystyle -\log f_{p}(A_{p}(\kappa ){\boldsymbol {\mu }};{\boldsymbol {\mu }},\kappa )=-\log C_{p}(\kappa )-\kappa A_{p}(\kappa )}$.

Notice that the entropy is a function of ${\displaystyle \kappa }$ only.

The KL divergence between ${\displaystyle f_{p}(\mathbf {x}$ ;{\boldsymbol {\mu _{0}}},\kappa _{0})} and ${\displaystyle f_{p}(\mathbf {x}$ ;{\boldsymbol {\mu _{1}}},\kappa _{1})} is:

${\displaystyle \log {\frac {f_{p}(A_{p}(\kappa _{0}){\boldsymbol {\mu _{0}}};{\boldsymbol {\mu _{0}}},\kappa _{0})}{f_{p}(A_{p}(\kappa _{0}){\boldsymbol {\mu _{0}}};{\boldsymbol {\mu _{1}}},\kappa _{1})}}}$

The matrix von Mises-Fisher distribution (also known as matrix Langevin distribution[5][6]) has the density

${\displaystyle f_{n,p}(\mathbf {X}$ ;\mathbf {F} )\propto \exp(\operatorname {tr} (\mathbf {F} ^{\mathsf {T}}\mathbf {X} ))}

supported on the Stiefel manifold of ${\displaystyle n\times p}$ orthonormal p-frames ${\displaystyle \mathbf {X} }$, where ${\displaystyle \mathbf {F} }$ is an arbitrary ${\displaystyle n\times p}$ real matrix.[7][8]

For ${\displaystyle p=3}$, the angle θ between ${\displaystyle \mathbf {x} }$ and ${\displaystyle {\boldsymbol {\mu }}}$ satisfies ${\displaystyle \cos \theta ={\boldsymbol {\mu }}^{\mathsf {T}}\mathbf {x} }$. It has the distribution

${\displaystyle p(\theta )=\int d^{2}xf(x;{\boldsymbol {\mu }},\kappa )\,\delta \left(\theta -{\text{arc cos}}({\boldsymbol {\mu }}^{\mathsf {T}}\mathbf {x} )\right)}$,

which can be easiliy evaluated as

${\displaystyle p(\theta )=2\pi C_{3}(\kappa )\,\sin \theta \,e^{\kappa \cos \theta }}$.

• Kent distribution, a related distribution on the two-dimensional unit sphere
• von Mises distribution, von Mises–Fisher distribution where p = 2, the one-dimensional unit circle
• Bivariate von Mises distribution
• Directional statistics

• Dhillon, I., Sra, S. (2003) "Modeling Data using Directional Distributions". Tech. rep., University of Texas, Austin.
• Banerjee, A., Dhillon, I. S., Ghosh, J., & Sra, S. (2005). "Clustering on the unit hypersphere using von Mises-Fisher distributions". Journal of Machine Learning Research, 6(Sep), 1345-1382.
• Sra, S. (2011). "A short note on parameter approximation for von Mises-Fisher distributions: And a fast implementation of I_s(x)". Computational Statistics. 27: 177–190. CiteSeerX 10.1.1.186.1887. doi:10.1007/s00180-011-0232-x. S2CID 3654195.