Gelman-Rubin statistic

The Gelman-Rubin statistic allows a statement about the convergence of Monte Carlo simulations.

Definition

${\displaystyle J}$ Monte Carlo simulations (chains) are started with different initial values. The samples from the respective burn-in phases are discarded. From the samples ${\displaystyle x_{1}^{(j)},\dots ,x_{L}^{(j)}}$ (of the j-th simulation), the variance between the chains and the variance in the chains is estimated:

${\displaystyle {\overline {x}}_{j}={\frac {1}{L}}\sum _{i=1}^{L}x_{i}^{(j)}}$ Mean value of chain j

${\displaystyle {\overline {x}}_{*}={\frac {1}{J}}\sum _{j=1}^{J}{\overline {x}}_{j}}$ Mean of the means of all chains

${\displaystyle B={\frac {L}{J-1}}\sum _{j=1}^{J}({\overline {x}}_{j}-{\overline {x}}_{*})^{2}}$ Variance of the means of the chains

${\displaystyle W={\frac {1}{J}}\sum _{j=1}^{J}\left({\frac {1}{L-1}}\sum _{i=1}^{L}(x_{i}^{(j)}-{\overline {x}}_{j})^{2}\right)}$ Averaged variances of the individual chains across all chains

An estimate of the Gelman-Rubin statistic ${\displaystyle R}$ then results as^[1]

${\displaystyle R={\frac {{\frac {L-1}{L}}W+{\frac {1}{L}}B}{W}}}$.

When L tends to infinity and B tends to zero, R tends to 1.

A different formula is given by Vats & Knudson.^[2]

Alternatives

The Geweke Diagnostic compares whether the mean of the first x percent of a chain and the mean of the last y percent of a chain match.^{[citation needed]}

Literature

• Vats, Dootika; Knudson, Christina (2021). "Revisiting the Gelman–Rubin Diagnostic". Statistical Science. 36 (4). arXiv:1812.09384. doi:10.1214/20-STS812.
• Gelman, Andrew; Rubin, Donald B. (1992). "Inference from Iterative Simulation Using Multiple Sequences". Statistical Science. 7 (4): 457–472. Bibcode:1992StaSc...7..457G. doi:10.1214/ss/1177011136. JSTOR 2246093.

References

1. Peng, Roger D. 7.4 Monitoring Convergence | Advanced Statistical Computing – via bookdown.org.
2. Vats, Dootika; Knudson, Christina (2021). "Revisiting the Gelman–Rubin Diagnostic". Statistical Science. 36 (4). arXiv:1812.09384. doi:10.1214/20-STS812.