Generalized linear array model

In statistics, the generalized linear array model (GLAM) is used for analyzing data sets with array structures. It based on the generalized linear model with the design matrix written as a Kronecker product.

Overview

The generalized linear array model or GLAM was introduced in 2006.^[1] Such models provide a structure and a computational procedure for fitting generalized linear models or GLMs whose model matrix can be written as a Kronecker product and whose data can be written as an array. In a large GLM, the GLAM approach gives very substantial savings in both storage and computational time over the usual GLM algorithm.

Suppose that the data ${\displaystyle \mathbf {Y} }$ is arranged in a ${\displaystyle d}$-dimensional array with size ${\displaystyle n_{1}\times n_{2}\times \dots \times n_{d}}$; thus, the corresponding data vector ${\displaystyle \mathbf {y} =\operatorname {vec} (\mathbf {Y} )}$ has size ${\displaystyle n_{1}n_{2}n_{3}\cdots n_{d}}$. Suppose also that the design matrix is of the form

${\displaystyle \mathbf {X} =\mathbf {X} _{d}\otimes \mathbf {X} _{d-1}\otimes \dots \otimes \mathbf {X} _{1}.}$

The standard analysis of a GLM with data vector ${\displaystyle \mathbf {y} }$ and design matrix ${\displaystyle \mathbf {X} }$ proceeds by repeated evaluation of the scoring algorithm

${\displaystyle \mathbf {X} '{\tilde {\mathbf {W} }}_{\delta }\mathbf {X} {\hat {\boldsymbol {\theta }}}=\mathbf {X} '{\tilde {\mathbf {W} }}_{\delta }{\tilde {\boldsymbol {\theta }}},}$

where ${\displaystyle {\tilde {\boldsymbol {\theta }}}}$ represents the approximate solution of ${\displaystyle {\boldsymbol {\theta }}}$, and ${\displaystyle {\hat {\boldsymbol {\theta }}}}$ is the improved value of it; ${\displaystyle \mathbf {W} _{\delta }}$ is the diagonal weight matrix with elements

${\displaystyle w_{ii}^{-1}=\left({\frac {\partial \eta _{i}}{\partial \mu _{i}}}\right)^{2}\mathrm {var} (y_{i}),}$

and

${\displaystyle \mathbf {z} ={\boldsymbol {\eta }}+\mathbf {W} _{\delta }^{-1}(\mathbf {y} -{\boldsymbol {\mu }})}$

is the working variable.

Computationally, GLAM provides array algorithms to calculate the linear predictor,

${\displaystyle {\boldsymbol {\eta }}=\mathbf {X} {\boldsymbol {\theta }}}$

and the weighted inner product

${\displaystyle \mathbf {X} '{\tilde {\mathbf {W} }}_{\delta }\mathbf {X} }$

without evaluation of the model matrix ${\displaystyle \mathbf {X} .}$

Example

In 2 dimensions, let ${\displaystyle \mathbf {X} =\mathbf {X} _{2}\otimes \mathbf {X} _{1}}$, then the linear predictor is written ${\displaystyle \mathbf {X} _{1}{\boldsymbol {\Theta }}\mathbf {X} _{2}'}$ where ${\displaystyle {\boldsymbol {\Theta }}}$ is the matrix of coefficients; the weighted inner product is obtained from ${\displaystyle G(\mathbf {X} _{1})'\mathbf {W} G(\mathbf {X} _{2})}$ and ${\displaystyle \mathbf {W} }$ is the matrix of weights; here ${\displaystyle G(\mathbf {M} )}$ is the row tensor function of the ${\displaystyle r\times c}$ matrix ${\displaystyle \mathbf {M} }$ given by^[1]

${\displaystyle G(\mathbf {M} )=(\mathbf {M} \otimes \mathbf {1} ')\circ (\mathbf {1} '\otimes \mathbf {M} )}$

where ${\displaystyle \circ }$ means element by element multiplication and ${\displaystyle \mathbf {1} }$ is a vector of 1's of length ${\displaystyle c}$.

On the other hand, the row tensor function ${\displaystyle G(\mathbf {M} )}$ of the ${\displaystyle r\times c}$ matrix ${\displaystyle \mathbf {M} }$ is the example of Face-splitting product of matrices, which was proposed by Vadym Slyusar in 1996:^[2][3][4][5]

${\displaystyle \mathbf {M} \bullet \mathbf {M} =\left(\mathbf {M} \otimes \mathbf {1} ^{\textsf {T}}\right)\circ \left(\mathbf {1} ^{\textsf {T}}\otimes \mathbf {M} \right),}$

where ${\displaystyle \bullet }$ means Face-splitting product.

These low storage high speed formulae extend to ${\displaystyle d}$-dimensions.

Applications

GLAM is designed to be used in ${\displaystyle d}$-dimensional smoothing problems where the data are arranged in an array and the smoothing matrix is constructed as a Kronecker product of ${\displaystyle d}$ one-dimensional smoothing matrices.

References

1. Currie, I. D.; Durban, M.; Eilers, P. H. C. (2006). "Generalized linear array models with applications to multidimensional smoothing". Journal of the Royal Statistical Society. 68 (2): 259–280. doi:10.1111/j.1467-9868.2006.00543.x. S2CID 10261944.
2. Slyusar, V. I. (December 27, 1996). "End products in matrices in radar applications" (PDF). Radioelectronics and Communications Systems. 41 (3): 50–53.
3. Slyusar, V. I. (1997-05-20). "Analytical model of the digital antenna array on a basis of face-splitting matrix products" (PDF). Proc. ICATT-97, Kyiv: 108–109.
4. Slyusar, V. I. (1997-09-15). "New operations of matrices product for applications of radars" (PDF). Proc. Direct and Inverse Problems of Electromagnetic and Acoustic Wave Theory (DIPED-97), Lviv.: 73–74.
5. Slyusar, V. I. (March 13, 1998). "A Family of Face Products of Matrices and its Properties" (PDF). Cybernetics and Systems Analysis C/C of Kibernetika I Sistemnyi Analiz. 1999. 35 (3): 379–384. doi:10.1007/BF02733426. S2CID 119661450.