Work

Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization

Year: 2015

Type: preprint

Abstract: Generalized Linear Models (GLMs), where a random vector $\mathbf{x}$ is observed through a noisy, possibly nonlinear, function of a linear transform $\mathbf{z}=\mathbf{Ax}$ arise in a range of applications in nonlinear filtering and regression. Approximate Message Passing (AMP) methods, based on loopy belief propagation, are a promising class of approaches for approximate inference in these models. AMP methods are computationally simple, general, and admit precise analyses with testable conditions for optimality for large i.i.d. transforms $\mathbf{A}$. However, the algorithms can easily diverge for general $\mathbf{A}$. This paper presents a convergent approach to the generalized AMP (GAMP) algorithm based on direct minimization of a large-system limit approximation of the Bethe Free Energy (LSL-BFE). The proposed method uses a double-loop procedure, where the outer loop successively linearizes the LSL-BFE and the inner loop minimizes the linearized LSL-BFE using the Alternating Direction Method of Multipliers (ADMM). The proposed method, called ADMM-GAMP, is similar in structure to the original GAMP method, but with an additional least-squares minimization. It is shown that for strictly convex, smooth penalties, ADMM-GAMP is guaranteed to converge to a local minima of the LSL-BFE, thus providing a convergent alternative to GAMP that is stable under arbitrary transforms. Simulations are also presented that demonstrate the robustness of the method for non-convex penalties as well.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.1501.01797

Authors: Rangan, Sundeep, Fletcher, Alyson K., Schniter, Philip, Kamilov, Ulugbek

Institutions: New York University, University of California, Los Angeles, The Ohio State University, École Polytechnique Fédérale de Lausanne

Language: English

Topic: Sparse and Compressive Sensing Techniques

Subfield: Computational Mechanics

Field: Engineering

Domain: Physical Sciences

Open Access status: green

Funders: National Science Foundation, European Commission

Awards: Message-Passing Strategies for High-Dimensional Inference, CIF: Small: Approximate Message Passing for Systems with Linear Mixing and Randomization, CIF: Small: Collaborative Research: Next Generation Communications with Low-Resolution ADCs: Fundamentals and Practical Design, CIF: Small: Soft Inference under Structured Sparsity

OpenAlex ID: W2950315427

Cites: 62

Cited by: 3

Related to: 20