CLSP: Linear Algebra Foundations of a Modular Two-Step Convex Optimization-Based Estimator for Ill-Posed Problems

This paper develops the linear-algebraic foundations of the Convex Least Squares Programming (CLSP) estimator and constructs its modular two-step convex optimization framework, capable of addressing ill-posed and underdetermined problems. After reformulating a problem in its canonical form, \(\mathbf{A}^{(r)} \mathbf{z}^{(r)} = \mathbf{b}\), Step~1 yields an iterated (if \(r > 1\)) minimum-norm least-squares estimate \(\widehat{\mathbf{z}}^{(r)} = (\mathbf{A}_{\mathbf{Z}}^{(r)})^\dagger \mathbf{b}\) on a constrained subspace defined by a symmetric idempotent \(\mathbf{Z}\) (reducing to the Moore-Penrose pseudoinverse when \(\mathbf{Z} = \mathbf{I}\)). The optional Step~2 corrects \(\widehat{\mathbf{z}}^{(r)}\) by solving a convex program, which penalizes deviations using a Lasso/Ridge/Elastic net-inspired scheme parameterized by \(\alpha \in [0,1]\) and yields \(\widehat{\mathbf{z}}^*\). The second step guarantees a unique solution for \(\alpha \in (0,1]\) and coincides with the Minimum-Norm BLUE (MNBLUE) when \(\alpha=1\). This paper also proposes an analysis of numerical stability and CLSP-specific goodness-of-fit statistics, such as partial \(R^2\), normalized RMSE (NRMSE), Monte Carlo \(t\)-tests for the mean of NRMSE, and condition-number-based confidence bands. The three special CLSP problem cases are then tested in a 50,000-iteration Monte Carlo experiment and on simulated numerical examples. The estimator has a wide range of applications, including interpolating input-output tables and structural matrices.