In this thesis, we tackle the fundamental problem of how to effectively and reliably calculate sparse solutions to underdetermined systems of equations. This class of problems is found in applied mathematics, electrical engineering, statistics, geophysics, just to name a few. This dissertation concentrates on developing efficient and robust solution algorithms, and applies them in several applications in the field of signal/image processing.
The first contribution concerns the development of a new Iterative Shrinkage algorithm based on Surrogate Function, ISSF-K, for finding the best K-term approximation to an image. In this problem, we seek to represent an image with K elements from an overcomplete dictionary. We present a proof that this algorithm converges to a local minimum of the NP hard sparsity constrained optimization problem. In addition, we choose curvelets as the dictionary. The approximation obtained by our approach achieves higher PSNR than that of the best
K-term wavelet (Cohen-Daubechies-Fauraue 9-7) approximation.
We extends ISSF to the application of Morphological Component Analysis, which leads to the second contribution, a new algorithm MCA-ISSF with an adaptive thresholding strategy. The adaptive MCA-ISSF algorithm approximates the problem from the synthesis approach, and it is the only algorithm that incorporate an adaptive strategy to update its algorithmic parameter. Compared to the existent MCA algorithms, our method is more efficient and is parameter free in the thresdholding update.
The third contribution concerns the non-convex optimization problems in Compressive Sensing (CS), which is an important extension of sparse approximation. We propose two new iterative reweighted algorithms based on Alternating Direction Method of Multiplier, IR1-ADM and IR2-ADM, to solve the ell-p,0
