Category Archives: Memos
Since long ago I wanted to write a post with an elementary introduction to soft / hard thresholding. The time has come to pay the dues, so here we go. Advertisements
A very nice reference for this post is [this blog post]. Mirror Descent is a first-order algorithm to solve a convex optimization problem In this post we will learn several equivalent formulations of this algorithm and discuss how they are related to each other.
Found a great overview of the subject with a lot of insight. Another brief intro is here.
Here is a recap of proper normalizations for the Fourier series/DTFT (i.e. torus/ domains), DFT (), and also a brief reminder of Plancherel’s theorem.
A useful little fact for constrained optimization. Let be a convex set, and consider and its projection on As quickly follows from the separation theorem with hyperplane containing for any the angle between and is obtuse. … Continue reading
If a function has a Lipschitz gradient, i.e. for any and then