Three weeks ago, I saw a nice mechanical recreation of a PCA (or total least squares) on Twitter: just by pulling some strings attached to a straw.
Some days ago, Joshua Loftus published Least squares as springs, where the author presents some nice visualisations, and explains that the cost function is the same (except for a constant) as the potential energy of springs attaching the data points to the regression line. As a result, if we take any line attached in this way to a point cloud (with the required constraints in place for the strings: vertical movement for regular regression; no constraints for PCA, the springs slide freely), then the system will oscillate until it reaches the state of minimum energy (i.e., meets the regression line).
Inspired by this and based on the Matlab code for the animation in this excellent StackExchange answer, I created a Shiny app that allows us to play with different parameteres:
- linear regression vs. PCA;
- covariance matrix for data generation;
- number of samples;
- initial angle and shift of the center of mass;
- velocity loss and inertia (which determines the damping ratio).
3 comentarios sobre “Least squares as springs, the Shiny app”
[…] by data_admin [This article was first published on R – Enchufa2, and kindly contributed to R-bloggers]. (You can report issue about the content on this page […]
[…] article was first published on R – Enchufa2, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here) […]
Total least squares or PCA is a special case of orthogonal distance fitting (ODF).
An example of ODF:
Fitting a Heart Curve
The latest example of ODF:
Cylindrical Bench Seat – Apple iPad Pro LiDAR