The First Principal Component
PCA is the most popular dimensionality reduction technique (IMHO it is basically a data transformation technique, viewing the same data using a better choice of coordinate system). In this write up, we're going to clear a common misconception about the very first principal component.
In almost every lecture or article explaining PCA, they refer to the first principal component as the 'line of best fit' for the data (not that I completely disagree). They mention the above statement as a passing remark, but 'line of best fit' is a hyperbole we commonly use in the context of OLS, do they actually mean it in that sense? (some people do mistake it to be in the sense of OLS method, which is wrong on so many levels). This statement warrants a clear explanation without which it can lead to serious misconception. Lets first see where this particular idea stems from.
You can find the full writeup and notebook here https://www.linkedin.com/posts/fazil-mohammed-4062711b2_is-the-first-pc-same-as-ols-best-fit-line-activity-6940892365851099136-jLGs?utm_source=linkedin_share&utm_medium=member_desktop_web