🌍 1. Manifold Theory

• Manifold Theory: The idea that data can appear low-dimensional locally but actually lie in a high-dimensional space.
• Analogy: Humans think Earth is flat locally, but in reality, it’s round.
• In ML, this helps explain why data might look simple but be complex underneath.

📏 2. Manifolds & Distance

• On a manifold, Euclidean distance may give you the wrong shortest path globally.
• But KNN works well because it uses local distance search, which stays accurate on small neighborhoods.

⚡ 3. Curse of Dimensionality

• KNN struggles in high-dimensional spaces.
• As dimensions increase, distances between points become more uniform → it’s hard to find true “nearest” neighbors → accuracy drops.

📐 4. Uniform Distances

• In high dimensions, distances between points tend to even out.
• This makes the idea of “nearest” neighbors less meaningful — they’re all equally far apart!

🕒 5. KNN Time Complexity

• KNN has time complexity O(nd):
  • n: number of samples
  • d: number of dimensions