RC RANDOM CHAOS

E-means Delivers 10x K-Means Speedup With Lower Loss

· via Hacker News

Original source

E-means: 10x faster k-means at equal loss

Hacker News →

Meta FAIR has released e-means, an open-source clustering algorithm that outperforms Lloyd’s standard k-means approach by roughly 10x on convergence speed while achieving better final loss values. Compared to GPU and CPU implementations of state-of-the-art methods, e-means delivers 10–45× wall-clock improvements on GPU and 8–15× on CPU.

The core innovation replaces Lloyd’s full-batch centroid updates with exponential moving average updates on mini-batches. This allows older assignments to decay gradually rather than persist indefinitely or vanish completely each epoch. Automatic scaling laws determine optimal batch size and step size from dataset characteristics, eliminating manual hyperparameter tuning. Additional improvements include refined initialization, dead-cluster recovery mechanisms, cosine learning rate scheduling, and a finalization stage that further reduces loss.

The algorithm’s advantage stems from mini-batch updates introducing productive noise into optimization, helping escape local minima where Lloyd’s deterministic full-batch approach stalls. Testing shows e-means reaches convergence-quality loss in 1.2–3 epochs versus Lloyd’s 25. The method is theoretically sound—setting batch size to the full dataset and step size to one recovers Lloyd as a degenerate case, confirming the framework while demonstrating Lloyd operates far from optimal parameters.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.