IPFS Cuts Content Publishing From 20 Seconds to Under 1 With Optimistic Provide

IPFS has long suffered from slow content publishing on its Amino DHT, where announcing a new piece of content to the network could take dozens of seconds and occasionally more than two minutes. The bottleneck lies in the Kademlia “provide” operation, which walks the DHT to locate the 20 peers closest (by XOR distance) to a content identifier, then pushes the provider record to all of them. The traditional algorithm stubbornly waits for responses from the three closest peers it finds, and in a churn-heavy permissionless network those peers are frequently offline, forcing repeated backtracking even when the real closest peers have already been discovered.

ProbeLab’s fix, called Optimistic Provide, swaps rigid waiting for statistical heuristics and shipped as the default in Kubo 0.39.0 after being published at IEEE INFOCOM 2024. It rests on three ideas: each node locally estimates the total network size, uses that estimate to stop the DHT walk and store records as soon as it is roughly 90% confident it has found the right peers, and hands control back to the publisher once a subset (around 15 of 20) of peers confirm storage while the remaining writes finish asynchronously. The network-size estimate is derived cheaply by piggybacking on the routing table refresh, treating the peer distances returned by each lookup as samples of a known statistical distribution, with a bias correction that downweights results from sparse or dense keyspace regions.

The payoff is a roughly tenfold speedup: median publish latency drops from around 13 to 20 seconds to under one second, while cutting network overhead by about 40%. In practice that means near-real-time publishing, letting developers and applications iterate and debug against freshly announced content instead of waiting through multi-second delays. Because the estimation reuses data nodes already collect, the gains come at zero additional networking cost.