Preventing cache stampede with share buttons

I recently purchased the Roots share buttons to see how you are grabbing and displaying share counts.

The approach in your share buttons is very similar to what we use on our site. Here’s a Gist

  1. create request URLs
  2. request share counts from each network
  3. json decode & store the result in a transient
  4. render server-side using transient data

The approach has worked very well for us, with one exception. A few months back, Facebook had a massive outage for over an hour. That is longer than our 15 minute transients. Once the transients started to expire, we got a cache stampede that took the server down with seg faults. I’m guessing that your script would have suffered the same fate.

We have a relatively high traffic site that can have 500-1000 active visitors at any given point in time, but it is a news site, so traffic tends to concentrate on a small number of recent articles. On each page load, the share count would try to retrieve a transient, which needed to be refreshed. The subsequent wp_remote_get calls would pile one on top of the other while they all waited for their timeout, thus incapacitating the server.

Reason for my post is that I’m wondering if you have thoughts on how to prevent such a cache stampede. It wouldn’t be seen at all on low traffic sites, but it was super painful when it happened to us on a high-traffic site.

Perhaps you could not have the transient expire. Set up a script that is called every 15 minutes via cron. Have the script do the API call. If the call is successful, delete the old transient and create the new one.

1 Like

Interesting idea, but not sure how that would work. How does the cron script know which post share counts need to be updated? That is why we do it when the article loads.

We’ve got 20k+ articles and growing, but on any given day about 5000-8000 get shown. Getting share counts for the non-rendered ones would be much heavier than it needs to be. Many older posts may only get shown 1-2 times in a day, while new posts could get shown 1k, 10k or 50k times

A cron job would be my go to solution too.

The alternative is to create a lock file when the transient is being updated and then delete it once the process is complete.