Uploads in Amazon S3 or similar

Hi guys!

How do you manage sites with really lots of uploads? I’m talking about 10GB+ of images
Do you use a plugin? Which one?

I’ve found these:

Pro version of the last one: Upload Your WordPress Media to Amazon S3 with WP Offload Media - Delicious Brains Inc

I’m really concerned about the reliability of these solutions.

Well, I interested in reading your opinions on the topic.

Cheers!

1 Like

amazon-s3-and-cloudfront is the old name for Offload S3 (they rebranded it with a new name etc). We use it on all our sites.

WP-offload-S3 PRO version is great. (works great with Migrate DB pro also for migrating/sync images from local to server and other way around.)

I suggest you have the “remove uploads from server” turned off. So if the bucket gets deleted or if some plugins like WPML have troubles loading images from S3 (it happens), you have a fallback to the server image until you fix the problem etc. There may be other better solutions to this also :smile:

You may need the Pro version though as that one automatically does the magic for you. The free one only works for the images you upload, so the images you already have there don’t get to S3, but with the PRO version everything goes to S3 automatically with a click of a button (jpg,png,css,js,… ).

You can also email deliciusbrains as they’re solid guys and can give you some help if you have any questions/concerns before buying etc. Check their videos before sending email though :wink: https://deliciousbrains.com/wp-offload-s3/videos/

Also the quality of Offload is nowhere near those themeforest ones etc. so the product is hi-quality.

Hi @darjanpanic!

Thanks for pointing me to the videos, they are really clear. The thing is that I think the pricing schema is a little messed up. $399 is too much for that 100k uploads. Also, what about sites with more than 100k files? Do I need 2 licenses? May have to ask them, I’ll send them an email

I’m sure the quality is good, but Humanmade S3 Uploads looks good looks good too!
Shame I don’t have console access with the current provider setup, we are working on it… but not yet

Do you tested it with plugins like Imsanity and EWWW? We use those, and have been of great help with both keeping image res and weight at a reasonable point.

Their model is a bit different but they try to be fair with it so big sites pay more then smaller ones etc. But i think 1 image with different sizes (the ones that wordpress or woocommerce make after upload) counts as just 1 in their licence.

Didn’t test with those but they seem nice so may do some tests in the future, thanks! For the compression we used mostly https://wordpress.org/plugins/tiny-compress-images/ But it’s not free :confused: so with your quantities the others may work better

Yes, 1 image count as 1 image regardless how many resizes it has.

For compressing we are using EWWW, I highly recommend it. It uses TinyPNG and JPG API, but somehow they manage to be cheaper.

Hey @darjanpanic do you use Offload S3 on bedrock based sites? I’m a little concerned about the directory structure causing problems.

I think i did used it a while back, but i can’t say i’m 100% sure. You’d need to test it a bit,maybe with the free version of the plugin. Just in case :smile:

Yep… I’ll first ask them, then try and see what happens. I’ll let you know

Hi @fgilio - I’m in a situation where we have a managed WordPress host, who places a premium on storage. They use 2n + 1 HA for storage + offsite copies, so they charge a fortune per GB as each file you write is replicated 2, 3 or 4 times, so we too are looking at offloading to S3 (and the Delicious Brains plugin was the most attractive for us).

I guess the scary thing is: for it to be effective, you have to take a leap of faith in deleting the images from your web server, and then serving them only from S3. The reliability of S3 itself isn’t a problem for me; it’s pretty robust, and if you enable versioning, you have roll-back options. The issue for me could be the rewriting of URLs, and having faith this would work with all of the other moving parts of a WP site…

The other consideration for us is that when we draw up contracts for clients, we have to tell them: part of your site is on this super secure cluster with 2n + 1 HA, snapshots, off-site backups, x hour SLA etc etc, and then other parts of your site aren’t (ie some is on the web server, some is on S3). It’s not a problem, but just takes some explaining.

I have now found a plugin (Imsanity) which will resize images in the media library to fit within certain dimensions (which you could use in conjunction with EWWW, I guess), and I have updated my terms of service to say that images on the web server are for web use, and the web server isn’t a file repo. Images may be resized to these dimensions (which currently is a max of 2000x2000), so please ensure you keep the originals. I believe the plugin can also resize images on the fly as and when users upload them.

Whether this hard-line approach is for you, I don’t know - but I am very transparent about it, and explain the reasons and have had no push back so far.

I reduced 21GB yesterday to 5.5GB by doing this…

I still have to figure out a way of dealing with PDFs though :smile:

With regards to the S3 offloading, I would be inclined to go with Delicious Brains, and pay for support. As @darjanpanic said, their support is great (I use Migrate DB Pro and when I had a few niggles using earlier versions, they would go above and beyond to sort it out), so if you do go down this route, at least you’ve got some backup with them.

I, just wanted to let you know that I am using the free version of WP offload plugin (paid version is very new) on a rather big website with ~200k medias and it works really well…
Being backed up by Delicious Brain also helps in confidence for this plugin!

I then use Thumbr.io on top of it for CDN and image resizing, with no issue so far

Hello @doug!
I understand your situation, taking the decision to remove local files is not an easy one. I know Delicious Brains is very dependable, ant that makes me a little less concerned.

We’ve been using a combination of Imsanity and EWWW since the beginning, and those have proven to be an enormous help in keeping the media library at reasonable scales. Nice you reached so good results!

I’ve send an email to Delicious Brains some days ago, but still waiting reply. I may just buy a small license and just try it out.

I’m really interested in this kind of solutions. Is it like imgix ?
So you don’t declare any particular image size within WordPress, store only a full size copy and let Thumbr.io take care of all the resizing? Sounds just to good to be truth.

Yes, this is the thing I really hate about images in WordPress. I have many image sizes that are relevant to only one CPT and producing them all for every uploaded image is a total waste of CPU and hard drive…

Tools like Thumbr.io or Imgix let you create thumbnails on the fly, which is awesome.
I wrote a simple plugin that plug into the wp_get_attachment_image_src and crop the image to any size you pass as a second argument.

The only downside after a few months working with this setup is that you cannot test it locally of course! I would lob to have the same feature built in so I don’t rely on a third party tool, like Drupal was doing with ImageCache.

Could’t agree more…

Would you share that plugin? I’m super interested in services like Thumbr.io or Imgix, and would like to try it out.

Also, what do you mean you can’t test it locally?

Hi, sorry was on vacation :smile:

I added my plugin publicly on github : GitHub - thomascharbit/Thumbrio-CDN: Thumbr.io WordPress integration
It’s a bit raw, no proper description or documentation, but it is based on WP Plugin Boilerplate… The interesting part is in https://github.com/FreshFlesh/Thumbrio-CDN/blob/master/public/class-thumbrio-cdn-public.php

I guess you can use this to start your own thing!

Also, what do you mean you can’t test it locally?

I mean, because i am now relying on an external tool (Thumbr.io) for cropping images, my local development environment needs to also use the same setup (upload to amazon s3 + thumbr account) otherwise if i work locally with no internet access WordPress cannot use Thumbr and will use the standard thumbnails as a replacement. This usually breaks your HTML layout!

1 Like

Thank you!

I thought I’ll break locally, but if the only limitation is the need of an Internet connection it seems pretty usable.

Will totally use a service like Thumbr.io for a couple of coming projects.