absolutedestiny: (Default)
[personal profile] absolutedestiny
So [personal profile] morgandawn  asked me to write up this guide as it may be helpful to people who:
  • Create fan art that needs hosting before being able to post on ao3 or other places (images, videos, audio files etc)
  • Need a place they can upload those media files that isn't youtube etc where take-downs are automated
  • Is the cheapest trustworthy place I can find (like, for a lot of you this might be effectively free)
  • Doesn't require training as a sysadmin to set it up
  • Allows porn (backblaze just say nothing illegal, bunnyCDN specifically say they allow it as long as it's legal)
This guide is just one way of doing this. Other people may have other ways, this is mine. It will not stop you from take-downs - every web host has some kind of copyright infringement take-down request process but this method doesn't automate that like youtube.

Edit: 22/12/18 - If you don't need direct linking to the files then using a service like dropbox, google drive, mega etc etc may still be fine for you. Those services are very competitively priced and if they work for you then feel free to use them.

Also, this isn't a general purpose web hosting guide for people that want to make their own website. That would be a different guide.  This is just for media files that you would link to on sites like this or twitter or ao3. It also doesn't rely on any kind of traditional website service like shared hosting offered by services like Dreamhost. Most of those services require media files you store on it to be used on a website your are hosting with them - so if you are hosting the files just to post on ao3 then they might shut down your access to the service.

This guide is for using services designed to store and host media files on the web.

If there is a fan with a server who is hosting your content, that is wonderful. Send them a big thank you card from the whole community.

If you don't have that, read on.

Some Terminology

In this guide I'm going to be referring to Cloud File Storage, sometimes called Block Storage. This is the place files are uploaded to. Reading this you may think of services like google drive, dropbox, box.net and so on. While these are cloud file storage solutions they do not offer easy means to provide access to those files on websites as they are more personal or business use solutions (e.g. the dropbox file download page instead of being able to directly link to the file).

The kind of cloud storage in this guide is the kind people making their own websites would use for hosting media assets. A popular service is Amazon's S3. However, S3 is not the cheapest and it's definitely not the most user-friendly. That's the kind of service that we'll use to store the files but I'm recommending a competitor.

I'll also talk about a CDN - a Content Delivery Network. CDNs are designed to mirror files stored somewhere and put them on a number of servers around the world so that when someone wants to download that file it is served up from a server near them. They are designed for the downloading part of hosting media. In this guide it's an optional step but can save you money if you expect to have a lot of people download your files.

Part 1 - Storing the Files on the Cloud with Backblaze B2

I've chosen Backblaze B2 for a number of reasons. It's very competitively priced, they are a well trusted company and they have a web interface for uploading files.

B2 charge based on storage and downloads. For storing files, they charge $0.005 per GB and the first 10GB is free. To put that into context, all the videos on luminosity's website total 6.25GB so she would not need to pay anything for storage until she made a lot more vids. Even if they didn't have a free tier, at $0.005 per GB that would only be 3 cents a month.

So, effectively free.

However, the real costs of media hosting isn't generally the storage part, it's the downloading part. That's where things actually start to cost money. B2 charges $0.01 per GB downloaded and the first 1GB every day is free. What does that mean in real terms? Well, it depends on how popular your content is honestly. My vid site gets downloads of around 40GB a month, so 40 cents a month or basically $5 a year(!). That's not even the real price because I'd only be paying for bandwidth in excess of 1GB each day, so it can be < $2 a year for my site.

Download pricing can be risky, though - if a big file gets linked somewhere and tens or hundreds of thousands of people download it, then you have a problem and could end up with a surprising bill. If 100,000 people in one month downloaded Scooby Road via B2 it would cost $407 for that one (big) vid. Thankfully, Backblaze have the option to set usage caps to avoid big charges which would protect you in the situation where suddenly the whole world wants a file of yours. You can set how much you are comfortable paying for a day of downloads and it will disallow downloads once that triggers.

1.1 Setting up a Backblaze B2 bucket

When you sign up for a backblaze account it actually gives you access to a number of their services. This can be confusing because they also offer a cloud backup service - a personal storage service for offline backups of your computer. That's not what we want (though it is a great service for online backups if you dont have a solution for that yet). For our media files, want B2 Cloud Storage.

Under B2 Cloud Storage, create a new bucket. Buckets are like hard drives, they will be where we store our media. Their names will also be in the external url so choose sensibly:

Creating a new b2 Bucket

Your bucket will need to be public - unlike their personal storage offering, the files will not be encrypted so Backblaze will be able to see your files. The world will also be able to see them if they know the url for a specific file but wont be able to see a directory of all your files, they'd need the url.

Once you've made a bucket you can upload files from the website. You can create folders as well for organization - you can't easily move files around after you upload them so if you are going to organize your files make the folders first.

B2 bucket file upload
Once you have uploaded a file, clicking the (i) icon next to a file will show you the friendly url that you can use to link to that file on a website:

b2 file url

Congratulations, you now have your media stored and hosted!

If you don't want to use the web interface there are a number of programs that can handle the uploads for you, such as FileZilla and some others.

1.2 (optional) Advanced Settings - CORS

You can probably skip this for most use cases. CORS is a complicated concept to do with giving permissions for websites on other domains (so sites not backblazeb2.com) to access the content in particular ways. This isn't needed for linking to a jpg or mp4 or mp3 or even embedding the file as an <img> or as a <video>. If a site is set up a certain way, it may need the ability to have access the file via javascript or some other means other than just showing it to the user. This is rare but if you need to set up access you can do that on the CORS settings for the bucket. You can give all sites full access to the files or you can choose which domain can. Most of you will never need to set this up, but know that it's there as an option for your bucket.

This is not the same as hotlinking protection which is not something b2 supports but is a common feature of CDNs. On that subject...

Part 2 (optional) - Using a CDN for really high traffic sites and finer control

The way a CDN works is that you give it a url (what they call a Pull Zone) and make a cdn version of that url. Then any media files that use the cdn url instead of the b2 one will download from the cdn. The CDN will then cache the file for the next person and use servers around the world to make seeing the file quicker. That's the idea, at least.

If you already know that you're going to have a ton of traffic to your files, you can mirror them on a CDN for even cheaper download prices. BunnyCDN have a high volume tier that instead of $0.01 per GB becomes $0.005 per GB (and is even cheaper once you hit 4 terrabytes!). They are the cheapest I could find.

Needing to use this CDN for the cheaper pricing will be a tricky calculation because BunnyCDN doesn't have a daily free tier like B2 does. This means that BunnyCDN will actually be more expensive (though still cheap) on days where people download less than 2GB. You also have to specifically use their High Volume tier as their regular tier is priced the same as b2 at $0.01 per GB. But, if you are having more than 2GB of downloads per day, BunnyCDN will save you money.

Obviously this wont be for everyone but setting up a CDN does have other benefits that might be worth not having a free download tier.

You can set up allowed referrers so, for instance, if you only want your images to show on https://archiveofourown.org and your personal website you can do that

You can set up your own domain name, if that's a thing you want to do. You can also have a lot more control over rate limiting, blocking countries, all kinds of things like that.

It might also offer an administrative barrier for copyright takedowns. If BunnyCDN get a takedown notice and suspend your account, you can look for another CDN or just fallback to using backblaze as all your content is still there. (*** I don't know this for sure because it's not happened to me - it is possible that BunnyCDN send along the information about where they are pulling the files from which might mean that b2 would also get a takedown request I don't know I'm just guessing).

Rather than step you through the setup process, you can read their own guide for setting up a CDN for a backblaze b2 bucket here.

Just know you need to make a High Volume pull zone based on your b2 friendly url. You will then get a cdn url you can use when posting your files.

That's all, folks!
Identity URL: 
Account name:
If you don't have an account you can create one now.
HTML doesn't work in the subject.


If you are unable to use this captcha for any reason, please contact us by email at support@dreamwidth.org

Notice: This account is set to log the IP addresses of everyone who comments.
Links will be displayed as unclickable URLs to help prevent spam.


absolutedestiny: (Default)

February 2019


Style Credit

Page generated Apr. 21st, 2019 10:14 pm
Powered by Dreamwidth Studios

Expand Cut Tags

No cut tags