Taking Control of My Cloud Storage
So I have been someone who has been working with a lot of sync services over the years. Dropbox, Mega, Google Drive, and even open source solutions like Seafile, OwnCloud/NextCloud, and Sparkle Share. But these have all been keeping me locked in with paying for more than I need and the commercial ones paying with my privacy. So I wanted to look into a way of handling my files in a cheaper and more reliable way.
So you might be saying services like Mega give you 50gb for free or Dropbox 1TB for only 9/month. I would say you are right, but unless you are really using all that space it is not a good deal price wise. That said, it is also a not a great deal if you don’t trust third parties with your data. Then you might also start to point out, how paying for hosting your own solution can’t cost that much more, but then again it can cost a lot for a server and the disk space. You also have to worry about its backups and the drives if you are not on a VPS. So my idea is to use Object Storage, something that I find to be really the best option for my needs.
So I want to start out by saying, Object Storage is not going to be the best solution for everyone depending on your use cases. If you are someone who is going to be using a ton of storage and want it at a cheaper cost, Dropbox’s price might be a great option for what you are looking for. Also depending on the simplicity of your setup and if you are going to share files with others. So again, this is why I say my solution is not for everyone.
To start this journey I decided to look at all the options out there. AWS S3, Digital Ocean Spaces, Google Cloud Storage(not Google Drive), BackBlaze B2, and some of the smaller companies out there. I started to see that most people’s storage price for good redundant storage was around 0.023 per GB on average with the exception of BackBlaze B2 being the cheapest at 0.005 per GB and OVH being 0.0112 per GB. With this in mind, I realized I had to factor in another cost since I am going the route of object storage, which was how much my bandwidth was going to cost. Meaning while BackBlaze looked great for my storage, it was actually going to cost me more if I was going to be downloading a lot from it or using it to share files.
Diving into the costs, I found that OVH was actually the cheapest option, and the 0.01/gb bandwidth was a steal compared to the competition. I also happen know it was reliable because I have been hosting Zadaroo’s files on it for years now. The competition is just absolutely expensive averaging around 0.09/gb of outgoing bandwidth, which was just going to be too much money for me if I am also going to using it for syncing and sharing files with others. The other major bonus is that OVH doesn’t charge transactional action charges such as puts, gets, and deletes which most of the competition does. So OVH won out the on what I was looking for.
I created two buckets for security reasons, one bucket as a private bucket allowing only me access to the files stored on it. Then I created a static site option, as the public option creates an XML index that lists all your files and I didn’t want that. I wanted some control of what people can see and not just navigate to the directory and see all my files in the public bucket.
Since I run Linux everywhere now, I use RClone and created a cronjob that runs every 30 minutes on my machine to sync any changes up with the service. It does a copy to local if there is anything new, then syncs any local files up. Then for my files I want to share, I use the public bucket which also syncs via cron to the public static site bucket.
The best part of this is the customization and extra features I get doing this. I can use my own domain like files.example.com/something.txt to send it to a friend or post to link on a forum or embed a picture somewhere. I only pay for what I use, and since I am using less than a terabyte I am not going to pay but for my 0.0112 per gig of storage and 0.01 per gig of outgoing bandwidth. All my uploading to OVH is free, so that is a great perk that only outgoing traffic costs. I also love that the storage is replicated with an SLA of 99.9999% but that doesn’t mean I should consider also cloning these buckets to another provider if I want that extra security.
I will release another post soon going further into the scripts I created and going into how you can do this and encrypt your data in your private bucket so even your provider you choose doesn’t know what is in it. All using the tools I talked about in this post. I believe in this new cloud era of not using servers but trying to do stuff as server-less as possible (even though I don’t like that terminology I am still using servers just not the classic way) because it really can be much cheaper and if it isn’t the trade-off for the extra cost can be a benefit as well. I will have to pay 11.20 for 1Tb of storage instead of the 9, but I know it is replicated across the zone I picked in OVH’s object storage and that the bandwidth is more premium vs throttled to me getting a VPS that would only have 100Mbps connection that is shared with other VPS’s on the host, no need to worry about patching the server for vulnerabilities, all of that is handed off to OVH. Granted Dropbox is a way to go if you want that route, but if you like to go the route you can tinker and use your own domain and such, object storage is great.
( Picture provided by Thomas Hart https://flic.kr/p/acCF3G )