Originally posted on the 8th Light Blog
Imagine this:
You’re moving an application from a shared server to a cloud system. This application has a massive folder of files saved onto the server that is growing constantly. When you get to it, it is about thirty-five gigabytes.
You could just move the application to a single server with an Elastic IP Address on AWS. You could. I mean, you think that’s the only way out, right? You can’t have multiple servers in a load balancer, which you want, for an application that needs to modify a shared set of files. Depending on the person, they’d get different content.
And, no, you can’t change the application code and have the files be saved to S3.
So you have to do a single server, right? How do you even calculate the EBS volume size on a server with a file system that is continuously growing? Do you just hope you’ll move the files one day to S3 and you won’t have to worry about running out of disk space on the primary production server? Do you create a script that somehow monitors the disk space? Do you push for the need to move to S3 as before that was not an option?
Good news: Those questions don’t have to be answered! There is salvation and it is AWS’s Elastic File System.
Amazon’s Elastic File System is basically a way to create a scalable file system and be able to mount it to multiple servers. The files are being saved to a place where it’ll never run out of storage and you get to distribute the application across multiple servers and have them sharing the same file set. Magic! Magic with a caveat: Now that the files are being shared between multiple servers, the possibility of concurrency issues comes up. If you’re going to use EFS, you need to make sure that the servers don’t conflict with each other’s writes and edits to the shared file system. If that’s not an issue, go for it: mount that EFS and let that shared state bring you to the next level of the cloud.
And that’s what I did. The above anecdote was my introduction with EFS and what made me a believer in it.
There are times when you need a place to store files as if it were a file system either across multiple servers or as merely a scalable solution. Times when you can’t change how the files are being saved. You can only change where they are being saved. Yes, it would probably be better to save the files to S3, but not all of us have that luxury. Sometimes, we can’t change the implementation of a system; but that doesn’t mean we can’t increase the scalability of a system. It just means we have to get more creative with how we do it.
Creative that time was EFS. I’ve since added EFS to my AWS tool set. By being aware of tools like EFS, the implementation details for software take on new dimensions. Workarounds don’t have to exist if the infrastructure fits the software. There are problems with software that do not have to be solved with code or even have to exist in the cloud. It can be solved with making good use of cloud infrastructure, such as EFS.
You’re in the cloud: use the cloud.