Recently, after deploying a web app on a dedicated server, we realized that the disk storage would be a future bottleneck to the business of the application.
It would require an exponential space during the time of app usage since it'll eventually be used in many simultaneous locations at once, while communicating with a central server uploading hundred of megabytes of assets.
We decided to move the storage away from our dedicated server to the Amazon AWS S3 to increase scalability of the application.
Easy stuff, we could say at this stage. Just switch some configuration in Paperclip in order to use S3.
Well, the problem was that the app was already in production, so the change required us to move all the existing assets to the S3 in order to prevent losing data (obviously!).
During our research we found an easy fix to migrate all the assets to S3. The solution all over the Internet is quite self-explanatory; just send the folder where your assets are stored to the S3 (the well-known public/system folder)!
This would work in a situation where the url of Paperclip configuration was previously defined. The default schema that Paperclip uses ends up in a file with the location like:
To prevent losing time figuring out where /000/000/ came from, the explanation for all the leading 0’s, and in order to create a flexible solution, we decided to make our own strategy.
We need to move all the model attachments;
We need to preserve all the styles;
This needs to be done without locking the server;
S3 bucket should be different according to Rails environments (test, production, staging);
We need to have minimum downtime.
First we need to be able to switch the configuration on Paperclip to use S3 after migration, and to use our server during migration (we need to get the attachments files).
Lets see a typical model with attachments:
To prepare the model to be migrated to the S3 service we need to add a switch that can be easily activated.
Lets create an s3_migrate.yml file with some configuration:
From line 1 until 4 is the normal configuration for the Paperclip S3 storage, in line 5 we added the switch to activate the S3.
If it's not activated, the application should serve the assets from the local server. Next we change our model definition in order to reflect these changes.
In this snippet we store the Paperclip options in a hash (PAPERCLIP_STORAGE_OPTIONS) and load the file containing the migration options. We merge the S3 settings if activated is true.
For now we've prepared our model to switch to S3 after the migration process is complete.
In order to migrate our assets to S3 we will need a custom task to perform the action. Here is the snippet responsible for it.
Create a file named paperclip_migrate.rake and put it on lib/tasks folder and paste the code above.
Run the task:
**After finishing the migration we need to update our s3_migrate.yml and activate the S3 storage. **
Just change the line activated to true. Restart the webserver and you are done!
Note 1: The path setting in s3_migrate.yml should be equal to the one in the line 44 of the task. Change that according to your scenario.
Note 2: After completing the migration and assuring that our assets were successfully migrated to S3 you can delete the files that are still stored in public/system.
Lad os gentage vores krav til dette problem og validere hver enkelt af dem.
Når vi tilføjer muligheden for at specificere klassen og objekterne dynamisk, kan man udføre opgaven med de modeller, der ønskes, eksempel:
Fuldført!
I vores opgave, efter at vi har kontrolleret, at der er mindst et objekt i vores database, kan vi forespørge det for at hente alle de definerede stilarter:
Det vil returnere, hvad vi forventer:
Fuldført!
Vi bruger en rake-opgave til at udføre migreringen, så den kan udføres uden at påvirke brugen af applikationen i produktionsmiljøet.
Fuldført!
I vores s3_migrate.yml vi definerer banen for aktiver for at bruge Rails.env variabel, så aktiverne gemmes i henhold til det miljø, som applikationen kører.
Fuldført!
Vi har kun brug for en enkelt genstart til webserveren! Så vi kan sige, at kravet var...
Fuldført!
Og det handler om alt hvad du behøver at vide for at migrere Paperclip-aktiver fra en lokal server til Amazon S3.
Har erfaring med backend-systemer til internettet ved hjælp af avancerede teknologier. Fuld stak-udvikler og produktproducent. Rubin/Rails, NodeJS, ElectronJS.
People who read this post, also found these interesting:
.webp)
.webp)
.webp)
.webp)
.webp)
.webp)
.webp)
.webp)
