Duration 22:17
16+
Play
Video

RailsConf 2019 - How to migrate to Active Storage without losing your mind by Colleen Schnettler

Colleen Schnettler
Owner at Bitmapped Designs
  • Video
  • Table of contents
  • Video
RailsConf 2019
May 2, 2019, Minneapolis, USA
RailsConf 2019
Request Q&A
Video
RailsConf 2019 - How to migrate to Active Storage without losing your mind by Colleen Schnettler
Available
In cart
Free
Free
Free
Free
Free
Free
Add to favorites
2.54 K
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About speaker

Colleen Schnettler
Owner at Bitmapped Designs

I am the owner and primary developer at Bitmapped Designs, a full service software consulting company. I specialize in creating mobile responsive applications built with Ruby on Rails. I build everything from forward facing websites to enterprise software that simplifies your business and your life. I can also consult on existing Rails applications.Prior to starting my business I worked as a hardware and software systems integration engineer in the shipbuilding industry. My work focused on mechatronic and IoT devices for shipboard use.

View the profile

About the talk

RailsConf 2019 - How to migrate to Active Storage without losing your mind by Colleen Schnettler


Active storage works seamlessly in new rails applications - but how many of us only work on new applications? Migrating to Active Storage can be a daunting task on a production application. This talk will explain active storage, why you might want to use it, how it modifies your database, and the benefits and drawbacks of migrating your existing application. I’ll walk you through my painful journey migrating an existing application. You will leave this talk with a better understanding of the inner workings of active storage and with the confidence to tackle your own migration. This talk is appropriate for all levels of skill and no prior experience or knowledge of active storage is required.

Share

Hi everyone. How about this amazing railsconf 2019? I hope you I hope you all have had an amazing week like I have and I really appreciate you coming to my talk cuz I know you're probably tired. My name is Colleen and I run a Ruby on Rails Consulting business. I'm here today to teach you a little bit about my Adventures or Misadventures as they were migrating a production application from Shrine to active storage using Amazon S3 storage. Actually you Shrine when I

did this for my client, but for the purposes of this talk, I'm going to use paperclip because the five people that responded to my Twitter poll said that use paperclip more than trying. But first I'd like to start with a little story. So, how did I get here? I was contacted by a cool new startup looking for a rails developer to do just that migrate their solution from Shrine to at the storage. I was excited to work with this company. And this was the first time I was going to get to use active storage and I was very excited to use active storage. I had

actually attended the active storage talk. I believe it was railsconf last year. So I was feeling quite confident in my ability to migrate this application to active storage. So for those of you who are not yet on Rails 5.2, let's start with what is active storage. So active storage is an easy way to attach files to active record objects in store those files in cloud-based storage. Have you ever needed to add an avatar to a user or maybe a resume to an applicant active storage helps you take care of all of

those file attachment needs. Well, that's great Colleen, but paperclip is working fine for me. Why should I go through the trouble of switching? That's a good question. Why should you migrate to access storage? Well the first and possibly most important reason is because active storage is now the built-in solution for handling file uploads to cloud storage and rails. Supports Amazon Google and Microsoft and this one's fun. There's no additional migrations needed. Maybe if you remember with paperclip every time you add a new

file, you have to write a new migration at the store just different. It doesn't work that way. And if I still haven't convinced you paperclip is deprecated. So you're out of luck. So I stopped at the contract. And the first thing I did was I went and looked at the at the storage docks. So in my experience the documentation for rails is usually excellent and active stores appear to be no different Step 1 install active storage step to configure cloud storage step 3, add an attachment to a model and step for

let the magic of rails extrapolate away all of the heavy lifting for you and it just works. Well, has anyone tried to migrate an application to active storage following these steps if you have tried? You might know that implementing active storage in a new application. You can follow the steps and it is relatively easy, but migrating to active storage can be quite challenging. Why is that? Well at the storage is fundamentally different from paper clip. People could work

by attaching file data to the user table. So for example here we have an avatar on a user. So if we added an avatar to our user using paperclip, it's going to change the users table it add these four columns your users table. I didn't include the whole table here. So you could actually see what paper clip does at the store just different. Active storage Prius two new tables the active storage attachments table me active storage blobs table. So if we revisit our steps. I'm going to say that step 3

had an attachment to a model will at the storage is not going to be able to access the data since there's currently nothing in your access storage tables. But we can't do stuff through yet, but we can do Step 1 and step 2. So Step One is install active storage create the tables. And then you need to configure your cloud storage. So the way this is set up right right here is we have an Amazon which is going to be our production storage and Amazon Dev, which is our death storage. I created this little contrived example for this talk so you can see I came up with a very clever bucket name there a

really fun bucket for Colleen which was unique. So do me, but when we do this on our production application, this is how we had it set up an as well and it's really going to depend on your setup, but I would highly recommend testing this on a Dev bucket on your cloud storage provider. and after you can figure it in storage. Yemul you then have to configure it on a / environment basis. So what I'm showing you here is development. And as I said, you're going to configure configure it to use Amazon death and production would be using Amazon. Okay, great. So

that took like 1 minute. So at this point you already have active storage installed and now your access words tables exist in your database. So let's talk about step 3. I have change step 3 to say move Avatar data from the user table to the active storage tables. Well, how do we move data from one table to another in our database? All right text so we are going to write a rape test together and let's talk about this right tasks. We're going to be moving a good amount of data and we're not it's not a one-to-one as we have one user table and to active

stores table. So we're also going to be mapping some data. So the only way to make this work is to understand what we are doing. I don't really think there's a copy and paste solution for this particular problem. Do let's talk a little more about what we are trying to do. So we are moving the state up. From users table. I'm going to show you again. to the active storage attachments inactive storage blobs, and we're technically copying it over there for now, but I'm so I don't know about you, but I find reaching

into my database with SQL to change records on a production application to be a little bit scary. Plus I was told I was going to have to write SQL. This is from last year. It seems to be the case here. So before we jump into what the rake task is going to be let's talk about the attic storage table because as I said, you really need to understand what you're doing here. So the first table I want to talk about is the active storage attachments table. We're going to start with the name,

which is the name of your attachment in this case Avatar. Then you have your polymorphic Association column user and the user ID and then you have your blob ID. Hey set, the table one now table to his blob table. So we look at the blobs table. The key is the location of your current file and Amazon S3 storage. And then you have your file name your content type. Bite-size. Skip that one All right. So how do these tables relate to one another? So I'm going to do one table at a time. So on your left is the users table and on your right is the active storage attachments

table. So user becomes a record type the eye becomes our record ID. And the name becomes just Avatar. Okay. So now I have users table user table on the left and the blobs table on the right and we have Avatar file name that's from our user table is going to go to a blob as the filename Avatar content type is going to go to the content type and file size is going to go to bite size. So let's get started on that rig tasks. So the good people of thoughtbot put together the skeleton of a task. That's an excellent starting place. As I mentioned they actually

using migration as I mentioned. I would recommend using a rate ask so if we look at this. If we look at this we get our blob ID and then these two. Statements or just defining our insert statements. So this is actually all pretty cut and paste for you. After that What's Happening Here Is We're looping through all of the models and pulling out the attachment names. The important thing to realize here is this code that's used to pull out. The attachment name is specific to paperclip cuz that's how paperclip

names the files on your user table. Right? So that's what we're looking at right there Avatar underscore file underscore name. So we do this is the same side the same piece of so if you look at this that had a specific. Just pulling on your avatar string in that is specific to paperclip. So as you were going through depending on what gem you are migrating from you have to be aware of this and all this is doing is pulling out the string Avatar. so once we get the attachment names the next step is to Loop through

the models and their Associated attachments. And as a side note, but I wanted to share if you only have one or two models with attachments or one model with one attachment. You don't have to do all of this. You can just call out the model and the attachment name instead of looping through every single model looking for attachments. so the thing I wanted to show you here is this the reason I want to show you this is this is instant, which is just the instance

of your user. So this is instance our taxes Avatar in our example. So user. Avatar. Passed out blank that statement is dependent on the relationship paperclip creates between user in Avatar. That is important. It's important because This is going to take two deploys. So, why does this process required to two boys will the raped a square building right now needs that user. Avatar relationship defined by paper clip. I just showed you in circle and Nez active storage

tables cuz it's it needs a place to put to move the data to to put the data so nice to put the data in the attic storage tables. Now active storage needs data in the active storage in the active storage table. So you can't run active storage without first running the right tasks and raped path is dependent on paper clip and we will revisit this right. So let's go back to our right tack. This right here is okay. So this is just calling her blob insert statement. And the thing I wanted to point out here are the

key and checksum Method and the other ones are just you know, user. Avatar filename content type file size, but I want to call out the key and check some for a few reasons. You going to have to write these methods yourself? I didn't actually include my solution cuz your solution is going to be so specific to your paper clip configuration and your Amazon S3 configuration. And they okay. So the key. Is where at the storage is going to look for your files as a funny or frustrating aside, depending on how you want to look at it. I was using

paperclip. So I assume the key would be a user. Avatar. Path. So that's what I put in my Redcat. Well, maybe it was a way I had my S3 bucket setup or my paperclip configs that actually returned a forward slash. And because of that forward slash when active storage went to look for my files could not find my files. So everyone knows that he's or hard. So that's a that's a potential Pitfall if you're going through this process and then some so we when I get to some production we're about 80,000 images, so it wasn't too many. So I actually opened

each image and ran it through the MP5 process. I think some of the gems actually provide the checksum for you. So that'll just be depending on what you're migrating from. All right. So that is then all of those records. We need to write the very last step is just writing to your attachments table. And that's your attachment which we discussed is the string Avatar model name, which is our user instance ID. Okay, excellent. So that is the whole rib cap. So the next

thing to do after you have run your break fast is figure out if it worked. So the quickest way to figure out if it were is to actually see if you've created the correct number of blob records and attachment records. If you're feeling feisty as you can go into your database take one record from your user table and see if it has transpose correctly to your attachments tables and your blob Stables. But if you're not that's fine. We'll figure it out when we get there. All right. I feel like I kind of bed read through a lot of

code there. So let's do a brief overview of what we have done. So we created the active storage Tables by installing active storage and running the migrations. We can figure the active storage cloud storage. So that was storage. Jama and that was considering on a / environment basis. Some kind of long we wrote the whole rake task to create that use your avatar records in the attachments and blob stayed up tables and we source that data from the user table or whatever table hat currently has the file attached to it. And

we have hopefully confirmed that records were created in the active storage tables. So we don't actually know what the records are right and less you took the time to actually put peak in your database and look we don't know if they're ripe but we know they exist so that's good enough to move on to the next step. Okay, before you move on to the next step, I would highly recommend checking out a new Branch technically you do not have to do this. You can push one branch up Runner a task and then push

the second Branch off without the storage but for testing I think it's a lot easier to do a new Branch. This is my preferred method as I mention. I got the key wrong the first time so I had one branch with paperclip in a raid pass and another Branch with active storage run the rake task use active storage. If it doesn't work, you can blow out the active storage records fix The Wraith cast rewrite to the tables and try again. As I said, here's our deploy run the right test then go to your active storage models and views. So

now I will show you that. Alright, so now we have installed active storage. We have data in our tables are active storage tables. So now we can actually preferably on a new branch in my opinion. Now, we can actually change our code and are models used controllers and test to use the active store active storage functionality. So the thing I really want to show you this is you know, this is why it look so easy in the docks, right? Cuz you just do has one attached but it only works

to know if they're the reason I wanted to show you this is I wanted to show you the bottom here. I wanted to show you views. If you look at the view so you can see if you're going to be using multiple sizes of images use something called variance. And the cool thing about variance is you can just pick your image size kind of on the Fly you aren't hamstrung at a specific sizes that you free to find. So let's talk a little bit more about Berrien cuz if you're working with images as I was there very important. The paper clip paper clip I

think pre-process is all your image size is so they're going to give you your whatever they are large thumb medium. Whatever size is your working. So active storage is going to do a late fee transform on the original blob on the fly has the airplane and we'll just passed the variance so that the processing is only going to happen the first time it's generated. So here I was in this process of working for this client migrating this application and I had a rake task. I knew it was working. I had looked at my database stores to

find my files and I ran it. I'm probably 30%. Very image-heavy websites important to note at least 30% of the images were blurry. That's how I felt right then. So why were 30% of our images blurry they were blurry because active storage uses mini magic for image transformation mini magic does not support the advanced image processing that we have been using with Shrine not I think it's pretty important. That was a really big pain point for us. But fortunately there will be a happy ending. So we

did this I want to see if this was eight months to a year ago. And I feel like we're a little early to the active storage party mainly because of this image processing snafu we had to deal with Fortunately for us. Rail 6 should be solving stop this specific issue at the storage on real 6 has deprecated many Magic and is now using the image processing gem. So fortunately that image, I believe it was like the resize to fill resize to fit that did not work with me magic. alright,

so we have already done. What's up there to play with paper clip run the rake task and the active storage tables and the next step is to deploy with the active storage models and beus implemented that I just showed you. And if that works, then you have completed well made good progress on your migration to act as storage. So let's revisit all of our stuff. All right, so we installed active storage. configure the cloud storage move the Avatar data from the user table to the active stores tables and now active storage

can work its magic and it should just work. The end. Thank you for coming.

Cackle comments for the website

Buy this talk

Access to the talk “RailsConf 2019 - How to migrate to Active Storage without losing your mind by Colleen Schnettler”
Available
In cart
Free
Free
Free
Free
Free
Free

Access to all the recordings of the event

Get access to all videos “RailsConf 2019”
Available
In cart
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “IT & Technology”?

You might be interested in videos from this event

September 28, 2018
Moscow
16
167
app store, apps, development, google play, mobile, soft

Similar talks

Chris Salzberg
VP Engineering at Self-employed
Available
In cart
Free
Free
Free
Free
Free
Free

Buy this video

Video

Access to the talk “RailsConf 2019 - How to migrate to Active Storage without losing your mind by Colleen Schnettler”
Available
In cart
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
575 conferences
23117 speakers
8618 hours of content