I am passionate about solving / crafting technical problems through code. With 13+ years in custom software development, I have worked across technologies which includes full-stack, mobile and DevOps.View the profile
I'm a passionate developer who likes challenges, especially the ones difficult to solve or not solved by many. In my 7+ years with ThoughtWorks, I have got opportunities to work on various things like developing micro-services, re-architecture and re-designing systems, building infrastructure platform, data analysis and what not. I find it exciting to learn and solve something new everyday at work.View the profile
About the talk
Kubernetes lets you deploy applications in ways that are highly available and resilient, and can utilize the underlying resources more efficiently. This can bring huge cost savings to the business, especially when you’re entrusted with running PostgreSQL clusters on Kubernetes, rather than over traditional VMs.
In this talk, we share our experiences of running and managing 100+ PostgreSQL instances across multiple use cases and environments, for a leading technology start-up business. We take you through some of the challenges we faced, and the various tools techniques we adopted to work around them. Join us if you’d like to understand how we addressed the following issues:
- How to handle the ephemeral nature of Kubernetes?
- How to resize disks with zero downtime?
- How to change disk type from standard to SSD with minimal downtime?
- How to upgrade PostgreSQL servers with minimal downtime?
- How to redirect the read-only queries to slaves?
Good morning, everyone will be talking about our experiences and tell me a Saint Cloud so far our contacts while we are talking about the Earth will be restricting. Our conversation around the country so far this context. So maybe before we get started I just between producers and everything. Experience for the last 2 years on running hundred plus databases on Fort Bliss. Yep. So yeah, unless you know, I'm with you so I'll be not even present for her, and we have been working for the last 2 years on the Pokemon. It
doesn't. I think so. So before we go inside. Do you think those girls are Cuban artists? Are any other Cloud, right? So, so let me get contacts on what problem are trying to solve, right? So the temperature of a pot or a container, you want to call, you know, if you wanted this world. So when we mean ephemeral ephemeral, what we mean is like the process of the continent has very short list of that liquor. These are some of the example if you run a process which is like the best Cuban artists. Are you can pre-order the group of containers in mist of containers. Wait, I'm
content. I don't know if you see in the Pod, usually, what happens is everything, the process one? Who is Scotty, main process, right now, if I can only process one, before we start our bar that shaped like the process display current tax rate, if you reschedule the part. So what do you mean by reschedule? It comes back again, always it gets a new IP address because there's no guarantee that nobody is giving that the wifey address of a part is going to be constant.
Five Nights at Freddy's fighting inside the park right now for both girls at least or any database or state full application straight. We need some other way to save the day. So that it is persisted. Even after the even after a lifetime of a pot. So that is the third problem on the floor to you. So very interesting problem, right? And there can be a nice one. One note that can be Lord of Parts, a lot of postgres database, or one of the process. If I can simply fit, it takes more results and resourceful and I mean, I'll see if you
are memory, it can take a toll on the 4th if you're talking about so which means the worst case of that scenario, Maybe that's the performance of my part can get affected because of neighbor my spray. Some of the problems that we have before we get out of this, and if he wanted the second condition, right? How do you connect, can you connect using the same my papers? But in this case, you can't ride even if your contribution to this conversation, you can really will go to that. You can open your favorite, are you can change, whatever? It can save it, and
come back right here. You can do any of that because I'm going to save, it does about both of the file itself as both of Alan. Even if you look at the first one in the process, There's no control first because everything that they report the nature of the body. Let's go get inside the lake water, right? So if possible, we need a hardware on a mission where we're at right at Mission can be hosted in the cloud or it can be a battle that we are maintaining anywhere, but we need a machine and we aren't, we need a
prosthetic. Anything that we need is obviously The Fosters binary trade for one, so we can download and install. But if you're talking about, we need some concentration management word, a constable chair for puppet set up for the user settings like that, the tortoise we need disk. So why do we need an external describe? What would happen? If I don't know what will happen. If my Wii. I want to start this process process somewhere else on this disk mounted somebody else inside my first class
just just do not run. I said what you said the Constitution for restricting who can access my post, I need a positive contributions, like a minimum connection buffer and things like that for my pulse blister and I don't know what how do I do the same thing? In Japan at the state. So far, we are going to use Docker and Docker images. You're not going to miss that does not end. DACA have soda official images of Snapper, different versions of oscars, so he's going to be strictly straight. So we need some
volume volume, volume, some things that will live lifelong a kind conversation volume claim. So I'm claiming for persistent volume, it doesn't eat this. Not dependent on any hole in a provider. For example, if even if I ask from Google or Amazon or Microsoft azure. Sapore's pizza at me when I mean support rape. They have two horses to Van atta's, right? If I am going to post my own job and then I have to make sure that I I have this confidence between us. I provide me, the store is closed, but if you go for a documented solution is comes by default that
is no problem. That we have, we have a guard, the sticks with the context of the board is going to be safe. So we need other way of doing the one-ways obviously, but use a mounting a disc. But the problem is that is what how, how easy will it be? If I die to change the configuration so far, this is officer and called configmap. So it's the nature of it itself is to save configurations. Amorphous. FR by default rated state to bring it back so that are different. But when it comes to running state for applications to use something called the state was
serving in the interest of time will not go deep inside, why we are going to use to send things like that. But if you have questions about rent in image, So if you see here, this is Lisa, this is the place where I'm saying do not run. That said what you said, same thing you always run as user 999. So by default, when when I run my first breath with this default administrative service, change this number. I didn't know, how do I load my consecration concentration. Can mint be a conflict without
conflict map for this. If you see a conflict map is like a True Value store, right? So, I can save a simple details, like the posters user database and things like that. And I can also save a configuration file, so I can save up. So this can't be mounted at a show you how to do that. This can be used as an emu. So both of this is supported by this portrait. So if the part the contact map where I'm user using it s e n v. I can load it, doesn't he invariably?
And I am trying to load it as a file. I can say that K. This is a volume and I can mount my volume so I can not go deep inside. That is something called secret, which was similar to configmap, but you can be encrypted and stored something. Here. I'll wait around for you. so, the Bulls guys are starting Okay, so it is running but it is not ready. Stokes electric inside. So this is me, and so we can go inside apart to see what processes running and things like that, right? So if you see what is
dual speech or try to physical trait, So, this connects because I how to make sure that I connect via host so far. No. That is because we don't have the HP account for it. Let's be a conference. It's actually beat for this purpose. We are only restricting to local but this can be modified right at 2. So I think that is, okay. In the next day using a service or something like that. Underwater, I'm sorry. so if you see hear my consecrations right, as you can see here,
So these players are here, obviously the Ford comes back, so everything didn't really want to see. If you look at a time, most of the problem is, Even If part was Ben comes back, we're able to hold back, the contribution file for workers, compensation file and very bottom on back to discuss. What do you think? Auntie Maps, secrets and help us to make this particular news report to not be that much? Okay, so the second problem right? That we have his help so
I have a Bose customer, right? So what will happen if the school system so that is what we are. We need these things right again and we need some change. It has to follow from what was trying to connect it and it has to be a lot from this chamber that I'm going to help and then I told us to do the video birth backup for it so that I can set up a swimming or water application. I want to again for the session, we are going to use the EZ Pass back up with streaming application.
I thought we could sing So, I'll delete the ones that I already have. Dragon here, I have to stay. So one of the primary and one of the second ring, so they are similar except that I created application user. When I started dating events in the primary because if the problem here is that I have a boyfriend when he has to be first running for right or for me to create an application, so he can be moved anywhere. But again for us to become a very simple, we're doing it
after the primary stars in this sandwich. So, that is the biggest music and we are also running back up here. Okay, so I'll also run this one. Are you thinking to take a little more time to set up and things like that to the post office and show you like the Saturn petition status and things like that? Taking time, so let it run. What is the primary ghost town? Because what we are so far apart, can I mean it is not, but it can die, right? It's not like a video where it is isolated. So now,
I have a primary, my primary goes down, I need some way to your door, automatically fell over and make this and buy a slim, because if you see a ladybug talking about hundred 201st street, so we can wake up in middle of night to sacred heart rate. So I can. So we have more problems with my neighbor know how to find a way, you know, it's safe or save. I mean, the part are you listed? It has not said, what are the other mechanisms that I can do to have an idea?
How do I cut my client connections to? This is going to be a problem for me to do the maintenance. But here it is not that we are some of the things that is still not. So so if you see one of the most known person, a person who talks about how do you think straight. It has a mixtape for work load. It has a different kind on its own and things like that. Right? But you know especially for running Postgres are a store like red on its own, are not designed for being a distributor system, right? So, even though his office,
all this, we need someone manager of someone who can do more things for us to have a high level. Tinker did the problem, right? So, even though it offers, everything like that for sale country map, secret volume, BBC everything is out there but still we cannot run oppose. This old air because the probability of your body before getting dyed is very high because if one of them apart from one, that's like this, with two paddle board, and then one with communication, you can do is mandatory in point. I'm just not possible.
So what is the national day? So what I'm proposing always right. I think the solution is not there in 2 minutes. I don't think we can look further than keeping it is because all the TV. We're all exhausted to solve this problem. I think we need to find our own solution to this problem. Otherwise you won't be able to do this without any downtime. So what are the dates of the problem here? Is this what happens is this? The problem is there's two things I had one as for this confrontation at the house with constellation, right. News that
will do for for primary and secondary aluminium star. The potato pasta supposed to say he is the master look for it and then forward it somewhere. Outside the postal system saying, hey, this is asleep if you can probably get himself or someone who's been. So if he does the master, he is asleep so information. And before I start to fight and then I'm just proposing that leaves have essentially store star store in Austin. What we need is a business Process Management in Slovenia vs, Arjun Kapoor. Guess what I'm saying is at least have a process that will start the
first first responses of this particular process has to look for the store. What is the role of the? Both of us are going to start on that? It will all the hits B Firewater talked about an event Like something, I'm dropping. The second thing, you know. The thing is, we need someone to constantly prove whether the master says you're not. But I think the Moss is not healthy in the bin and tell everyone singing monsters Dawn, everybody act, and then all
the process of Switzerland, okay? Now I need to become a certified. Start myself as a master stand by for an important as we need to Roxy. Roxy. Roxy Hunter Innovations Master. Look like So, what I'm suggesting is, we need these for things for us to successfully run, a hundred, plus you need these for different things. Unless you have the food is going to be very difficult to run. First of all, I'm saying is the central store and we'll just go to the conference or the
third one, which is keep watching. And if something goes on, that is clear what you need for the four different conference. Why we need for this conference? Yes, definitely. Indianapolis. Is a problem statement, clear? Right. So what's the problem we're dealing with and how is going to help us? The first one, right? The first it's already on the bus. In Play Store here right now, trying to make a higher royalty, post without a fight, right? To make it available
in. The store is not dead or intestines is not going to work. Because everything go today on this particular store to work. So some of the options we have, right? One as we can use on kubernetes configmap itself as one of the solution that is we can use cancel that option is a definite thing that's supposed to be one of them. But there are four possible that you can choose any of advice on how to use case, but just any phone of this, for the very easy to distribute
and then go to rehab. What is going to happen? Is right. The Sentinel keeper and proxy doesn't know anything about anything. That's the only thing they know is that they will talk to the store to know the information where is touring. What when the party starts up? Did you say, hey, I am the keeper store and the store. I want to see some information. This party started. Play. To all this thing. I'm going to say proxy. Keep on Sentinel and there's a store together
by Pinson. Post saying I'm running gear store. Allen Sentinel. Okay, here it is. Raining play Danny to take one as a master Alan proxy. It knows from the store. mustard greens in the Stop playwright. But as he pondered and proxy or a single incidence rate doesn't solve the problem. Okay so what we can do it late, even here, probably we can bring it before taking a loaded gun pointed at corresponding mustard and then run if you look at it, right? Sort of 3 all the time and Wally. Portable
person has the power to demolish Play sulfur. I didn't do nothing. You was all brought from the right to make some forty-year-old thing. And we're trying to put it in human nature and we want me to do. We need to look for who is a master? And then if the primary keeper goes down, right senses will automatically figured out the haters. Not did you Alec neumeister? If that happens. Then one of the skipper will become master master streaming data from the You just playing with it? Comes back in the store. Hey, who am I?
Play. No, no, no, no problem. Proxy proxy, internal proxy. The plant will not know that. Yes, he doesn't have it, if it doesn't have any data by itself is, Yeah so basically what they didn't lose weight, all the things do start pulling right. They have to update them from the store continuously is not dead. So it has been done. So, one example can be like this until after the sponsored, the leader doesn't respond so that other person goats themself as leader and the other people in their system. Try to follow the leader
of the council and they have something you can put a time to live right by the key. Again. He's going to vanish from the store. Is that the NWC? This is going to be there only for a minute. Who is The Keeper of the Angels Monday? We don't have to do any of that. There are solutions out there. I'm picking up too. Many bodies are quite popular. Does exactly the same way. They brought in the distributor, ecosystems into this and they kind of helping us to solve this problem, right? It stolen and pepperoni and
So in this case it is, which was stolen to do that. So whatever is pork is already there probably quickly open up the stolen. What does it? It's 8 and you should go look at it right, whatever we spoke. It will be here exactly the architecture. For the pain. If I think that what we talked about it, right? Exactly. The same and Roxy. Roxy, Roxy and it'll be like in salt and water comes in. It doesn't write a woman model in the front is completely disconnected. All the process. I'll do a quick demo of this.
Tour de Luis, Jose PG County. She also has you because we need to manage hundred plus we can't have a single. So basically, what this will do is, It has ended me all the necessary on your face for me with the name. What was his name again? Right? Music talk right. Sentinel everything I also took what's the value? Are you quickly change into? Okay, just cancel. I am simply going to take you cereal. Let's watch for it. What does the back and what we're talking
about, the stolen back and concentrate. We have a console as our back end and then we have a proxy and we have keeper Animal Center. Aldi's in Gurnee. I thought so once everything is in Chili's, probably we can actually do it in the meantime, if you have any questions, Artemis Vehicle Administration, Annapolis Mall. We wanted to show like what happens during a switch over? So, is this part of this for no? In the interest of time? I'll just proceed. Okay, I'm going to talk about other uses here. Play. If you want to increase bixby's raises a very common occurring thing,
right? If you saw the posters with her particular size of thing, and it showed me how to do it. So difficult to a person volume and the disk size Waterbury have a TV changer number to Ministry. Take care of our meddling in his face and then attach to the Cashman in Port Angeles, school board meetings, you know where I live in the percent volume, playing whatever you were created, as only thing to do. So that when is my getting started to feel sorry, using a CD disk. And then from there,
you can spin off. Another set of this water. Talked about with the attached and it started leaking dinner from here to there. Once a completely transfer, you can completely get rid of the world. Auburn vs version. So the second time because once the volume. You can't do anything right now. What do you want to do and then use the PC update command. Once it is done, stop your part. And then Do you want to show this even if we killed the? I mean the primary touch and we
will start becoming the family. So one thing that we wanted to close with rate is what were our learning of a running to the post and interpret post cuz right. So one thing is the country store right now. So if you see here except up the context or was very important, right? It has to be highly available and distributed so we can fix the problem with that was upset that we faced was when does Master of in downright Earnest or became unavailable. Because we entered
when we went with console or we can even go with etcd, that's all it is. That you have a little more, a stable gets the unicorn of the console part goes down. It does it say in the sensitive data, be more replica. So it can respond. So it became a little better after we started using concert, right? But this is one So you ought to be very careful as to what story are you sing us a sack on fixture. So the next finger that I think I called out a couple of times in a bird
noise. So Navea usually only one posed to be running the stuff and processes running. Right. Which means the other of a human eats that are for the number to post a status on that note. I can actually affect me for the stupidest helps us, right? So that is a concept called resource resource request unlimited, so you can restrict how much of a resource a particular part can use. And you can also say, what is the meaning of that? I want Sebastian that I can. Inside the pants down, decorate, the scheduler takes care of
memory. I want for my boss restaurant and if I want to see if I say that that is scheduled for inspection for Postal Service running. I'm the next to nothing, right? This is a suggestion from Google to have a separate state license plate for application for the supposed to let any other application that our status as data, that means a normal application, which is fine. But killing the primary might have a small town terrorized by don't deserve deployments itself. On the last
but not least. Even though we have everything, right? It's always safe to have a daily backup, all the system. As I need a zillion. That's all fine, but what if someone wants it, nobody will be able to stop because you have a stand by following the primary. Any questions? So all that a different reasons we didn't do any cause analysis, but I was going to say, don't have this extension sounding really use them. We can do one on one. Like we have a Pac-Man job, man. All the things they've used for different reasons.
It is very hard to go to Cloud as of now, Damn sweet, even David stand. Document is masturbate right person. Leo Bolanos using Google comes back in time to actually kind of application application form. We are rebuilding from primary Monday, adding all the options, you can take it how you want to do this deposition in this case, we use application slots to stream from the one. You can also do was ask anything of yours. I think also available data table
to the streaming in case of descent a dingo sound, right? We can switch and I can bring them back. So this from this primary to sync to other posters from that, you'll have a standby for that answer. So you can switch over. Under plus, you know, I think you're not understand by the sentences of humanist. Not going to post this. We have a hundred of photosynthesis. You can actually answer this question. So just think about it can be used in BMF Solutions right there. And if that answers your question,
Twice, The Keeper. What we spoke about, right. I think you can take me to any of the things, like, what do you need all these components to run. Give me the questions. Have you tried that? We didn't try because we we found Solutions and we went with this because it's very more scalable and you can put those in the solution is same across communities in VM, we don't have to do anything specifically for 2 minutes. One more question. So yeah, it was one more thing. So the examples that we showed our available online in this report, so if you want to look at it, you can have a look
Buy this talk
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.