Duration 25:14
16+
Play
Video

Data Catalog for data discovery and metadata management

Shekhar Bapat
Product Management at Google
+ 1 speaker
  • Video
  • Table of contents
  • Video
Google Cloud Next 2020
July 14, 2020, Online, San Francisco, CA, USA
Google Cloud Next 2020
Request Q&A
Video
Data Catalog for data discovery and metadata management
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Add to favorites
12.59 K
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About speakers

Shekhar Bapat
Product Management at Google
Shruti Thaker
Global Data Manager at JO HAMBRO CAPITAL MANAGEMENT LTD

I have a background in engineering and product management, and am passionate about creating solutions to address customers’ data analytics needs.Specialties: Product Management, Business Development, Engineering, Startups, Big Data, Data Privacy, Cloud Computing, Data Warehousing, AWS, Amazon Web Services, Amazon Redshift, ETL, Relational Databases, NoSQL, Hadoop, Distributed Systems, Scalability, SaaS, Algorithms, C++, Python, SQL.

View the profile

I am a problem solver, I love getting a challenge and then noodling over it to come up with efficient innovative solutions. I have a strong history of working on global teams to implement truly global systems and solutions. I have built my reputation by taking on difficult challenges where data, technology and process meet and delivering intuitive solutions that drive growth and insights.

View the profile

About the talk

Data discovery and metadata management is a common pain point for most enterprise customers. Data Catalog is a managed service that addresses this pain point with a scalable and performant solution. Data Catalog offers integration with DLP for auto detection and tagging of sensitive data and facilitates better data governance. Data Catalog also provides support for non-Google Cloud data assets and enables customers to discover, understand, and manage ALL their data.

The session is co-presented with Shruti Thaker from BlackRock, a Data Catalog customer. Shruti shares how BlackRock is using Data Catalog to create an effective metadata solution for BlackRock data assets.

Speakers: Shekhar Bapat, Shruti Thaker

Watch more:

Google Cloud Next ’20: OnAir → https://goo.gle/next2020

Subscribe to the GCP Channel → https://goo.gle/GCP

#GoogleCloudNext

DA103

product: Data Catalog; fullname: Shekhar Bapat;

event: Google Cloud Next 2020; re_ty: Publish;

Share

Hello, I'm Shaker. Bob her on the product manager for the decathlon, working for Google cloud and I'm here to talk about this. I got love for you to Discovery and method. Is a management author from Black Ops with the end of the presentation, and she's going to share that experience. You can simply for cannabis users to discolor understand and manage all day. So did I can't even Bait A lot of feedback from customers but they said this is breathed life into the search for mind. If I did, I said, but they told us that

they wanted one unified single-pane-of-glass and that is what we deliver. And I'm surprised about the Genesis Google, I'm back. Externalize, the system that you wanted to do something different, from what I expected in the first thing, I wanted to do it so it's ready and talk to it. You don't need to register your car so you can do a full scan of the data or save money on it. I need spies to specify that want to be getting my sequel, Poston Seafood, companies to do. And as you can see,

The iPhones that look, her and tablet gives you a unified view on a little bit about the big stores. All the metadata in Bonita, He's very Smiley Foundation. Google search technology, interest Gmail, as you know, have more than billion users. And so we know that this is very scalable technology. Is like his new here in the ER, Double Boolean, the facts on this phone. I don't like the following. So I'mma start your time aboard the user Jeremy's that people to support a SpongeBob and

you know things are looking at it. The last night and get a job for completed normally. The process that uses the access to and there's no easy way of providing. They said she'd be keeping just one since ages. Is it actually approved and make it a little less this message and it's sex with you again for the second column. For the following indicates the inspector Bellingham. The women are the billies by Dave. I meant in the city of support this time. That disputed using the stamps in the state approved us

using Google to send, you know, how can you search users to the sympathy vote for the business users who actually want to do the broom down for the use of needles in his socks? Heidi's I want to look at home or at a certain time you have realized more complex ready by combining this and on all the time and not the full power of the purse technology. One of the really important things that I want to mention and I was talking to his dad, but I am security is built into the personal

philosophy. Did I mention you have a built into the building. The one who had access to different bags and the Beast? No, the next thing that I want to mention is that everything we believe, you, can do whatever you want to do. You want the I want the face mask. And in order for you to do the same thing in large amounts of matter are in this before 7 language Library, so that you can create all the action behind. You are so many of our customers actually have their own

friend and also, and the use the API to power the back in because this is a much more stable. At this point, I would like to invite a shoobie from laptops to share her experience using data catalog. The reasons why did she was to partner with Google and see you there. I will administer. Why does cats log me in to Black Rock and making available for investors to use in that research and Analysis? We tried to find out the dates that was, it was very different to the more traditional financial data that already existed

within within the structures that where I made a choice. So what we were saying, lots of great enemy has lots of different formats and vendors and working inventors, that really didn't connect with financial industry as a rule, The day she was coming in lots of different file types as well. On the other side, we had a spectrum of uses. We had people buy geographically diverse from San Francisco to Hong Kong from Sydney, to London, as well as people who have different levels of

comfortable, and that's with the, with how to get to the day. So, we have people who are very, very comfortable just getting into the server. Just a lie environment finding saw that they need putting it on analyzing yet. On the other side, we had lots of users who really needed that interface that front end which allowed them to go in and search for data using keywords or tags and then be able to find that day too. And then work with that, What we found was that we really weren't seeing something that could help us with both of these issues

within Black Rock. And so, we decided to go until to call ASAP Google Cloud to see if there is something that they could help us with. So we stopped at OK Google cloud in 2018 and we explain to them what we were trying to do. And luckily for us at that time back there was just putting it into Alpha sorry we were able to get in there and talk to check her and his team and explain what we were trying to do and he assured us, it was possible to find a solution using Google's ice castle.

Because the cast of Wild n Out. So when we started working with with Google Cloud, it was, it was possible for us to buy has helped shape the product and find a solution that would work for us. In tandem the great things by the base Casa log is that it's really well integrated into the Google Cloud ecosystem. When you're looking at one part of this puzzle, which is the dates catalog, you're not then having to build apis, do bills connections into other parts of Google, it's already connected to Big Prairie. It's already connected to storage

and compute so is quite Easy for us to be able to come up with an integrated and complex fronts. And we should really provide our users, a really powerful way to find an estate sale. We were also able to very simply creates templates price on the front end to start with. And then as our templates became more complicated, we worked with her on the team to put out, then placed into an Excel follows. You could then upload. DirecTV a price on is that she made our lives a lot easier. And where I go to

create tags. As we wanted them, we were able to say, okay well we would like Asset Management as a tag. We would also like a different Sexes. It's time that we were able to put them in there, wasn't anything that was created, which really gave us the freedom that we needed to develop our own solution. As the catalog has moved to the to the general access phase. New functionality has been added including including automatic and tagging offensive information. We would like to start using

dots in the next iteration of our products. Following is a short demo of the part that we built using based catalog as ancient and stated on gcp, which relies on the cheesy potato casserole for school functionality. Find all the dates that the way to go to that. I am to do for the simple analysis. Where possible all of the cats taste. Spit out like ice cream in two different sections. Each one of the drawers. In each section. Is this all fate, bisectors

weekend. In a way that people can touch the date, that is by looking at me inside, on the left hand side, or if they know that they can use those to do such as for this time. Are you going to do is search using Italian style? I'm going to look for As you can see what she thinks. Supposed to page and the demographic, it will open up the dates that I just read me all of the tables available. As you can see, there's a new thing with a to table available by take on that on the right hand side. You will notice that we now see a preview of the table as well as some

information that allows the user to understand how fresh the data is. I don't see if they need permission to the day to in GC PB crary. What project name starts with the four? You can see that we can also teach other people the top 50 Rose off each day to Fat buddies available. This is refreshed on a nightly basis. Have an idea of what the dates that looks like behind numbers involved at cetera. Can also download the sex as a CSV if needed. Desert have available here in Vietnam.

You can also, however, I was going to take the car at Kay's, Bakery. If you have outfits already in toddler or you can download Pictionary if that's available. The way you created this. Application is, we went in today to put the birthday girl in Mississippi. A bunch of data Tech podcast. Was that we needed to have multi-select on some of our such as region or country. And any way to do that was to create a regional country, track template and have food yet. Yes, or no.

Fortunately we were able to work with Google in a format in Excel which we were able to upload a CPU, just make things a lot easier and quicker than creating a template on the front end. we then went through all of our dates, but in big fairy and find all of them using this chart template The next step is to stop attacking dating site as they come in and around boarded using various data on boarding. Is it quick for a and see what can be built using big pass log? We found it incredibly useful to be able to go in and connect

and allow users to go in and search the title without knowing the name of the day or knowing the big light switch with the dates. We have to go farther with Google to create more functionality to our product, eating dates. Catalog the demo. So what are the next things you can do as a user? The first thing I want to point out is that you can launch data catalog, right of it. There is no set up an alarm in Texas, all the metadata in the background continuously. So all you need to do is to go to the store, the girl in the console. Cloud. Google.com, go to decathlon. You can stay. If you want

to start using this enable the epr and all the data assets are already discovered up to you, just for the simple keyword search or the powerful sorcerer. You can also use the lp to auto tag by our data in the DMV integration is available through the DMV you on. It is also available through the API and the slim gives you the details of how to do this to tag. And lastly, if you are one of the users who has data scattered in, all these different files and you're finding it really, really hard to

discover, does they have it? You can use one of your necklace and all the connectors are listed in the Google so you can download them, they're open source connectors and you can start using them. Happy birthday. In German.

Cackle comments for the website

Buy this talk

Access to the talk “Data Catalog for data discovery and metadata management”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free

Ticket

Get access to all videos “Google Cloud Next 2020”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Ticket

Similar talks

Grant T Timmerman
Developer Platform Engineer at Google
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Jerzy Foryciarz
Senior Product Manager at Google
+ 1 speaker
Ivan Gusev
DevOps and cloud architect at OpenX
+ 1 speaker
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free
Craig Wiley
Director, Product Management CloudAI Platform at Google
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free

Buy this video

Video

Access to the talk “Data Catalog for data discovery and metadata management”
Available
In cart
Free
Free
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
635 conferences
26170 speakers
9693 hours of content
Shekhar Bapat
Shruti Thaker