Duration 29:30
16+
Play
Video

Evolving Streaming Applications

Niels Basjes
Principal IT-Architect / Corporate Inventor at bol.com
  • Video
  • Video
DataWorks Summit Barcelona 2019
March 19 2019, Barcelona, Spain
DataWorks Summit Barcelona 2019
Video
Evolving Streaming Applications
Available
In cart
Free
Free
Free
Free
Free
Free
Add to favorites
210
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
  • Description
  • Discussion

About speaker

  • Niels Basjes
    Principal IT-Architect / Corporate Inventor at bol.com
Allround IT specialist with a Business degree. What does that make ? Someone who can easily switch from IT possibilities and limitations to business needs and desires. I'm the kind of person who comes up with IT solutions that makes both of these worlds meet in practical solutions.

About the talk

The reality of software systems is that the business needs they intend to service will change over time. So applications must be created that are able to evolve and follow these changing needs.

Welcome to the world where we are building high-volume distributed streaming applications using systems like Apache Flink, Spark and Kafka. Applications that are assumed to run 'forever' never go down.

So what happens if a business need changes? How can you make a streaming application that can evolve without breaking all the downstream applications that depend on it? Roll out the new producers first? Roll out the new consumers first? How do I avoid going down? But wait! Systems like Kafka persist records for weeks; so how do you handle the fact that there can be several different schemas in the Kafka topic at a certain point in time? Can you deploy a new application that reads both formats?

In this presentation Niels Basjes (Avro PMC) will go into the ways bol.com has chosen to handle these effects in a practical way. He will describe how the "Message" format and the schema evolution features of Apache Avro are used in conjuction with Apache Flink to make applications really 'evolvable'.

How do we make sure all applications are able to find the schema specifications, what can we do to ensure schemas stay 'evolvable,' and what were the pitfalls we ran into? Join us and find out.

Share

Cackle comments for the website

Buy this talk

Access to the talk «Evolving Streaming Applications»
Available
In cart
Free
Free
Free
Free
Free
Free

Video

Get access to all videos “DataWorks Summit Barcelona 2019”
Available
In cart
Free
Free
Free
Free
Free
Free
Ticket

Similar talks

Gábor Hermann
Software Engineer at bol.com
Available
In cart
Free
Free
Free
Free
Free
Free
Sara Sylvan
Scania
Gustav Ranby
Development Engineer at Scania Group
Sarah Hantosi Albertsson
Data scientist at Scania Group
Available
In cart
Free
Free
Free
Free
Free
Free
Volker Janz
Senior Software Developer at InnoGames GmbH
Available
In cart
Free
Free
Free
Free
Free
Free

Buy this video

Video

Access to the talk 'Evolving Streaming Applications'
Available
In cart
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
439 conferences
17532 speakers
6299 hours of content