Duration 23:42
16+
Play
Talk video

Building Workflows With Celery

Josue Balandrano
Research Engineering / Scientist Associate IV at Texas Advanced Computing Center
  • Video
  • Video
DjangoCon US 2018
October 14 2018, San Diego, USA
DjangoCon US 2018
Video
Building Workflows With Celery
Purchased
In cart
Free
Free
Free
Free
Free
Free
To favorites
2.44 K
I like 0
I dislike 0
Purchased
In cart
Free
Free
Free
Free
Free
Free
  • Description
  • Discussion

About speaker

  • Josue Balandrano
    Research Engineering / Scientist Associate IV at Texas Advanced Computing Center
Josue Balandrano Coronel works at the Texas Advanced Computing Center as a Software Engineer helping build Science Gateways. Josue has been a DjangoCon US and PEARC speaker. He is also passionate about complexity and how to remove complexity from software development. Josue is a Celery core contributor.

About talk

Topic: IT

Task Queues is a topic which most developers will eventually have to dive into, specially in today’s web development world. The idea is really simple: whenever one has any functionality which might take too long to perform, one can spawn a process which will take care of this functionality without having to block the app’s main loop. A task queue will use worker processes to execute these long-running tasks and the user does not have to wait until the task is done. Instead, an acknowledged message is presented to de user while the task is executed in the background. This concept is really important when building web applications. HTTP Requests have timeout and making the user wait a long time for something to finish is not a good user experience practice. Usually, these tasks are used in groups creating a workflow where the work is distributed into smaller tasks.

Celery is usually the first project one encounters when searching for task queues and Django. I have been using Celery for over four years. The Celery project is one of the most robust task queues out there. It is certainly not the only task queue. And, it can be difficult planning the correct architecture for a specific workflow. This talk will explain enough of Celery’s basics to understand how to build workflows with Celery.

Building workflows with Celery is never straight forward. This is mainly because Celery offers the building blocks to build workflows but it tries to move out of the way. By not being too intrusive, Celery allows building complex workflows. I will explain common patters and tips to successfully use celery to build workflow of different complexities.

Outline
  1. Introduction

    1. Basics of Celery’s architecture
    2. Caveats when using Celery
  2. Workflows

    1. What are Workflows and why use them
    2. Celery’s building blocks
    3. Common workflows techniques
    4. Real World Example
  3. Tips and Tricks

    1. Caveats
    2. Testing
    3. Local development
    4. Using Celery in production

Share

Cackle comments for the website

Buy this talk

Access to the talk «Building Workflows With Celery»
Purchased
In cart
Free
Free
Free
Free
Free
Free

Video

Access to all videos «DjangoCon US 2018»
Purchased
In cart
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic «IT»?

You might be interested in videos from this event

September 28 2018
Moscow
16
76
app store, apps, development, google play, mobile, soft

Similar talks

Matt Chapman
Manager, Data Engineering at mPulse Mobile
Ryan Sullivan
Web Team Lead at Wharton Research Data Services
Mariatta Wijaya
Platform Engineer at Zapier
Purchased
In cart
Free
Free
Free
Free
Free
Free
Chloe Condon
Developer Evangelist at Sentry
Purchased
In cart
Free
Free
Free
Free
Free
Free
Patrick Arminio
Backend Engineer at Verve
Purchased
In cart
Free
Free
Free
Free
Free
Free

Buy this video

Video

Access to the talk 'Building Workflows With Celery'
Purchased
In cart
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv you get access to our library of the world's best conference talks.

Conference Cast
142 conferences
7193 speakers
1698 hours of content