Duration 41:29
16+
Play
Video

BYOP: Custom Processor Development with Apache NiFi

Andy LoPresto
Security Engineering Manager at Andy LoPresto
  • Video
  • Video
DataWorks Summit Barcelona 2019
March 18 2019, Barcelona, Spain
DataWorks Summit Barcelona 2019
Video
BYOP: Custom Processor Development with Apache NiFi
Available
In cart
Free
Free
Free
Free
Free
Free
Add to favorites
1.64 K
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
  • Description
  • Discussion

About speaker

Andy LoPresto has been at Cloudera (previously Hortonworks) since 2015, where he currently leads the Data In Motion Security team. In this role he serves as both a Committer and Project Management Committee Member for Apache NiFi, an open source, robust, secure data routing and delivery system. Andy focuses on security concerns within NiFi including identity management, TLS negotiation, data protection, access control, encryption and hashing. Andy is also involved with the sub-projects -- Apache MiNiFi, which drives edge data collection, including secure command and control and immediate data provenance and governance; and Apache NiFi Registry, which provides complementary applications for flow versioning, development to production lifecycle, asset management, and extension library services. Over the last few years, he has presented about NiFi and MiNiFi in Barcelona, Singapore, Tokyo, Melbourne, Berlin, Sydney, Brussels, Portland, and San Jose.

About the talk

Apache NiFi, a robust, scalable, and secure tool for data flow management, ships with over 212 processors to ingest, route, manipulate, and exfil data from a variety of sources and consumers. But many users turn to NiFi to meet unusual requirements — from proprietary protocol parsing, to running inside connected cars, to offloading massive hardware metrics from oil rigs in the most remote environments. Rather than posting a community request for custom development or offloading unusual demands to unnecessary external systems, there’s an answer in NiFi. Learn how NiFi allows you to quickly prototype custom processors in the scripting language of your choice against live production data without affecting your existing flows. Easily translate prototypes to full-fledged processors to optimize performance and leverage the full provenance reporting infrastructure. Discover how the framework provides conventions to streamline your development and minimize common boilerplate code, and the robust testing framework to make testing easy, and dare we say, fun.

Expected prior knowledge / intended audience: developers and data flow managers should have passing knowledge of Apache NiFi as a platform for routing, transforming, and delivering data through systems (a brief overview will be provided). The intended audience will have experience with programming in Groovy, Ruby, Jython, ECMAScript/Javascript, or Lua.

Takeaways: Attendees will gain an understanding in writing custom processors for Apache NiFi, including the component lifecycle, unit and integration testing, quick prototyping using a scripting language of their choice, and the artifact publishing and deployment process.

Share

Cackle comments for the website

Buy this talk

Access to the talk «BYOP: Custom Processor Development with Apache NiFi»
Available
In cart
Free
Free
Free
Free
Free
Free

Video

Get access to all videos “DataWorks Summit Barcelona 2019”
Available
In cart
Free
Free
Free
Free
Free
Free
Ticket

Similar talks

Andy LoPresto
Security Engineering Manager at Andy LoPresto
Available
In cart
Free
Free
Free
Free
Free
Free
Niels Basjes
Principal IT-Architect / Corporate Inventor at bol.com
Available
In cart
Free
Free
Free
Free
Free
Free
Cindy Maike
VP Industry Solutions at Cloudera
Available
In cart
Free
Free
Free
Free
Free
Free

Buy this video

Video

Access to the talk 'BYOP: Custom Processor Development with Apache NiFi'
Available
In cart
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
416 conferences
16730 speakers
6016 hours of content