Kafka Streams vs other stream processing libraries (Spark Streaming, NiFI, Flink

Stephane Maarek | AWS Certified Cloud Practitioner,Solutions Architect,Developer
A free video tutorial from Stephane Maarek | AWS Certified Cloud Practitioner,Solutions Architect,Developer
Best Selling Instructor, Kafka Guru, 9x AWS Certified
4.7 instructor rating • 41 courses • 922,316 students

Lecture description

Short lecture comparing Kafka Streams to other streaming libraries

Learn more from the full course

Apache Kafka Series - Kafka Streams for Data Processing

Learn the Kafka Streams API with Hands-On Examples, Learn Exactly Once, Build and Deploy Apps with Java 8

04:48:46 of on-demand video • Updated July 2021

  • Write four Kafka Streams application in Java 8
  • Configure Kafka Streams to use Exactly Once Semantics
  • Scale Kafka Streams applications
  • Program with the High Level DSL of Kafka Streams
  • Build and package your application
  • Write tests for your Kafka Streams Topology
  • And so much more!
English [Auto] So one last lecture just to reflect on what happens. So this was awesome, what we did in a previous lecture, I really hope that you see the full extent of it. And in the next section, I promise we're going to see how to code this, really understand what happens at the code level. OK, so just before we step into it, I know you guys are going to ask me a lot of questions and this question is going to be recurring and recurring and recurring. And it is should I use Cafcass Dreams or Spark Streaming or Nephi or Flink or really any other library? And there is no right or wrong answer. It really depends on your use case and what's best suited for it. All these libraries are constantly evolving, constantly changing, and really things change. But as of today, let me tell you the differences spark streaming. Nephi flank all these and Flink at least part streaming does Microbe's each and Cafcass streams does per data streaming. So this is pretty much what you want. Do you want real, real time or do you want microdots real time. There is a cluster required for sparks to ring for Nephi, for Flink, I know this for a fact, OK, but in Cafcass dreams, you don't require any closer. You see, we ran our Cathe costumes application with one command. We didn't start with Cafcass from Cluster whatsoever. And I promise you know, Cafcass from Cluster W started OK, so Sparks trimming Nephi and Flink require a little bit more maintenance as to how do you launch an application, how to scale an application, et cetera. So I really like that. About Cafcass Streams is also very easily, but just adding Java processes. So we'll see this in the next lecture. How to scale a Cafcass application. But there's no reconfiguration required. There's no cluster again, is just adaba processes, OK? It also has exactly one semantics on Kafka and Spier. I find Flink for now implement at least once. So this is really awesome because Cafcass Streams is so close to Kafka, has been developed by the Kaffee guys that it really leverages Kafka for its full capabilities and now does exactly one semantics. All right. Whereas other streaming library experts, Nephi and Flink, that just take the data out of Kafka and forget they was coming from Kafka. So that's that's something that it's a bit weak on their side. And I'm sure that over time they will evolve and provide exactly one semantics. But for now, Cafcass Streams is the only library that does provide that Cafcass Streams is all code based. OK, so so is pasturing, so is Flink. But Nephi isn't Nephites drag and drop. And then finally, you have a query question that answer this in details that you can look at the link as opposed to link, but you can see my detailed answer right here. So I hope this will answer most of your questions if you'll have some questions asked in the Q&A. But this course is solely dedicated towards extremes and Cafcass streams. There's only one thing it's kept catechetical. OK, so hope you're excited. And then the next section, we're going to get deep into the code and to an application. So see you soon.