Kafka Streams vs other stream processing libraries (Spark Streaming, NiFI, Flink

Stephane Maarek | AWS Certified Solutions Architect & Developer Associate
A free video tutorial from Stephane Maarek | AWS Certified Solutions Architect & Developer Associate
Best Selling Instructor, Kafka Guru, 9x AWS Certified
4.7 instructor rating • 37 courses • 615,200 students

Lecture description

Short lecture comparing Kafka Streams to other streaming libraries

Learn more from the full course

Apache Kafka Series - Kafka Streams for Data Processing

Learn the Kafka Streams API with Hands-On Examples, Learn Exactly Once, Build and Deploy Apps with Java 8

04:48:46 of on-demand video • Updated November 2020

  • Write four Kafka Streams application in Java 8
  • Configure Kafka Streams to use Exactly Once Semantics
  • Scale Kafka Streams applications
  • Program with the High Level DSL of Kafka Streams
  • Build and package your application
  • Write tests for your Kafka Streams Topology
  • And so much more!
English [Auto] So one last lecture just reflects on what happens. So this was awesome what we did and a Proust lecture. I really hope that you see the full extent of it and in the next section I promise we're going to see how to code this really understand what happens at the code level. OK. So just before we step into it I know you guys are going to just need a lot of questions and this question is going to be recurring and recurring and recurring and it is should I use Cafcass streams or Sparke streaming or Nephi or Earthlink or really any other library. And there is no right or wrong answer. It really depends on your use case and what's best suited for it. All these libraries are constantly evolving constantly changing and really things change. But as of today let me tell you the differences sparse streaming 9:5 link all these link actually sparks streaming does micro batch and Cafcass streams does per data streaming. So this is pretty much what you want do you want real real time. Or do you want microbiota real time. There is a cluster required for Sparke streaming for 9:5 for flying. I know this for a fact. OK but in Cafcass streams you don't require any closer. You see we ran our cathe because trimly application with one command we didn't started Cafcass from cluster whatsoever. And I promise you know Gaveston cluster was started. OK so Spark's trimming Nephi and flank require a little bit more maintenance as to how do you launch an application how to Scheller application etc etc.. So I really like the back half of streams is Kel's also very easily but just adding java processes. So we'll see this in the next like sure how to scale a Cafcass stream's application but there is no reconfiguration required there's no cluster again. You just add Gemer processes. OK. It also has exactly one semantics on Kafka and expire in lifeI in Flink. For now implement at least once. So this is really awesome because Cafcass streams is so close to Kafka has been developed by the Caffey guys that it really leverages Cafcass for its full capabilities and now does exactly one semantics whereas others Jamon libraries Xmarks my thigh and flank that just take the data out of Kafka and forget they was coming from Kafka. So that's that's something that it's a bit weak on their side and I'm sure that over time they will evolve and provide exactly what semantics. But for now Cafcass stream's is the only library that does provide that Cafcass streams is old code based. Ok so so is parsed your meaning so is. But no is incentivized drag and drop. And then finally you have a quirk question that answer this in details. You can look at the link as opposed to link but you can see my DTL answer right here. So I hope this will answer most of your questions if you don't questions asking the Q&A but this course is solely indicated tourist cafs streams and Cafcass streams is only one thing. It's Cascada Kafka. OK so hope you're excited and in the next section we're going to get deep into the code and to an application. So Hughson.