Big Data for Managers

Name: Big Data for Managers
Rating: 4.5 (1705 reviews)

A foundation course for big data that covers the big data tools for various stages of a big data project and now chatGPT

Created byGanapathi Devappa

Last updated 8/2024

English

What you'll learn

Confidently lead a big data project in your organization
Differentiate big data technology from traditional technology
Talk about big data solution stages and cluster sizing with your development team, architects and CTOS
Select tools required for various stages of your big data project
Build an action plan for your big data analytics project using 5 Ps model
New: Explore chatGPT to analyse your data and help in project management with 1 hour of added content

Course content

10 sections • 107 lectures • 5h 8m total length

Introduction to the course4:05
Introduction to the course
Course prerequisites and course structure1:30
Big data sizes1:47
Case study: Traditional solution vs Big data solution5:03
Activity : Calculate the data sizes for big data projects in your organization0:07
Use the provided template to list the big data projects (or any projects) in your organization and estimate the data sizes

Introduction1:34
3 Vs of Big Data2:56
Industry examples of big data4:30
Big data analysis and visualization2:44
Traditional vs big data technology4:18
How is big data technology different?9:06
Big data solution stages2:25
Apache Hadoop and HDFS4:17
Map reduce and Yarn5:21
Pig, Hive and Spark5:11
Things to remember1:39
Activity: Technology type selection0:10
Down load the technology excel sheet and open it in Microsoft excel. Various technology features are listed and you need to choose whether the feature is big data or traditional technology from the drop down. Result will show correct or incorrect.

Introduction1:24
Big data solution stages0:48
Big data storage characteristics4:35
No-SQL databases2:19
HDFS4:06
Hbase6:16
Cassandra5:45
Mongo DB and Impala3:46
Sizing your cluster2:35
Things to remember1:27
Download the cluster sizing template to use for your big data projects
Activity: Size your big data cluster using the template0:03
Down load the Cluster_sizing_template excel sheet and open it in Microsoft excel. Use it to size various big data projects in your organization.
Activity: Solve these storage exercises1:40
Apply what you have learned in this section to estimate various storage solutions

Introduction1:00
Solution stage Ingestion0:43
Sources and types of data for ingestion5:34
Big data ingestion tool features2:14
Ingestion of batch data : Sqoop and Distcp3:18
Streaming data ingestion using Flume2:41
Kafka : A messaging system1:58
Apache Flink3:56
Nifi for data ingestion1:26
scenarios for big data ingestion4:26
Data ingestion diagram0:44
Things to remember1:02
Activity: Ingestion problems0:02
Apply what you have learned so far in the course to solve these big data ingestion problems.
Quiz 1

Introduction1:23
Traditional and new types of data visualization for big data5:27
Tools for big data visualization : Tableau, Qlikview and Zeppelin2:14
Java script charts for visualization0:33
Visualization summary0:51
Big data security4:26
Kerberos and Apache Knox2:05
Apache Ranger and Apache Sentry3:22
Best practices for big data security1:12
Opensource software and support2:20
Cloud Computing2:36
Big data on Cloud0:35
Big Data on AWS2:09
Big Data on Azure0:44
Big Data on Google Cloud0:50
Multi-Cloud, Hybrid Cloud and Edge Computing2:35
What is Serverless Processing?2:25
Big Data Vendor: Cloudera1:55
Big Data Vendor: MapR1:43
Snowflake: A Cloud Data Warehouse2:26
Things to remember1:03
Demo: Setup big data cluster on EMR and access Spark and S3 using Zeppelin6:00
Amazon web services provides a ready to use big data cluster service called Elastic Map Reduce or EMR. In this demo, I will show you how to create a big data cluster on EMR with Spark, Hadoop and Zeppelin already setup, Access Spark using Zeppelin, load data stored on S3 into Spark, apply map-reduce type of processing in Spark, access the results in Zeppelin using sql and visualize the results graphically in Zeppelin.
Demo: AWS EMR Serverless22:30
Serverless is the current trend and in this demo, you can learn how to setup a serverless EMR cluster on Amazon Web Services and run a simple job. While the previous demo was complicated, this one is very simple.

Introduction0:11
Agenda for chatGPT1:01
What is Generative AI?1:35
Big Data for Generative AI1:03
Top 10 AI Tools3:52
History of chatGPT2:33
Data Analysis with chatGPT1:58
chatGPT Pricing - Updated for GPT-4o1:33
Data Security with chatGPT1:20
chatGPT Demos1:04
chatGPT Demo 1: Explore the power of chatGPT11:17
chatGPT Demo2: Data Analysis using chatGPT (Previously used Noteable)8:23
chatGPT Demo3: Code Interpreter for Data Analysis and Visualization8:27
chatGPT Demo 4: Analyze data from Kaggle8:55
Custom Instructions1:32
Things to Remember1:03
Activities0:27
Do these activities to sharpen your chatGPT skills.

Demo and practice activity: Create a bucket on Amazon S34:22
This is added on request by one of the students. In the Amazon EMR demo earlier, I have used an already existing S3 bucket. This demo shows how to create a bucket and upload a file on to Amazon S3.
Answers to storage exercises0:01
On request, I have added the answers to storage exercise so that you can verify your resolutions.
Answers to Ingestion exercises0:03

Requirements

Should have software project experience as a team lead or manager

Description

This course covers the required fundamentals about big data technology that will help you confidently lead a big data project in your organization. It covers the big data terminology like 3 Vs of big data and key characteristics of big data technology that will help you answer the question 'How is big data technology different from traditional technology'. You will be able to identify various big data solution stages from big data ingestion to big data visualization and security. You will be able to choose the right tool for each stage of the big data solution. You will see the examples use of popular big data tools like HDFS, Map reduce, Spark, Zeppelin etc and also a demo of setting up EMR cluster on Amazon web services. You will practice how to use the 5 P's methodology of data science projects to manage a big data project. You will see theory as well as practice by applying it to many case studies. You will practice how to size your cluster with a template. You will explore more than 20 big data tools in the course and you will be able to choose the tool based on the big data problem.

I have recently(24th July 2023) added one hour of content on chatGPT with Top 10 AI tools and four demos on using chatGPT for project managers and data analysis providing visualization of data as well as prescriptive analysis of data. These lectures and 4 demos will provide the latest information on the AI technologies and how it can benefit an organization. I am sure you will be amazed by this content.

Top companies now offer this course to their employees. I am glad to share some of the five star comments about the course:

This course really exceeded my expectations! Not only it covers the concepts and the overall view of a Big Data project landscape but it also provides good examples of real case studies, that help reinforce the contents presented. Great course!

This course is great! I have learnt many useful things. The case studies are very enlightening. I strongly recommend. Thank you very much.

otimo tecnicamente, excelente

Didatica muito boa e o conteudo conforme esperado

Who this course is for:

Any team lead or manager who wants to learn about what Big Data is all about and start or lead some big data projects in their organization

Big Data for Managers

What you'll learn

Explore related topics

Course content

Introduction5 lectures • 13min

Big data characteristics12 lectures • 44min

Big data storage12 lectures • 35min

Big data ingestion13 lectures • 29min

Big data analytics10 lectures • 24min

Big data visualization, security and vendors23 lectures • 1hr 11min

Big data projects11 lectures • 30min

Generative AI including chatGPT17 lectures • 56min

Conclusion1 lecture • 2min

Add ons3 lectures • 4min

Requirements

Description

Who this course is for: