Apache Spark Project World Development Indicators Analytics

Explore the world development indicator dataset, with country data in country.csv (region and income group) and indicator data in indicator.csv, enabling spark analytics and time-series dashboards.

Explore the world development indicators categories—economic, social, demographic, health, and environmental—and use over 1.4k indicators to build spark visualizations and comparisons.

Understand the WDI data structure, including attributes like country name, region, ISO codes, indicator names and codes, year, and value. Metadata adds context for time-series analysis in spark.

Explore how the world development indicator dataset pulls data from World Bank and partners like United Nations, IMF, UNESCO, OECD, and World Health Organization into a central, reliable data hub.

Install Java 11 by updating libraries and installing OpenJDK 11, verify the Java version to confirm 11.0.27, and prepare for Apache Zeppelin installation.

Set up the java environment by editing the /etc/profile to configure java home, java path, and java jre, then verify with echo $JAVA_HOME.

Install and run Apache Zeppelin on Ubuntu by downloading Zeppelin 0.12.0, extracting the tarball, starting the daemon, and accessing the notebook interface on localhost:8080.

Install docker desktop on Windows by downloading the installer, running as administrator, and launching the application to manage containers, images, volumes, and build features.

Launch Apache Zeppelin on docker desktop for windows by pulling the latest Zeppelin image, running a container on port 8080, and starting or stopping it while checking logs.

Install and configure Apache Spark with Zeppelin, start a local Spark shell, and connect Zeppelin to a Spark interpreter to run and test Spark code.

Import the world development indicator underscore old zeppelin file into the Zeppelin environment, then select and upload the world development indicator dot zip file to access the complete code.

Apache Zeppelin is a web-based notebook for data ingestion, discovery, analytics, and visualization. It supports multiple interpreters, including Spark, Python, JDBC, markdown, and shell, enabling collaborative, real-time work.

Explore Apache Zeppelin's features and benefits for big data analytics, including multi-language support, real-time execution, and native integration with Spark, Hive, HDFS, and Hadoop for collaboration and sharing across teams.

Explore the Zeppelin notebook UI, its toolbar, paragraphs, and interpreter bindings to run code, visualize results, and enable collaborative, interactive data engineering and data science workflows.

Learn how to format Apache Zeppelin notebooks with markdown to document, annotate, and present data; apply headers, emphasis, lists, links, images, tables, and code blocks for clear collaboration.

Learn to create, edit, and run paragraphs in Apache Zeppelin using Spark and SQL, with code and markdown blocks. Explore paragraph anatomy and practical tips for modular data pipelines.

Explore the Apache Zeppelin UI to create notebooks and run paragraphs using Spark and Markdown interpreters. Learn to manage paragraphs, outputs, and interpreter settings from localhost:8080 or a cluster.

Explore how Apache Zeppelin converts data into visual insights with tables and charts like bar, line, pie, and scatter plots, and learn to label axes for dashboards.

Explores five default charts in Zeppelin and demonstrates creating bar, pie, area, line, and scatter charts from an employee data frame, highlighting interactive visuals for business decisions.

Configure and use the Apache Spark interpreter in Apache Zeppelin to write, run, and visualize Spark jobs via a notebook interface, with Spark SQL, DataFrame, and RDD support.

Compare RDDs and DataFrames in Apache Spark within Zeppelin, highlighting RDDs for fine-grained control and DataFrames for optimized, SQL-like data manipulation and visualization.

Run Spark SQL queries on dataframes within Apache Zeppelin and visualize results with built-in charts. Cache dataframes to speed repeated queries and enable interactive analytics.

Visualize spark outputs in Apache Zeppelin by turning spark data frames into interactive tables and charts using temp views and the percentage SQL interpreter, with bar, line, and pie charts.

Track Spark job performance and tune execution in Apache Zeppelin, using Spark Web UI, caching, repartitioning, and broadcasting joins to optimize pipelines.

Load csv data into spark data frames for country and indicator datasets from world development indicators project, using header true and infer schema true, and show the first 20 rows.

Create temporary views for Spark SQL queries to access data frames through SQL, enabling quick exploration and filtering with Spark SQL in Zeppelin.

Analyze income inequality with the Gini index using the World Development Indicator data set in spark sql on Zeppelin. Track China's Gini time series to compare trends across countries.

Analyze youth literacy rate using Spark SQL on the World Development Indicator dataset, ranking countries by literacy in 1990 and 2010, and visualizing results for policy insights.

Explore trade as a percentage of GDP for India and China to compare openness and global integration, using a Spark SQL query and a line chart.

Analyze exports of goods and services for India and China using World Development Indicators, Spark SQL, constant 2015 USD; note China's WTO surge in 2001 and India's IT services growth.

Analyze imports of goods and services for India and China using the World Indicator dataset with Spark SQL to explore trade balances, industrialization, and policy implications.

Analyze GDP per capita at purchasing power parity for India and China using Spark SQL and World Development Indicator data set. Track living standards and income convergence across decades.

Analyze poverty headcount ratios at $2.5 per day for India and China using the World Development Indicators and Spark SQL to compare historical poverty trends.

Analyze life expectancy at birth for India and China using spark sql with the World Development Indicator data set, compare trends, and visualize results with a line chart.

Analyze urban population growth in India and China using Spark SQL and the World Development Indicators dataset to compare urbanization trends and their impact on infrastructure, housing, and SDG 11.

Analyze infant mortality rates for India and China using Spark SQL and the World Development Indicators dataset to assess healthcare quality and access.

Compare the ten poorest countries in 1962 and 2014 using Spark SQL and the World Development Indicator data set, analyzing GNP per capita in current US dollars.

Compare the ten richest countries in 1962 and 2014 using Spark SQL and the World Development Indicators dataset, analyzing GNP per capita ny.gnp.pcap.cd in nominal dollars.

Analyze average income trends from 1960 to 2014 in seven rich countries using spark SQL on the World Development Indicator data set, focusing on GNI per capita ny.gnp.pcap.cd.

Analyze income trends in four poor countries from 1960 to 2014 using Spark SQL and the World Development Indicator data set, measuring gross national income per capita in US dollars.

Compare four countries Malawi, China, Luxembourg, and the United States in 1962 using GNP per capita in nominal dollars, analyzed with Spark SQL to illustrate global inequality.

Compare 2014 average income across Malawi, China, Luxembourg, and the United States using GNP per capita, ranked by income with Spark SQL, highlighting global inequality and China's rise.

Explore life expectancy in France from 1960 to 2013 using spark sql and world development indicators dataset, with the indicator code s p dot d y n dot l 00.

Analyze birth rate trends in the G7 from 1960 to 2013 using Spark SQL and the World Development Indicator, highlighting declines and implications for aging populations and policy responses.

Explore a world development indicators analytics project built with Apache Spark, running on a free public community edition server, with step-by-step code explanations and guidance to publish the project online.

Download and unzip the provided files uploaded in the code to access the mandatory resources on Windows, Linux, or Mac, and locate them in the course folder.

Learn how to create a free Databricks account by navigating sign-up flows, entering work email, confirming your registration, and signing in to access the platform.

Log in to Databricks and create a free community edition account by filling details, getting started for free, and logging in to access the community edition home page for practice.

Adjust playback speed, video quality, and captions to tailor your course taking experience. An auto-generated transcript supports accessibility in the Apache Spark project World Development Indicators Analytics course.

Log in to Databricks, drop the world development indicators file to upload, open it, and import to complete the normal import process.

Explore the world development indicators analytics project using Apache Spark to load World Bank data, analyze global indicators, and visualize the results in Spark.

Explore the world development indicators dataset in the Apache Spark project, featuring 247 country entries and 31 columns. Identify key fields like country name, indicator, and value for analytics.

Log in to the website, open the cluster section, and name the new cluster sparklers before creating it and watching the Supercluster move from pending to active.

Explore Spark notebook basics by connecting to a cluster, executing code on the cluster, and using notebook cells to insert, edit, and document workflows with magic commands.

Learn to load data into a Spark DataFrame for world development indicators analytics by enabling header and inferred schema, uploading files, and previewing schema and data for analysis.

Demonstrate how the Gini index measures income distribution and inequality using world development indicators, displaying country-specific values by year and comparing nations such as China and Argentina.

Learn to extract and plot the youth literacy rate from world development indicators by joining country and indicators tables, filtering by indicator code, and querying for 1990 and 2010.

Analyze trade as a percentage of GDP for China and India using world development indicators, group results by country, and visualize trends in an Apache Spark notebook.

Explore exports of goods and services in constant US dollars using world development indicators, examining trends for India and China and the impact on poor countries.

Examine imports of goods and services and compare them with exports by extracting values and country codes from the indicators table, then plot a line graph by year and country.

Explore GDP per capita within world development indicators by analyzing the indicator, visualizing the graph, and using grouping and plotting options to compare India and China.

Explore poverty alleviation through world development indicators analytics by comparing countries such as India and China, decoding indicators and country codes, and interpreting the resulting graph.

Compare life expectancy at birth between India and China, explain how yearly changes are calculated from indicator values, and interpret the life expectancy data as it appears in the table.

Explore urban population growth in India and China and compare how it changes with youth trends. Learn to plot and visualize line graphs using indicators data to inform important decisions.

Explore how infant mortality, as a measure of health care, compares India and China, noting data gaps for China from 1960 to 1968 in the visualization.

Examine the ten countries with the lowest average income in 1962 and 2014, comparing income values across years using world development indicators data.

Explore the ten countries with the highest average income in 1962 and 2014, and analyze shifts in income rankings within the world development indicators analytics project.

Explore average income trends from 1960 to 2014 in rich countries using world development indicators. Learn how to plot the graph by grouping values by country name.

Analyze average income in poor countries from 1960 to 2014 using world development indicators, focusing on Malawi and the Central African Republic, and learn to plot and compare values.

Explore 1962 indicators for Malawi, China, Luxembourg, and the United States, and learn to configure a bar chart with country names and values.

Analyze average income in 2014 across four countries—Malawi, China, Luxembourg, and the United States—and order the results by income ascending, with the option to change the selected countries.

Explore life expectancy in France from 1960 to 2000 using a line chart to visualize indicators and plot the x values for the selected country data.

Analyze G-7 birth rates from 1960 to 2013, grouping by country and indicator, to chart trends for Canada, France, Germany, the United Kingdom, Italy, Japan, and the United States.

Explore how to view world per capita income in 2013 by selecting indicators and a country, then display the data on a world map with the appropriate plotting options.

Publish your workbook to the web by selecting publish, copy the link, and save it for easy sharing in any browser to a recruiter.