Apache HBase : Hadoop Column Oriented NoSQL Database

Explore the Hadoop framework, an open source Java platform for distributed processing of massive data sets across clusters of commodity hardware using simple programming and map reduce programming.

Explore how hdfs, a distributed file system, enables high-performance data access across clusters by splitting files into blocks replicated on data nodes under a name node master for fault tolerance.

Big data describes the massive volume of structured and unstructured data that exceed traditional processing tools, spanning terabytes to exabytes from sensors, devices, decisions, and social media.

Explore how hadoop mapreduce enables distributed processing of large data sets on commodity clusters by dividing input into subproblems, mapping data to key-value pairs, and reducing to aggregated results.

Apache HBase scales horizontally by adding hardware to act as a single unit. It provides strong consistency, region splitting and redistribution, failover between servers, and a Java API.

Compare hdfs and hbase: hdfs is fault-tolerant and high-throughput for large data with limited random lookups; hbase provides random, low-latency access to large tables on hdfs using column-oriented storage.

Identify when to use HBase: ideal for high-volume and unstructured data, scalable storage on existing hardware, with timestamped versions managed automatically in a column-oriented store.

Use HBase for millions to billions of rows; for few thousand rows, a DBMS may be better, since HBase is not a replacement for relational databases and lacks join optimization.

Explore the building blocks of Apache HBase: tables, rows, columns, cells, and column families, and see how region servers and regions organize data and support API operations.

Explore how a column family groups related columns under a prefix and stores them together, with cells identified by family and qualifier and defaulting to the latest timestamped value.

HBase stores data on disk by column family, placing personal data and demographic data in their respective column families, with each column family having multiple files.

This lecture explains how hbase uses timestamps as versions within a cell, with a default limit per column family and queries to fetch latest or historical values by time.

Learn how Apache HBase operates on a Hadoop-based column-oriented NoSQL database, including starting daemons, the master and region servers, and using general commands to manage clusters, regions, and tables.

Explore data definition language (ddl) in hbase for defining database structures. Create tables with a table name and column family, then describe, list, check existence, and drop after disabling.

Learn to manage HBase tables using DDL commands to disable, enable, and verify status, including batch operations on tables by prefix.

Explore data manipulation language in HBase, demonstrating put, get, delete, delete all, scan, and count, and how to enable, disable, truncate, and drop tables.

Explore the core components of Apache HBase: the master server coordinates requests, region servers handle data subsets, and the admin and data interfaces manage schema and data operations.

Zookeeper coordinates the master and region servers in HBase, handling region management, heartbeats every 10 seconds, and failover to recover from region crashes.

Learn how a client interacts with an HBase cluster by using zookeeper to locate the master, access the plane to retrieve and update records, and understand the master-slave architecture.

Enable Bloom filters on column families in Apache HBase, leveraging a probabilistic data structure to test membership with space-efficient memory and faster lookups.

Explore the structure of HBase HFile, detailing edge files and store files built from blocks, their memory and write-ahead workflow, data segmentation, indexing, and optional compression.

Explain the write-ahead log in Apache HBase, showing how region servers durably log updates and deletes to enable recovery by replaying the log after crashes, including log rolling.

Explore server-side filtering in HBase using custom filters with get and scan to return a subset of data, including column prefix and timestamp filters to reduce bandwidth.

Demonstrates how the only filter retrieves only the key components during scans in HBase, showing table creation, column families, data insertion, and querying the airline data.

Explore the FirstKeyOnlyFilter and PrefixFilter in HBase, demonstrating how the first value is returned per row and how a row prefix narrows results.

Explore column prefix filter and multiple column prefix filter in Apache HBase, including syntax and examples on airline table scans using column qualifiers like source and destination.

Learn how to apply column and page filters in HBase with ColumnCountGetFilter and PageFilter to limit columns and pages during table scans, with practical examples.

Explore inclusive stop filters and qualifier filters in HBase, using scans to stop at a row or apply compare operators to qualifiers to control which rows and columns are returned.

Learn how to use the value filter in HBase to return values that meet a comparison or prefix condition during a scan, specifying the table, column family, and qualifier.

Integrate HBase with Pig by setting up a Hadoop environment, loading data into HDFS, and creating a table with a column family, then loading and inspecting the data.

Learn to integrate HBase with Pig tool part II by creating hdfs directories, copying local data to hdfs, and validating content with ls and cat.

Explore integrating HBase with Pig tool through hands-on steps to create HBase tables, define schema, load and verify data via MapReduce workflows, and query with scans.

Import data from a school database into Apache HBase using Sqoop, detailing table creation, data insertion, and the Sqoop import workflow to transfer RDBMS data to HBase.

Learn to integrate Apache HBase with Sqoop by creating a MySQL database and table, inserting data, configuring Sqoop to load into HBase, and verifying the loaded records.

Explore integrating Apache HBase with Apache Hive, building a Hive data warehouse, creating an employee database and table, loading a sample dataset, and validating data via select queries.

Learn to integrate HBase with Hive by creating an HBase table and column family, mapping to a Hive external table, and loading DSV data from HDFS for verification.

Create a table in Apache HBase using Java API by instantiating the configuration, building a table descriptor with column families, and invoking the createTable method to deploy an empty table.

Insert data into an hbase table with java by creating a configuration, instantiating the table, and using a put with a column family, qualifier, and value, then save and close.

Learn to scan an HBase table with the Java API by configuring, instantiating scan and table objects, selecting column families and qualifiers, and printing results.

Demonstrate how to disable an HBase table using Java by instantiating the admin with a configuration, verifying with isTableDisabled, and invoking disableTable with the table name, then printing the result.

List all tables in an HBase instance using the Java API, print their names from table descriptors, and enable a disabled table with enableTable after checking isTableEnabled.

Delete a table in Apache HBase using the Java API by disabling the table, parsing its name, calling the delete table method, and verifying deletion by listing tables.