Build Your Own Super Computer with Raspberry Pis

Name: Build Your Own Super Computer with Raspberry Pis
Rating: 4.4 (543 reviews)

Create a computer cluster using Raspberry Pi boards - everything from hardware, software, design and networks

Created byBen Heidorn

Last updated 4/2016

English

What you'll learn

Build a super computer using multiple Raspberry Pi 2 B+ computers with Raspbian
Create a working Raspberry Pi from a basic Raspbian installation and customize it to your needs
Create and spool up slave Raspberry Pi nodes in a fraction of the time it takes to set up each one
Manage the Raspberry Pi computer cluster entirely using the Linux command line interface with comfort and ease
Install OpenMPI, compile and run multi-processor programs from the ground up
Learn the fundamentals for writing a multi-threaded program in C
Create a computer network using your home router, or expand to a production-level network with a managed switch
Export and mount drives and communicate between your Raspberry Pis all on the command line

Course content

13 sections • 51 lectures • 2h 7m total length

An Introduction to the Course4:46
An introduction to the course with a full rundown of the curriculum and a preview of the cluster you'll build.

Materials for 2-Pi Cluster1:00
A PDF with all of the materials needed to build a two-node Raspberry Pi cluster, the starting point for scaling up to an eight-Pi cluster (and larger!)
An Overview of the Materials to Build a 2-Pi Cluster3:27
An overview and introduction to all of the materials needed to build a two-board Raspberry Pi cluster - the first step towards building a cluster of any size.
Mac OS X - Downloading the NOOBS Package0:43
Download NOOBS with the Raspbian Operating System - we'll use this package to install Raspbian to your Raspberry Pis.
Mac OS X - Download and Run SD Formatter 4.0 to Format New SD Cards2:07
Format a Micro SD card so it's prepared for the NOOBS package. Mac OS X version of the lecture.
Windows - Download and Run SD Formatter to Format SD Cards2:20
Format a Micro SD card so it's prepared for the NOOBS package. Windows version of the lecture.
Mac OS X - Install NOOBS to the Newly Formatted SD Card0:38
Transfer the NOOBS package to the newly formatted Micro SD card. Mac OS X version.
Windows - Download and Install NOOBS to SD Card2:22
Download and transfer the NOOBS package to the newly formatted Micro SD card. Windows version.

Plugging in a Raspberry Pi for the First Time6:05
An overview of plugging in the Raspberry Pi for the first time, including some advice and general pleasantries.
Installing Raspbian with NOOBS1:20
An overview of the first boot up, and how to install Raspbian using NOOBS.
First Boot after Raspbian OS Installation0:59
The brief follow-up to the Raspbian installation, which leads into the OS configuration.
Raspi-Config - Raspberry Pi Starting Configuration1:59
Configuring the new Raspberry Pi and preparing it to be a cluster node.
Mac OS X - Using the Terminal to SSH into your Raspberry Pi1:40
Securely access the Raspberry Pi from afar using SSH. Mac OS X version.
Windows - SSH Into Your Raspberry Pi with Putty Terminal2:12
Securely access the Raspberry Pi from afar using SSH and the Putty Terminal. Windows version.
Updating & Upgrading Raspbian, and Installing the Necessary Packages4:52
Update and upgrade your Raspbian distribution, and then install all of the necessary packages for running your new cluster.
Linux Command Sheet 11:00
A review of all of the Linux commands used in this section.

Plugging in the Second Raspberry Pi to the Cluster2:26
Plug in the second Raspberry Pi node and start to build a cluster.
Setting up the Configuration on the Second Pi (raspi-config)1:39
An overview of the configuration for the second (and all additional) Raspberry Pi nodes.
Setting Up and Testing the Packages on the Second Pi3:00
We'll update, upgrade and install all of the packages that we installed the first time, but this time with a bit more brevity. This setup prepares our slave node which will be cloned to make our army of Raspberry Pis.

Setting Static IP Address on Raspberry Pi2:53
How to set up a static IP address for your Raspberry Pis directly through the network interface.
Setting Static IP Addresses with a Router2:57
An example of how to set the static IP addresses for a router that automatically sets dynamic IP addresses. There are multiple methods for setting the static IP address, depending on your router model and configuration - this is a way to set the static IP address for the Medialink router.
SSH Between Raspberry Pi Nodes1:58
Building on the earlier lecture about securely accessing your Raspberry Pis from an external computer, you can also switch between each Raspberry Pi node remotely.
Set Up Hostnames2:10
Setting up the /etc/hosts file to attribute each Raspberry Pi node to a specific hostname.
Linux Command Sheet 31:00
A review of all of the Linux commands used in this section.

Creating a Common User for All Nodes2:28
Create a specific user on all Raspberry Pi nodes with an identical user ID, which allows us to automate logging into multiple nodes and running multithreaded programs across multiple nodes.
Generating SSH Keys for the Common User4:00
Generate a secure authorized key for the newly created user which allows SSHing into the individual nodes without entering a password each time.
Linux Command Sheet 41:00
A review of all of the Linux commands used in this section.

Creating and Mounting Drives6:32
Create a new folder on the master Raspberry Pi node to export and mount a common drive across all nodes in the cluster in order to run a multithreaded program across multiple nodes.
Running a Multithreaded Program Across Multiple Pis5:07
Now the fun part: run a multithreaded program - the Pi calculation - across multiple Raspberry Pis.
Automatically Mounting Drives on Boot-Up5:20
Automatically mount the drive on the master node when the additional nodes are booted up.
Linux Command Sheet 51:00
A review of all of the Linux commands used in this section.

Materials for 8-Pi Cluster3:00
A PDF of all of the materials needed to build an eight-node Raspberry Pi cluster, as well as the materials needed to add a single additional node to the cluster, and additional optional materials.
An Overview of the 8-Pi Cluster Materials3:11
An overview of the materials used to build an eight-node Raspberry Pi cluster - a platform for building an "octo-pi" that can be used to expand to clusters of any size.
Set Up Hosts for All Nodes2:14
Update the /etc/hosts file to reflect all of the Raspberry Pi nodes planned for the cluster prior to backing up the SD card and creating clones.
Backup Raspberry Pi SD Card Image to Computer - Mac4:05
Using the command line, we'll copy the existing secondary Raspberry Pi disk image to a computer as a backup and to create cloned Raspberry Pi nodes to expand the cluster.

Create New Nodes: Write Raspberry Pi Image to New SD Card3:28
Create an additional Raspberry Pi SD card by copying the disk image to a new Micro SD card.
Create New Nodes: Update Hostname2:26
Using the newly cloned image, boot up the new Raspberry Pi node and update its hostname.
Setting Static IP Address on Additional Raspberry Pi Nodes1:51
Update the newly cloned Raspberry Pi's static IP address to distinguish it from the other nodes and match the IP address set in the /etc/hosts file.
Linux Command Sheet 61:00
A review of all of the Linux commands used in this section.

Requirements

Students will need to own a laptop or desktop with an internet connection and an SD Card Reader / Writer
Materials for this class can vary based on your ambitions, but the minimum required materials are: 2 or more Raspberry Pi computers, 2 or more ethernet cables, a router or network switch, 2 or more micro-USB power supplies (5 Volt, 1.5 to 2 Amps), a wired USB keyboard, 2 or more micro SD cards (8 or more GB), a monitor with an HDMI input, and an HDMI cable.

Description

Have you ever wanted to build your own super computer? Do you want to write programs that can run simultaneously on as many computing nodes as you wish? Do you want to learn what it takes to combine the processing powers of two, four, eight or more Raspberry Pis and run simulations at a fraction of the time and cost of running on a mainframe or even a high-end desktop computer?

If you've answered yes to any of these questions, then this course is for you. In this step-by-step series of video lectures, you'll learn how to set up your own computer cluster using cheap, affordable computers - and it doesn't get any more affordable than the Raspberry Pi.

Whether you're a Raspberry Pi enthusiast, a scientific researcher in need of cheap, expandable processing power, a programmer looking to expand their horizons with multithreaded algorithms, or a student looking to learn more about using Linux to build computing clusters, then you'll find plenty of value in this class as you build your own Raspberry Pi super computer.

These lectures are specially tailored to the Raspberry Pi 2 B+ board using Raspbian (a distribution based on Debian), which is a good place to start if you're unfamiliar with the Linux command line or building computer networks. If you are comfortable with the various flavors of Linux, you can use this course as a guideline for what it takes to build your own cluster using a different distribution.

Lastly, this class is a "living course", which means as we receive feedback and requests for specialized tutorials or updates, we'll continually update and add more to improve the quality and content of the course.

Who this course is for:

This course is designed for anyone with a zeal for learning how computers work - especially with an interest in creating your own computer cluster
You should be comfortable using a Linux command line, but you don't have to be knowledgeable - every single step of the way is recorded and documented to follow along easily
If you love Raspberry Pis and own more than one, building a computer cluster is a fun way to learn how to get them to interact together and use all of their computer processing power combined

Build Your Own Super Computer with Raspberry Pis

What you'll learn

Explore related topics

Course content

Introduction1 lecture • 5min

Assembling Materials7 lectures • 13min

Getting Started with the Master Raspberry Pi Node8 lectures • 20min

Running a Multithreaded Program Using OpenMPI5 lectures • 13min

Adding the Second Raspberry Pi to the Cluster3 lectures • 7min

Setting Up the Network5 lectures • 11min

Creating the Common User for All Nodes3 lectures • 7min

Running a Multithreaded Program on Both Raspberry Pis4 lectures • 18min

Creating Our "Octo-Pi": Building an Eight Pi Super Computer4 lectures • 13min

Spooling Up a New Computing Node4 lectures • 9min

Requirements

Description

Who this course is for: