Running Elasticsearch & Kibana in Elastic Cloud

Bo Andersen
A free video tutorial from Bo Andersen
Lead Developer
4.5 instructor rating • 4 courses • 62,333 students

Lecture description

Running Elasticsearch and Kibana in Elastic Cloud is the easiest way to get up and running quickly.

Learn more from the full course

Complete Guide to Elasticsearch

Learn Elasticsearch from scratch and begin learning the ELK stack (Elasticsearch, Logstash & Kibana) and Elastic Stack.

12:10:23 of on-demand video • Updated September 2020

  • How to build a powerful search engine with Elasticsearch
  • The theory of Elasticsearch and how it works under-the-hood
  • Write complex search queries
  • Be proficient with the concepts and terminology of Elasticsearch
English Before showing you how to install and run Elasticsearch and Kibana on your own machine, I want to show you an even easier way; by using Elastic Cloud. Elastic Cloud is a hosted and managed solution for those who want to easily deploy an Elasticsearch cluster and avoid managing the underlying infrastructure. And best of all, it's free! You just need to click the link that I have attached to this lecture and enter your e-mail address to get 14 days for free. You don't need to add a credit card or anything, so you won't be billed for anything after your trial expires. If you don't have any experience with Elasticsearch, then this is a great way to get started. The trial should be sufficient for you to mess around with Elasticsearch while you work your way through this course. If you need more time, you can always install Elasticsearch and Kibana on your own machine later on. Note that your trial starts when you start up a cluster and not when you create your account. Alright, so let's get started setting up a cluster. I have created an account in advance by entering my e-mail address on this page, which is the one linked to in this lecture. Once that is done, you will see this page when logging in. What we need to do, is to create a cluster. I haven't told you what a cluster is yet, but I will get back to that soon. In Elastic Cloud, this is referred to as a deployment, which is just a less technical term for a cluster. Let's click the "Create deployment" button to get started. The first thing we need to do, is to give our deployment a name. I will just give it a name of "Complete Guide to Elasticsearch," but you can enter anything you want. Next, we need to choose a cloud provider. That's because Elastic Cloud runs on top of a cloud provider, being either Amazon Web Services, Google Cloud Platform, or Microsoft Azure. Which cloud provider you choose mostly matters if you need to deploy Elasticsearch to production and you have the rest of your infrastructure with one of these providers. For the purpose of completing this course, it makes no difference, and you can just leave it at the default selection. Next up, we should select the region in which we want our deployment to be placed. It really doesn't matter what you choose here. The only difference is network latency. I will just choose Frankfurt because that's pretty close to me, so that will give me a low latency. We can choose which version of the Elastic Stack that we want to deploy. Unless you have a reason for doing otherwise, you should leave it at the latest version. The other two options are not useful to us, so I will just leave those unchecked. The next step is to choose a deployment template. These templates define the specifications of the hardware that will be used to host the deployment. As you can see, there is a recommended template named "I/O Optimized." If we click the "Default specs" link, we can see the specifications of the hardware that this template will deploy. As you can see, this template actually deploys quite a few things for us. In terms of Elasticsearch, it deploys two nodes. That's because it's best practice to run at least two nodes in a production environment. For development purposes, we would be fine with a single node, though. If we were to scale Elasticsearch beyond six nodes, a dedicated master node will automatically be added. We haven't covered what that is yet, so I won't get into that now. It also optionally deploys a dedicated node for machine learning jobs. We are not really interested in that for the purpose of this course, though. What we are interested in, though, is the virtual machine that will host our Kibana instance, because we will use that for communicating with Elasticsearch. The template also deploys a machine for Application Performance Management, but we won't be using that either. There are a number of templates that are optimized for various kinds of workloads. I won't get into the details of them, because you need more knowledge of Elasticsearch and its architecture before such an explanation will make sense. I will just choose the recommended template, because that is more than enough for our use case. That's actually it! It is possible to customize the deployment though, in case the templates don't suit your needs, but the template that we selected is completely fine for development purposes. So let's hit the "Create deployment" button to start everything up. After a short moment, you will be taken to a new page showing the progress of the deployment. The process of starting everything up, is going to take a couple of minutes, so I will just fast forward to when the deployment is ready. Both Elasticsearch and Kibana will be secured with a username and password, which you can find on this page. It is important that you save the password somewhere, because it won't be shown to you again. You can, however, reset the password in case you lose it. The deployment has finished, and all of the instances are ready. You can find various deployment information here, but let's head back to the main page and navigate to the deployment page from there. Now we see our deployment here, along with some basic information about it. Let's click it to go to the deployment page. This page shows us basic information about the deployment, such as the health status, and also contains a couple of actions that we can perform. Scrolling down to the bottom of the page, we can see all of the instances contained within the deployment. For the Elasticsearch nodes, we can see the memory and disk usage. Scrolling back up, we have something called a Cloud ID. This ID is needed when sending data to Elasticsearch from various Elastic Stack products, such as Beats and Logstash. We won't be using it in this course, though. What we will use, though, is the Elasticsearch and Kibana endpoints. You will need the Elasticsearch endpoint when importing test data in the next section, so now you know where to find it. We will also be using Kibana throughout this course. To open up the Kibana interface, simply click the "Launch" link. To log in, you need to type in the credentials that were shown after creating the deployment. This is also the case when communicating with Elasticsearch from the command line, by the way,| but more on that later in the course. That's it! That's how easy it is to set things up with Elastic Cloud. You can find various things in the left-hand menu, such as logs and performance insights, as well as a way to edit your deployment in case you need to do that. Anyway, you are now ready to proceed, meaning that you can skip the following installation lectures.