How to run a Jupyter Notebook on an AWS EC2 instance

Getting started

To leverage the AWS Cloud compute engine for hosting a Jupyter server, we must have an AWS account. With this account, we can initiate an EC2Elastic Cloud Compute is an Amazon service that allows the user to rent virtual machines to perform their own tasks instance, allowing us to configure and establish a Jupyter server. This, in turn, allows us to harness the capabilities of cloud computing for executing various machine learning tasks.

Step 1: Creating an EC2 instance

To create an EC2 instance, log in to the AWS console, type EC2 into the AWS console search bar and click EC2 in the dropdown. This takes us to the EC2 dashboard. Click “Instances” under the “Instances” menu in the left sidebar. From there, click “launch instances” in the top right corner.

Give this new EC2 Instance a name and select the AMI (Amazon Machine Image) as “Amazon Linux 2 AMI (HVM) Kernel 5.10, SSD Volume Type” AMI, with 64-bit (x86) architecture. In the “Instance type” menu, select “t2.micro.” In the “Key pair” section, select the “Create new key pair” option, enter the name of the key pair in the text field, and click “Create key pair.” This will create and download the key to the local machine. Finally, click “Launch instance” to create the EC2 instance.

Step 2: Customizing the EC2 instance

We need to add a new security group to our EC2 instance to allow the Jupyter server to run on this instance. From the EC2 dashboard, click the “Security Groups” option under the “Network & Security” tab under the left-side menu. Click the “Create security group” button from the top right side. Give the security group a name and a description, and select the VPC as the default VPC. Furthermore, click the “Add rule” button under the “Inbound rules” tab and add the following rules:

• Select “SSH” from the “Type” drop-down menu, and from the “Source,” select “Anywhere-IPv4.”

• Select “HTTPS” from the “Type” drop-down menu, and from the “Source,” select “Anywhere-IPv4.”

• Select “Custom TCP” from the “Type” drop-down menu. In the port range, enter “8888,” and from the “Source,” select “Anywhere-IPv4.”

Afterward, click the “Create security group” button at the bottom of the screen to create the security group. Now, we need to attach the security group to the EC2 instance. To do this, go to the “Instances” option from the left sidebar and select the instance we just created. From the top right, click the “Actions” button and go to “Security” and “Change security groups.” Search for the created security group and click “Add security group.” Click “Save” to attach the security group.

Step 3: Connecting to the EC2 instance

We will connect to the EC2 instance using the playground provided below. Make sure that you have downloaded the key pair .pem file so we can establish an SSH connection to the EC2 instance shell.

To connect to the EC2 instance, open the key pair file using a text editor, replace the content in the key.pem provided below in the code playground, and click the Run button. This will open a console to establish an SSH connection using the key.pem file. Execute the command given below in the terminal:

Make sure to replace the <EC2 PUBLIC IPv4 ADDRESS> with the public IPv4 address of the EC2 instance. To find the public IPv4 address, go to the EC2 dashboard and click the “Instance ID” of the EC2 instance to open the “Instance summary.” Here, we can find the “Public IPv4 address.”

Now, you can execute all the commands to run your Jupyter server provided below in this terminal window.

Step 4: Installing Jupyter Notebook

To install the Jupyter Notebook, first, we will need to install Miniconda to configure a virtual environment. We use Miniconda instead of Anaconda because Anaconda includes many data science libraries, while Miniconda provides a minimal environment that you can customize. To download and install the package, type the following commands in the EC2 instance shell window:

Agree to the license terms and wait for Miniconda to finish the installation. Afterward, add Conda to the environment variables path. To do that we need to set the PATH environment variable to the directory containing the Conda executable. To do that, run the command given below:

Here, $HOME indicates the root directory of the EC2 instance and $PATH tells where it can find the executable for Conda. We will also initialize Conda for shell by running the command below:

Now, we use Conda to create a virtual environment. To do that, use the following commands:

Finally, we will install Jupyter in our environment. To install Jupyter, run the commands given below:

Step 5: Running the Jupyter Server

To start the Jupyter server, we need to type the following command into the shell:

The --ip=0.0.0.0 flag tells the Jupyter Notebook server to accept connections from anywhere and the --no-browser flag is used to not open a browser window when the server starts running. We will see a link with the following format in the shell when we run the Jupyter server http://<PRIVATE IP>:8888/tree?token=<TOKEN>. Here, replace the <PRIVATE IP> with the Public IPv4 DNS to access the Jupyter Notebook. We can find the Public IPv4 DNS by navigating to the EC2 dashboard and clicking the “Instances” option from the sidebar. Then, click the “Instance ID” of the EC2 instance to open the “Instance summary.” Here, we can find the Public IPv4 DNS. Finally, paste the link with the following format http://<PUBLIC IPv4 DNS>:8888/tree?token=<TOKEN> into a new tab in the browser.

Congratulations! Our Jupyter Notebook server is up and running. We can connect to this server remotely from anywhere and utilize the power of the EC2 instance that we created for all machine learning and data analysis tasks. You can create an EC2 instance with better resources if you need more power for your use case; just make sure that you calculate the cost beforehand and set up a monitoring system so you don’t get charged extra.