The Development Environment#

The development environment for the cloud is complex because Docker and Kubernetes work differently on the major OS platforms and because of the need to install tools from the cloud vendors. In this lab you will have to choose the environment for yourself based on your machine and level of setup effort. There are tradeoffs that I will discuss so that you can make the best choice. You are not locked in. You can change your mind at any time.

Platform Variant Environment Notes
Linux Ubuntu Native
Cloud Shell, Local VM, Remote VM
Linux Other Local VM
Native, Cloud Shell, Remote VM		 Install Ubuntu in the VM
Windows WSL2 Native
Cloud Shell, Remote VM		 Native is the same as having a local VM
Windows non-WSL2 Cloud Shell
Remote VM		 Windows versions older than 10 should use the cloud shell.
Mac x86_64 Native
Cloud Shell, Local VM, Remote VM
Mac M1 Cloud Shell
Remote VM		 Docker containers built locally cannot run on Google instances, which are all x86_64. Local x86_64 VMs will be slow.
Chromebook Cloud Shell If you have a Chromebook come see me.

Google Cloud Shell and Editor#

Once you have access to Google Cloud Console you’ll be able to access the Cloud Shell Editor as shown in class. As you can see from the picture below the Cloud Shell Editor is similar to vscode. It’s based on Apache Thea, a close cousin of vscode.

Google's cloud shell editor

The editor runs on a Linux VM with all of the software installed except for Ansible. To install Ansible follow the instructions below:

python3 -m pip install pip 
pip install ansible

The editor is all you need for the course and has some real advantages, like being available wherever you need it. The disadvantage of the cloud shell editor is that it’s slower and less responsive than running vscode on your own machine. The VM that runs the cloud shell editor is quite limited so you may not be able to build and run larger Docker containers, which may be a problem.

Resetting the Cloud Shell Environment#

If you have used the Cloud Shell before and you want to start fresh you can reset your home directory in the Cloud Shell. This operation cannot be undone!

Native Development Environments#

By far the fastest and most convenient way to develop cloud applications is to setup your computer with everything you need. This gives you access to the most cost effective compute resources (as compared to buying a VM) and is usually the easiest way to manage files. However, unlike using a VM in the cloud your files are not accessible from anywhere and setting up a local environment can be tricky.

Ubuntu#

These instructions will setup either a personal Ubuntu machine or a jump box that you’re using to support another OS setup.

Make Sure curl is Installed#

The curl program must be installed and is used by the next steps.

sudo apt install curl

Install Google Cloud SDK#

sudo apt update
sudo apt-get install -y apt-transport-https ca-certificates gnupg
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
sudo apt-get update -y && sudo apt-get install -y google-cloud-sdk

Install Terraform (Optional)#

sudo apt-get update && sudo apt-get install -y gnupg software-properties-common curl
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt-get update && sudo apt-get install terraform

Install Python and Ansible#

sudo apt install -y python3-pip 
python3 -m pip install pip 
pip install ansible

Install GIT#

sudo apt install git

Install Docker Engine#

Warning

Do NOT install Docker if you’re using Ubuntu on WSL2. Docker is already setup to nest with your Windows installation. This will break it.

sudo apt-get -y update
sudo apt-get install -y apt-transport-https ca-certificates curl gnupg lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update -y
sudo apt-get install -y docker-ce docker-ce-cli containerd.io
sudo groupadd docker
sudo usermod -aG docker $USER

You have to logout and login again to assume the docker group.

Windows#

Docker on Windows creates and runs containers inside of a VM. On Windows 10 and 11 there is the option to run containers in the Windows Subsystem for Linux Version 2 (WSL2). Follow this guide on Setting up a WSL development environment. Not all of the steps are necessary. Perform the following ones:

  1. Get started

  2. Set up your Linux username and password

  3. Update and upgrade packages

  4. Use Visual Studio Code

  5. Set up version management with Git

  6. Set up remote development containers with Docker

At the end you should be able to get an Ubuntu prompt in vscode that supports the docker command.

Attention

Perform the steps in Installation on Ubuntu to make sure the Ubuntu VM is configured. DO NOT install Docker engine on Ubuntu, it’s already installed.

Mac#

Most of the tools that run on Ubuntu will run similarly on a Mac. One exception is that Docker Desktop creates a custom VM running Linux, instead of running containers natively. Local development should be possible but there may be hiccups along the way. Follow the guides below to setup what you need:

Warning

Docker does not yet fully support M1 Macs! Most things should work.

  1. Install Docker Desktop

  2. Setup vscode

  3. Install the Remote Development Extension Pack for vscode

  4. Install git

  5. Install Google Cloud SDK

Virtual Machine Environments#

Having trouble with your local environment? Is Windows/OSX too old? Is your computer too slow? If you answered “yes” to any of these the way to go is to create an Ubuntu VM and use vscode to connect to it over SSH. In this setup the front-end of vscode runs on your computer but the terminal and the files all live on the VM. This is the goto environment for most cloud developers because it’s less sensitive to changes to your computer. There are two variants of this setup:

  1. Run the VM locally on your own machine which is fast but resource intensive.

  2. Run the VM in the cloud which costs money but you’ll never loose your files.

Build a Local VM#

If your machine has the resources to run a VM then it’s best to use them. Running a VM will reduce your performance and battery life compared with the local setup but will be faster than running a remote VM.

Install VirtualBox and Vagrant#

On Windows, Linux and Intel-based Macs the best way is to use Vagrant to create your VM. Do the following:

  1. Download and Install VirtualBox

  2. Install Vagrant

Create a Vagrant Box#

In an empty directory create a new Vagrant box:

$ vagrant init ubuntu/focal64

Start the Box#

Start your new box. The first start will download the Ubuntu image and may take a while. Subsequent starts will be faster.

$ vagrant up

Vagrant creates a port-forward so that you can easily SSH into the new machine. You should see something like this in the output:

==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key

Configure SSH#

Add the SSH information to your ~/.ssh/config file. This will make it available for use in vscode. Put in the following stanza:

Host devbox
    HostName 127.0.0.1
    Port 2222
    User vagrant
    IdentityFile /path/to/Vagrantfile/.vagrant/machines/default/virtualbox/private_key
    StrictHostKeyChecking no

Important

Change /path/to/Vagrantfile to match the new directory you created.

Connect with vscode#

In the remote systems explorer in vscode you should see “devbox”. You might have to press the “reload” button. Selecting the devbox will open a new vscode window. That’s your development environment! Follow the instructions for setting up Ubuntu in this document.

Cloud VMs#

Using a VM in the cloud is a great way to go. You can scale the machine to meet your needs and you never have to worry about loosing your files. However, you will be charged money for the time that the machine is running, so remember to turn it off when you’re not using it. These instructions are for Google Cloud but your VM can be anywhere (I use opus for some development tasks).

Install the Google SDK#

This procedure uses the gcloud tool. Follow the Cloud SDK installation instructions.

Create a Project and Enable Billing#

Follow the instructions to create a project using the console (not the command line). The GUI will prompt you for a billing account, be sure to select your “Billing Account for Education” so that you use the credits you received through class.

Set the Default Project, Region and Zone#

It helps to have a default regions and zone. If you don’t have one set there has to be a --region and --zone argument to most gcloud commands. Here’s an inexpensive choice:

$ gcloud config set project your-project-id
$ gcloud config set compute/region us-central1
$ gcloud config set compute/zone us-central1-b

Create the Dev VM#

Use this command to create your dev box:

$ gcloud compute instances create devbox --machine-type=e2-micro --create-disk=auto-delete=yes,boot=yes,device-name=instance-1,image=projects/ubuntu-os-cloud/global/images/ubuntu-2004-focal-v20220110,mode=rw,size=32

Configure SSH to the Dev VM#

The gcloud command will configure SSH for you. Just run:

$ gcloud compute config-ssh

After this you should see the VM in your vscode remote explorer.

Start and Stop the Dev VM#

I’ve selected a VM that even if you leave on all the time you shouldn’t run out of money. However, it’s best to turn it off when not in use:

$ gcloud compute instances stop devbox

Every time you turn it off and on again it will get a new IP address. When you start the VM you have to reconfigure SSH with the new address:

$ gcloud compute instances start devbox
$ gcloud compute config-ssh

Connect with vscode#

In the remote systems explorer in vscode you should see “devbox”. You might have to press the “reload” button. Selecting the devbox will open a new vscode window. That’s your development environment! Follow the instructions for setting up Ubuntu in this document.