The Development Environment¶
The development environment for the cloud is complex because Docker and Kubernetes work differently on the major OS platforms and because of the need to install tools from the cloud vendors. In this lab you will have to choose the environment for yourself based on your machine and level of setup effort. There are tradeoffs that I will discuss so that you can make the best choice. You are not locked in. You can change your mind at any time.
| Platform | Variant | Environment | Notes |
|---|---|---|---|
| Linux | Ubuntu | Native Cloud Shell, Local VM, Remote VM |
|
| Linux | Other | Local VM Native, Cloud Shell, Remote VM |
Install Ubuntu in the VM |
| Windows | WSL2 | Native Cloud Shell, Remote VM |
Native is the same as having a local VM |
| Windows | non-WSL2 | Cloud Shell Remote VM |
Windows versions older than 10 should use the cloud shell. |
| Mac | x86_64 | Native Cloud Shell, Local VM, Remote VM |
|
| Mac | M1 | Cloud Shell Remote VM |
Docker containers built locally cannot run on Google instances, which are all x86_64. Local x86_64 VMs will be slow. |
| Chromebook | Cloud Shell | If you have a Chromebook come see me. |
Google Cloud Shell and Editor¶
Once you have access to Google Cloud Console you’ll be able to access the Cloud Shell Editor as shown in class. As you can see from the picture below the Cloud Shell Editor is similar to vscode. It’s based on Apache Thea, a close cousin of vscode.
The editor runs on a Linux VM with all of the software installed except for Ansible. To install Ansible follow the instructions below:
python3 -m pip install pip
pip install ansible
The editor is all you need for the course and has some real advantages, like being available wherever you need it. The disadvantage of the cloud shell editor is that it’s slower and less responsive than running vscode on your own machine. The VM that runs the cloud shell editor is quite limited so you may not be able to build and run larger Docker containers, which may be a problem.
Resetting the Cloud Shell Environment¶
If you have used the Cloud Shell before and you want to start fresh you can reset your home directory in the Cloud Shell. This operation cannot be undone!
Native Development Environments¶
By far the fastest and most convenient way to develop cloud applications is to setup your computer with everything you need. This gives you access to the most cost effective compute resources (as compared to buying a VM) and is usually the easiest way to manage files. However, unlike using a VM in the cloud your files are not accessible from anywhere and setting up a local environment can be tricky.
Ubuntu¶
These instructions will setup either a personal Ubuntu machine or a jump box that you’re using to support another OS setup.
Make Sure curl is Installed¶
The curl program must be installed and is used by the next steps.
sudo apt install curl
Install Google Cloud SDK¶
sudo apt update
sudo apt-get install -y apt-transport-https ca-certificates gnupg
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
sudo apt-get update -y && sudo apt-get install -y google-cloud-sdk
Install Terraform (Optional)¶
sudo apt-get update && sudo apt-get install -y gnupg software-properties-common curl
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt-get update && sudo apt-get install terraform
Install Python and Ansible¶
sudo apt install -y python3-pip
python3 -m pip install pip
pip install ansible
Install GIT¶
sudo apt install git
Install Docker Engine¶
Warning
Do NOT install Docker if you’re using Ubuntu on WSL2. Docker is already setup to nest with your Windows installation. This will break it.
sudo apt-get -y update
sudo apt-get install -y apt-transport-https ca-certificates curl gnupg lsb-release
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update -y
sudo apt-get install -y docker-ce docker-ce-cli containerd.io
sudo groupadd docker
sudo usermod -aG docker $USER
You have to logout and login again to assume the docker group.
Windows¶
Docker on Windows creates and runs containers inside of a VM. On Windows 10 and 11 there is the option to run containers in the Windows Subsystem for Linux Version 2 (WSL2). Follow this guide on Setting up a WSL development environment. Not all of the steps are necessary. Perform the following ones:
Get started
Set up your Linux username and password
Update and upgrade packages
Use Visual Studio Code
Set up version management with Git
Set up remote development containers with Docker
At the end you should be able to get an Ubuntu prompt in vscode that supports the docker command.
Attention
Perform the steps in Installation on Ubuntu to make sure the Ubuntu VM is configured. DO NOT install Docker engine on Ubuntu, it’s already installed.
Mac¶
Most of the tools that run on Ubuntu will run similarly on a Mac. One exception is that Docker Desktop creates a custom VM running Linux, instead of running containers natively. Local development should be possible but there may be hiccups along the way. Follow the guides below to setup what you need:
Warning
Docker does not yet fully support M1 Macs! Most things should work.
Install the Remote Development Extension Pack for vscode
Virtual Machine Environments¶
Having trouble with your local environment? Is Windows/OSX too old? Is your computer too slow? If you answered “yes” to any of these the way to go is to create an Ubuntu VM and use vscode to connect to it over SSH. In this setup the front-end of vscode runs on your computer but the terminal and the files all live on the VM. This is the goto environment for most cloud developers because it’s less sensitive to changes to your computer. There are two variants of this setup:
Run the VM locally on your own machine which is fast but resource intensive.
Run the VM in the cloud which costs money but you’ll never loose your files.
Build a Local VM¶
If your machine has the resources to run a VM then it’s best to use them. Running a VM will reduce your performance and battery life compared with the local setup but will be faster than running a remote VM.
Install VirtualBox and Vagrant¶
On Windows, Linux and Intel-based Macs the best way is to use Vagrant to create your VM. Do the following:
Download and Install VirtualBox
Start the Box¶
Start your new box. The first start will download the Ubuntu image and may take a while. Subsequent starts will be faster.
$ vagrant up
Vagrant creates a port-forward so that you can easily SSH into the new machine. You should see something like this in the output:
==> default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 127.0.0.1:2222
default: SSH username: vagrant
default: SSH auth method: private key
Configure SSH¶
Add the SSH information to your ~/.ssh/config file. This will make it available for use in vscode. Put in the following stanza:
Host devbox
HostName 127.0.0.1
Port 2222
User vagrant
IdentityFile /path/to/Vagrantfile/.vagrant/machines/default/virtualbox/private_key
StrictHostKeyChecking no
Important
Change /path/to/Vagrantfile to match the new directory you created.
Connect with vscode¶
In the remote systems explorer in vscode you should see “devbox”. You might have to press the “reload” button. Selecting the devbox will open a new vscode window. That’s your development environment! Follow the instructions for setting up Ubuntu in this document.
Cloud VMs¶
Using a VM in the cloud is a great way to go. You can scale the machine to meet your needs and you never have to worry about loosing your files. However, you will be charged money for the time that the machine is running, so remember to turn it off when you’re not using it. These instructions are for Google Cloud but your VM can be anywhere (I use opus for some development tasks).
Install the Google SDK¶
This procedure uses the gcloud tool. Follow the Cloud SDK installation instructions.
Create a Project and Enable Billing¶
Follow the instructions to create a project using the console (not the command line). The GUI will prompt you for a billing account, be sure to select your “Billing Account for Education” so that you use the credits you received through class.
Set the Default Project, Region and Zone¶
It helps to have a default regions and zone. If you don’t have one set there has to be a --region and --zone argument to most gcloud commands. Here’s an inexpensive choice:
$ gcloud config set project your-project-id
$ gcloud config set compute/region us-central1
$ gcloud config set compute/zone us-central1-b
Create the Dev VM¶
Use this command to create your dev box:
$ gcloud compute instances create devbox --machine-type=e2-micro --create-disk=auto-delete=yes,boot=yes,device-name=instance-1,image=projects/ubuntu-os-cloud/global/images/ubuntu-2004-focal-v20220110,mode=rw,size=32
Configure SSH to the Dev VM¶
The gcloud command will configure SSH for you. Just run:
$ gcloud compute config-ssh
After this you should see the VM in your vscode remote explorer.
Start and Stop the Dev VM¶
I’ve selected a VM that even if you leave on all the time you shouldn’t run out of money. However, it’s best to turn it off when not in use:
$ gcloud compute instances stop devbox
Every time you turn it off and on again it will get a new IP address. When you start the VM you have to reconfigure SSH with the new address:
$ gcloud compute instances start devbox
$ gcloud compute config-ssh
Connect with vscode¶
In the remote systems explorer in vscode you should see “devbox”. You might have to press the “reload” button. Selecting the devbox will open a new vscode window. That’s your development environment! Follow the instructions for setting up Ubuntu in this document.