Reserving and accessing a VM

In this course you will get familiar with a variety of database software. To make installation standardized and less of a hassle, we will use virtual machines provided by Duke virtual computing manager. Follow these steps to reserve a virtual machine for yourself:

1. Point your Web browser to the Duke VCM website, and log in using your NetID. Then click the "Reserve a VM" button in the middle.

If you see "Error You already have your maximum allotment of VMs. ", goto OIT Help and then click "chat now" or send OIT an email talking about increasing the max VM allotment due to class need. The approvement of increasing of max VM allowed will take hours so we suggest do it sooner rather than later.

In the following page, select from the drop down menu: "Plain VM: No Apps - Ubuntu18.04". Click "reserve" and agree to the terms of use.

To make efficient use of the shared VCM resources and reduce our carbon footprint, this VM may be powered down every morning at 6:00 AM. Then you need to log in to VCM and click the "Power on" button to turn your VM back on. You can un-check the Automatic power down check box to opt out of automatic shutdowns.

2. It may take some time for the VM to be ready for use. Once it is ready, you will access the machine via SSH, which requires a client program. On MacOS, you may use the Terminal.app that came with it. On Windows, you can use power shell that came with it, or you could install PuTTY, or MobaXterm so your connection info can be stored.

3. After obtaining required SSH client software, use the information displayed on the VCM web page to access your machine. The credentials we need are "Hostname", "Admin user", and "Admin password". Make sure you are using admin user "vcm" and the admin password (reveal by clicking the button). This step varies depends on what software you use. If you use a software with a graphical interface, please follow the interface to input those credentials at appropriate places. If you use a command line (such as MacOS Terminal), the command should be ssh vcm@vcm-XXXXXX.vm.duke.edu (replace with your Hostname. Hit enter to run the command). Then you might encounter a warning saying that the authenticity of host can't be established; it's okay to type yes to continue.

4. To conveniently transfer files between your local machine and the VM, we recommend using FileZilla. Establish the connection by choosing "sftp" protocol, not "ftp" protocol. The parameters "host", "user", "password" are the same as the ones when you are doing ssh. Leave the "port" parameter blank.

Setup the VM

Once you have created you fresh VM, you need to get it ready for the course. Make sure you have a fast, reliable internet connection before you start.

Note: Much of this step involves setting up git, which is great tool for collaborative development and version management. We use git (behind the scene) to keep your VM up to date with the myriad of tools and data that we will use in this course. While we don’t require that you know git, we encourage you to explore it further because it can be very useful for your team project.

Follow these steps carefully:

1. Point your Web browser to the Duke gitlab website, and sign in using Shibboleth (with your Duke netid). Once you are in, click on the profile drop-down on the upper-right corner, and select “Settings” and then “SSH Keys.” A form for add a new SSH key will open up waiting for you to fill out; we will come to back to this.

2. On your VM command line, type in these commands (hit enter key after pasting each one to execute it, then wait for it to finish):
wget -N https://www2.cs.duke.edu/courses/spring20/compsci316/static/init.sh
bash init.sh
Follow the instructions. You will be asked to copy and paste your public key to the Duke gitlab website, and don’t press [ENTER] until you’ve successfully registered your public key! Make sure you copy the entire public key (which is a single line of text that starts with ssh-rsa and ends with @duke.edu, and wraps to form multiple lines). Switch back to your Web browser, paste the copied output into the text box for “Key,” enter a title for your key (anything is fine), and then submit the form. After a short while, you should see confirmation that your key has been added (if not, you will have to try adding the key again before proceeding further). At this point, switch back to your VM shell and hit [ENTER] to continue.

Note: When running init.sh, if you type something wrong and find that your [BACKSPACE]/[DELETE] key doesn’t work, just hit [CTRL]-c and re-run bash init.sh when you are back at the shell prompt.

Note: Even though the public key wraps to multiple lines when displayed, it should be copied and pasted as a single line. If for some reason it becomes multiple lines in the text box that you copied into, just carefully delete the line breaks manually.

Note: When running init.sh, you might encounter a warning saying that the authenticity of host ‘gitlab.oit.duke.edu (…)’ can’t be established; it’s okay—just type yes to continue.

3. If all goes well, you will notice git pulling files from the course repository. Once it’s done, issue the following command in your VM shell:
/opt/dbcourse/sync.sh
This command will likely take quite a while as it installs a lot of tools necessary for the course. Throughout the course, you may need to re-run this command every now and then to keep your VM updated; we will remind you when it is necessary.

4. Reboot your VM for the softwares to be ready to use. You may do this from VCM web page or using command sudo reboot now. This will terminate the SSH connection. To access your VM you need to log in again (step 3 of the previous part)..