Assignments

The assignments this semester:

• Programming labs in the Go programming language (some weeks).
• A final project
• Weekly checkout survey/quiz on Canvas (most weeks)
• Readings and recommended readings to support the lecture material.

The course schedule includes release dates and due dates for the labs. This course is still under development so there may be some adjustments.

Checkouts

Checkouts are online surveys/quizzes on Canvas. Checkouts provide information to us, but we do not judge your answers: the graded score for each checkout is boolean: completed or not. We encourage you to engage with all the questions, but if you don’t like a question you can leave it blank.
Checkouts are due at 11:59pm ET each Thursday, or the posted due date on Canvas, whichever is later.

Labs

The labs ask you to write code within self-contained Go programs or modules. Complete them individually. The labs are designed to illustrate important concepts and may involve some experimentation to illustrate certain performance effects. Generally, labs release on Wednesday and are due Tuesday night almost two weeks later, so two or more labs are out at once. Per the schedule, the tempo is higher at the start to prime the pipeline, and no labs are due on exam weeks or Spring Break. We drain the pipeline before the second midterm to leave time for final projects.

• Duality. Explore the duality of shared memory and message passing for concurrency control.
• Servers. Server concurrency (workcrews) and effect of load skew on throughput and response time.
• Flows. Workflow emulation (e.g., MapReduce), data flow control, buffering, queue lengths, and impact of stragglers.
• Amo. Remote Procedure Call (RPC) with At Most Once execution through server-side duplicate suppression (reply cache); handling client failures and recovery.
• Names. Name domains and name resolution; abstractions for coordination services.
• Pense. A coin-like account service with a transact payment primitive and a recovery log, and sharding.
• Raft. Aspects of Raft consensus protocol for service replication.

Labs release as repositories in the course GitLab group.

• Fork the repository and clone the forked repository with git.
• Follow the instructions in the README.
• Use git to update, commit, and push in the usual fashion. Push early and often to protect your work.
• Submit using the GitLab button on Gradescope. The workflow should be familiar from other courses (e.g., 210).
• Watch for further guidance on Ed

Final Project

A self-directed final project applies concepts and methodologies to real-world systems. Projects may be done in groups of three. The intent is that you will experiment with an open-source software system in a cloud setting, and then: submit a repository with a short writeup (two pages max), give a short presentation in class at the end of the semester (LDOC, 5 minutes max), and present/discuss the project in a meeting with the instructor.

Readings

We draw readings from a variety of sources, including a few classic short papers (CSP, Lamport Clocks), a couple of full-strength research papers (Raft and e.g. Google File System), some tutorialish readings from ACM Queue or CACM (e.g., the Tail at Scale), relevant sections of the OSTEP text, and other sources. Web readings are linked through the course schedule. Other named readings on the course schedule are PDFs which can be found in the readings folder. Named readings in italics on the schedule are optional. We will define the scope of the readings more precisely in lectures and Ed notes.

The readings emphasize Google’s infrastructure architecture, primarily because they have published some good readings for us—and their ideas work and have shaped the field and industry. Google’s books and materials on Site Reliability Engineering are a good introduction to distributed mega-services and their operation.

Steps for Labs

The labs are distributed and submitted through the CS department GitLab site coursework.cs.duke.edu and submitted through Gradescope. It is similar to CompSci 210 and other courses, and the setup is similar, except that you will be submitting code from your laptop rather than a Duke-managed container. See the Resources tab.

For each lab:

• Log in to the coursework GitLab service in your browser with your NetID.
• Once logged in, click over to the group page for the course repository
• Click into the lab’s page/repository and read the README there.
• Fork the repository by clicking its Fork button.
• Use a git clone command with the ssh link (code button) to clone the forked repository into a local folder in your container.
• Follow the directions in the assignment’s README with care.
• To protect your work, use git commit and push often.
• For projects with a partner, pull any updates from your partner before you commit and push your own updates.
• Add, commit, and push a text file (statement.txt) listing the nature and sources of any assistance that you receive for the assignment, and anything else you want to tell us.
• Log in to Gradescope with your NetID, click into the assignment’s page, and submit it using the GitLab button there. It redirects you to coursework to authorize and select which repository to submit. Select it and push the big button. Done and done!

Take time to set up git for a smooth workflow. Time you invest to become familiar with git will pay off later: see the Resources tab.