Advanced Topics in Cloud Networking and Computing

Graduate Seminar, University of Maryland, CS, 2025

The course aims to explore latest advances in cloud networking and computing in light of emerging workloads (e.g., large-scale data analytics), including communication platforms, compute parallelism, and datacenter networking. The class will discuss the latest developments in the entire networking stack, the interactions between networks and high-level applications, and their connections with other system components such as compute and storage. The course combines group readings and presentations of influential publications in the field, lectures by the instructor, talks by invited speakers, and a project etc.

Instructors

  • Instructor: Prof. Alan Zaoxing Liu
    • Class Time: Tue/Thu 3:30-4:45PM, IRB 1207
    • Office Hours: IRB 5138 by appointment

Annoucements

  • Week of 2/17/25, the reading will be Megatron-LM.
  • Week of 2/3/25, we will assign the first reading of the semester, DeepSeek-V3 tech report.
  • Paper review link is here.

Prerequisites

All levels are welcome. Recommended experiences with computer networking and software systems, including one or more of CMSC330, CMSC412, CMSC414, CMSC417, etc., or permission of the instructor. The assignments and projects assume students have familiarity with programming (e.g., Python and C/C++).

Textbook

There are no mandatory textbooks for this course, but every class will have corresponding readings from research papers. A reading list with links to the papers will be provided.

Course Overview

  1. Paper Reviews: Each student reviews 1 paper/class from top conferences or journals. Submit reviews before the class in four sections, including summary, paper strengths paper weaknesses, and detailed comments.

  2. Paper Presentations: Each student will select papers from the paper reading list (the list will be provided, selections are first-come first-serve) and present that paper during a lecture. The presentation will be followed by a technical discussion.

  3. Lectures: In each topic, the instructor will give one or two introductory lectures, followed by paper presentations by class participants.

  4. Open-Source Project: In Spring 2025, this class will group the students into two major open-source projects:

    • Topic-1: Cloud Infra Observability and Databases
    • Topic-2: Networking for Machine Learning

Academic Conduct Statement

Academic Conduct Statement including expectations for academic honesty, reference to consequences for cheating or plagiarism, course-specific guidelines for, e.g., extent of allowable collaboration on assignments, and URL for Academic Conduct Code.

Grading

  • Class participation: 10%
  • Paper reviews: 20%
  • Paper presentation: 20%
  • Project: 40%
  • Open-source engagment: 10%

Seminar Schedule (Keep Updating)

DateTopicsNotes
Tue 1/28Course Overview (slides)*A Datacenter Infrastructure Perspective for ML, (HPCA’18)
Thu 1/30Project Review (slides) 
Tue 2/4Visitor Talk, Fuheng Zhao, UCSBTalk: Advancing Approximate Queries with Innovative Data Summaries and Generative Models
Thu 2/6Time-series DB + NCCL 
Tue 2/11Data Parallelism*DeepSeek-V3 Technical Report,
Thu 2/13Pipeline Parallelism*GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism (NeurIPS’19), *DeepSeek-R1 Technical Report
Tue 2/18Transformer Model + Hybrid Parallelism 
Thu 2/20Hybrid ParallelismMegatron-LM

Paper Reviews

The goal of the reviews is to get you comfortable of reading research papers in the software systems and networking space.

  • Students are expected to write reviews for the papers in each class. We will give scores based on the top 90% of the reviews. This means it is ok if you miss 10% of the reviews throughout the class.
  • Your reviews are due at noon one day before (Monday noon for Tuesday classes; Wednesday noon for Thursday classes). So the presenter of the paper can have time collect all your questions and we can discuss in class. For the lectures we have guest speakers, we will collect the questions and please raise your question in class.

Project Proposal and Project Pitch Presentation

The project proposal is not graded but it serves as the good basis for your individual meeting with the instructor and for your pitch presentation. Each student should give a 10-minute talk on your project ideas. The talk should include

  • What problem are you solving?
  • Why it’s an important problem?
  • What are the potential challenges you may face in solving the problem?
  • What are the first steps (your plan for the next month)?

Midterm Project Report

  • Describe the problem you plan to solve, why it is novel/unique, what the major challenges.
  • Describe the detailed design for your project and what you have implemented/evaluated so far.
  • Describe the remaining challenges, how you would address them, and your plan for the remaining time.
  • The midterm report should be about 2-4 pages and serve as a starting point for your final project report (see detailed requirements for the final report below)

Final Project Presentations

This should be similar to a workshop talk. You might consider covering the following content (not necessarily in the same order):

  • What problem are you trying to solve?
  • Why is it an important problem?
  • What’s your basic solution to the problem?
  • What are the challenges in the problem?
  • How did you solve these challenges? Or how do you plan to solve the challenges?
  • Some preliminary results
  • Future directions