Data 101: Data Engineering

UC Berkeley, Fall 2023

Ed Datahub Lecture Recordings Gradescope Extenuating Circumstances Regrade requests

Professor Lisa Yan

She/Her/Hers

yanlisa@berkeley.edu

Course contact: data101@berkeley.edu

Schedule

Jump to current week

Week 01

Thu 8/24

Lecture 1Welcome to Data Engineering!

Pre-Semester Form

Week 02

Mon 8/28

Discussion 1Review: Joins, SQL, Aggregation, Sets and Bags

Solution

Tue 8/29

Lecture 2SQL Review

Notes, Code, Code HTML

Thu 8/31

Lecture 3 Relational Model & Algebra

Notes

Week 03

Mon 9/4

Labor Day (no discussion)

Tue 9/5

Lecture 4Subqueries, Aggregation

Notes, code, code HTML

Thu 9/7

Lecture 5Window functions, Sampling, String manipulation

Notes, code, code HTML

Fri 9/8

Project 1SQL

Due Thu 9/21, 5pm

Week 04

Mon 9/11

Discussion 2Relational Algebra, Views, Subqueries

Solution

Tue 9/12

Lecture 6 DML, DDL, Constraints

code, code HTML

Thu 9/14

Lecture 7Foreign Keys, Index Selection

code, code HTML

Fri 9/15

MultiVitamin 1Multivitamin 1

Due Thu 9/28, 5pm

Week 05

Mon 9/18

Discussion 3String Manipulation, DML, DDL and Windowing

Solution

Tue 9/19

Lecture 8Query processing, Optimization I

code, code HTML

Thu 9/21

Lecture 9Query processing, Optimization II

code HTML

Project 1 Due, 5pm

Fri 9/22

Project 2Query Performance

Due Thu 10/5, 5pm

Week 06

Mon 9/25

Discussion 4Indexes, Query Optimization

Tue 9/26

Lecture 10[Guest Lecture] Amy Wang, Databricks

Thu 9/28

Lecture 11Data Models: Relations, Tensors, Dataframes

MultiVitamin 1 Due, 5pm

Fri 9/29

MultiVitamin 2Multivitamin 2 Release

Due Thu 10/12, 5pm

Week 07

Mon 10/2

Discussion 5Query Optimization II

Tue 10/3

Lecture 12Data Preparation I

Thu 10/5

Lecture 13Data Preparation II

Project 2Due, 5pm

Fri 10/6

MultiVitamin 3Multivitamin 3 Release

Due Thu 10/19, 5pm

Week 08

Mon 10/9

Discussion 6Data Models

Tue 10/10

Lecture 14Data Cleaning I

Thu 10/12

Lecture 15Data Cleaning II

MultiVitamin 2 Due, 5pm

Fri 10/13

Project 3Data Transformation

Due Thu 10/26, 5pm

Week 09

Mon 10/16

Discussion 7Data Cleaning

Tue 10/17

Lecture 16Normalization + ER

Thu 10/19

Lecture 17Semistructured Data: NoSQL, JSON, XML

MultiVitamin 3 Due, 5pm

Fri 10/20

MultiVitamin 4Multivitamin 4 Release

Due Thu 11/2, 5pm

Week 10

Mon 10/23

Discussion 8Normalization + ER

Tue 10/24

Lecture 18MongoDB I

Thu 10/26

Lecture 19 [Guest Lecture] Rehan Durrani, Ponder

Project 3 Due

Fri 10/27

Project 4Mongo

Checkpoint due Thu 11/9, 5pm; final due Thu 11/30, 5pm

Week 11

Mon 10/30

Discussion 9MongoDB

Tue 10/31

Lecture 20MongoDB II

Thu 11/2

Lecture 21SQL Modifications, Transactions, and Recovery

MultiVitamin 4 Due, 5pm

Fri 11/3

MultiVitamin 5Multivitamin 5 Release

Due Thu 11/16, 5pm

Week 12

Mon 11/6

Discussion 10Transactions

Tue 11/7

Lecture 22Parallel and Distributed Computing

Thu 11/9

Lecture 23Guest Lecture TBD

Project 4 Checkpoint due, 5pm

Week 13

Mon 11/13

Discussion 11Paralellization

Tue 11/14

Lecture 24OLAP + Visualizations

Thu 11/16

Lecture 25Sampling, Data Vis

MultiVitamin 5 Due, 5pm

Week 14

Mon 11/20

Discussion 12OLAP, Visualizations

Tue 11/21

Lecture 26Guest Lecture TBD

Thu 11/23

Thanksgiving Day (no class)

Week 15

Mon 11/27

Discussion 13Sampling, Data Visualizations

Tue 11/28

Lecture 27Distributed Transactions

Thu 11/30

Lecture 28Closing Thoughts

Project 4 Final due, 5pm

RRR Week

Tue 12/5

RRR Week (no class)

Thu 12/7

RRR Week (no class)

Finals Week

Fri 12/15

Final Exam Final Exam (8-11am)