Data Mining

Instructor : Jeff Phillips (email) | Office hours: Thursday morning 10-11am @ MEB 3442 (and directly after class in WEB L104 --> MEB 3442)
TAs: Harshini Keerthi Vasan (email) | Office hours: Tuesday 2:00 - 4:30pm @ MEB 3409
      + Mattia Grespan (email) | Office Hours: Tuesday 9 - 11am; Thursday 11-12noon @ MEB 3409
      + Sanjana Aravindan (email) | Office Hours: Wednesday 9:30-10:30am @ MEB 3419; Friday 12-2pm @ MEB 3409
      + Bonan Wang (email) | Office Hours: Mondays 8-10am @ MEB 3423
Spring 2019 | Mondays, Wednesdays 3:00 pm - 4:20 pm
WEB L104
Catalog number: CS 5140 01 or CS 6140 01

Data mining is the study of efficiently finding structures and patterns in large data sets. We will focus on several aspects of this: (1) converting from a messy and noisy raw data set to a structured and abstract one, (2) applying scalable and probabilistic algorithms to these well-structured abstract data sets, and (3) formally modeling and understanding the error and other consequences of parts (1) and (2), including choice of data representation and trade-offs between accuracy and scalability. These steps are essential for training as a data scientist.
Algorithms, programming, probability, and linear algebra are required tools for understanding these approaches.
Topics will include: similarity search, clustering, regression/dimensionality reduction, graph analysis, PageRank, and small space summaries. We will also cover several recent developments, and the application of these topics to modern applications, often relating to large internet-based companies.
Upon completion, students should be able to read, understand, and implement ideas from many data mining research papers.

The book for this course will mostly be an in progress book on the Mathematical Foundation for Data Analysis (M4D). However, the lectures will follow more closely my related Data Mining course notes, and in several cases, these have not made it into the above book (yet?).
We will also often link to two other online resources that cover similar material, either with a more applied or theoretical focus:
MMDS(v1.3): Mining Massive Data Sets by Anand Rajaraman, Jure Leskovec, and Jeff Ullman. The digital version of the book is free, but you may wish to purchase a hard copy.
FoDS: Foundations of Data Science by Avrim Blum, John Hopcroft and Ravindran Kannan. This provide some proofs and formalisms not explicitly covered in lecture.

Videos: We plan to videotape all lectures, and make them available online. They will appear on this playlist on our YouTube Channel.
Videos will also livestream here.

Prerequisits: A student who is comfortable with basic probability, basic linear algebra, basic big-O analysis, and basic programming and data structures should be qualified for the class. A great primer on these can be found in the class text Mathematical Foundation for Data Analysis.
There is no specific language we will use. Python is often a good choice, although some parts may be simpler in just Matlab/Octave. However, programming assignments will often (intentionally) not be as specific as in lower-level classes. This will partially simulate real-world settings where one is given a data set and asked to analyze it; in such settings even less direction is provided.
For undergrads, the formal prerequisites are CS 3500 and CS 3130 and MATH 2270 (or equivalent), and CS 4150 is a corequisite. We recommend undergraduates take a new course CS 4964 (Foundations of Data Analysis) before this course, but it is not currently required, and many students have done well without having taken this course. I will grant exceptions to the pre-requisites for students with (a reasonable grade in) Foundations of Data Analysis.
For graduate students, there are no enforced pre-requisites. Still it may be useful to review early material in Mathematical Foundation for Data Analysis.
In the past, this class has had undergraduates, masters, and PhD students, including many from outside of Computer Science. Most (but not all) have kept up fine, and still most have been challenged. If you are unsure if the class is right for you, contact the instructor.

For an example of what sort of mathematical material I expect you to be to be familiar with, see chapters 1 and 3 in Mathematical Foundation for Data Analysis.
Schedule: (subject to change)
Date Topic (+ Notes) Video Link Assignment (latex) Project
Mon 1.07 Class Overview vid
Wed 1.09 Statistics Principles (S) vid M4D 2.2-2.3 | MMDS 1.2 | FoDS 12.4
Mon 1.14 Similarity : Jaccard + k-Grams (S) vid M4D 4.3-4.4 | MMDS 3.1 + 3.2 | FoDS 7.3
Wed 1.16 Similarity : Min Hashing (S) VID M4D 4.6.6 | MMDS 3.3 Statistical Principles
Mon 1.21
Wed 1.23 Similarity : LSH (S) vid M4D 4.6 | MMDS 3.4 Proposal
Mon 1.28 Similarity : Distances (S) vid M4D 4 - 4.3 | MMDS 3.5 + 7.1 | FoDS 8.1
Wed 1.30 Similarity : Word Embed + ANN vs. LSH (S) vid M4D 4.4 | [Ethics Read] | MMDS 3.7 + 7.1.3 Document Hash
Mon 2.04 Clustering : Hierarchical (S) vid M4D 8.5, 8.2 | MMDS 7.2 | FoDS 7.7
Wed 2.06
SNOW DAY -- no class
Mon 2.11 Clustering : K-Means (S) vid M4D 8-8.3 | MMDS 7.3 | FoDS 7.2-3
Wed 2.13 Clustering : Spectral (S) vid M4D 10.3 | MMDS 10.4 | FoDS 7.5 Data Collection Report
Mon 2.18
Wed 2.20 Streaming : Misra-Greis + Count-Min (S) vid FoDS 6.2.3 | MMDS 6+4.3 | BF Analysis Clustering
Mon 2.25 Regression : Basics in 2-dimensions (S) vid M4D 5-5.3 | ESL 3.2 and 3.4
Wed 2.27 Regression : Lasso + OMP + Comp. Sensing (S) vid M4D 5.5 | FoDS 10.2 | Tropp + Gilbert Frequent
Mon 3.04 Regression : Cross-Validation and p-values (S) vid [Ethics Read] | M4D 5.5 | ESL 3.8
Wed 3.06
Mon 3.11
Wed 3.13
Mon 3.18 Dim Reduce : SVD + PCA (S) vid M4D 7 | FoDS 4
Wed 3.20 Dim Reduce : Random Projections (S) vid FoDS 2.9 Intermediate Report
Mon 3.25 Dim Reduce : Matrix Sketching (S) vid MMDS 9.4 | FoDS 2.7 + 7.2.2 | arXiv
Wed 3.27 Dim Reduce : Metric Learning (S) vid M4D 7 | LDA Regression
Mon 4.01 Noise : Noise in Data (S) vid MMDS 9.1 | Tutorial
Wed 4.03 Noise : Privacy (S) vid McSherry | Dwork Dim Reduce
Mon 4.08 Graph Analysis : Markov Chains (S) vid M4D 10.1 | MMDS 10.1 + 5.1 | FoDS 5 | Weckesser
Wed 4.10 Graph Analysis : PageRank (S) vid M4D 10.2 | MMDS 5.1 + 5.4
Mon 4.15 Graph Analysis : MapReduce (S) vid MMDS 2 | Final Report
Wed 4.17 Graph Analysis : Communities (S) vid M4D 10.4 | MMDS 10.2 + 5.5 | FoDS 8.8 + 3.4 Poster Outline
Mon 4.22
Wed 4.24 Graphs
Thu 4.25 Poster Day !!! (3:30-5:30pm) Poster Presentation

This course follows the SoC Guidelines

Latex: I highly highly recommend using LaTex for writing up homeworks. It is something that everyone should know for research and writing scientific documents. This linked directory contains a sample .tex file, as well as what its .pdf compiled outcome looks like. It also has a figure .pdf to show how to include figures.