AM 272 -- Tentative Plan
 
               Updated often -- SUBJECT TO CHANGE
 
 
Lecture Month   Date      Topic(s)
===========================================================================
1       Jan      27	Course information, historical overview, review 
			  of basic concepts; Entropy and the Asymptotic 
			  Equipartition Property (AEP); fixed-rate 
			  lossless compression
2	Feb	  3	Relative entropy and hypothesis testing; the 
			  method of types; definitions and basic types
			  properties; error exponents for data compression
3	Feb	 10	The strong converse for data compression;
			  large deviations: Sanov's theorem and 
			  Crame'r's theorem
--	Feb	 17	NO CLASS
--	Feb 	 20	EXTRA CLASS: Details in Crame'r's theorem;
			  the Gartner-Ellis theorem
4	Feb	 24	The conditional limit theorem; Gibbs' 
		 	  distributions; Pythagorean identity;
			  data processing inequality for relative ent;
			  Pinsker's inequality
5	Mar	  3	Heuristic derivation of the Maxwell-Bolzmann 
			  distribution; equivalence of ensembles 
			  in statistical mechanics; propagation of chaos.
			  Information-theoretic proof of the convergence
			  theorem for Markov Chains.
6	Mar	 10	Hypothesis testing: Neyman-Pearson tests,
			  optimal error exponents, Bayesian hypothesis
			  testing. Further connections between 
			  info theory and statistics.
7	Mar	 17 	Fisher information and estimation, Cramer-Rao
			  inequality, parametric families; review of 
			  the entropy rate and the AEP for Markov chains
--	Mar	 24	NO CLASS
8	Mar	 31	Variable-rate lossless data compression,
			  Kraft inequality, the codes-distributions 
			  correspondence; redundancy, Shannon codes,
			  and their optimality. Elias' codes for the
			  integers
9	Apr	  7	Examples of universal codes. MLE codes, 
			  counting, Minimum Description Length (MDL),
			  mixture codes, predictive coding
10	Apr	 14	MDL and model selection;
			  entropy and Poisson approximation;
			  some continuous information theory
11	Apr	 21	The redundancy of IID sources:
			  Shtarkov's upper bound and Rissanen's
			  lower bound
12	Apr	 28	Practical data compression: Arithmetic coding,
			  Lempel-Ziv coding
13	May	 5	GUEST LECTURE: Matt Harrison & Mokshay Madiman:
			  universal *lossy* data compression, the
			  lossy version of maximum likelihood, a lossy
			  version of the MDL principle

IN-CLASS PRESENTATION SCHEDULE

------------------------------------------------------------

** DAY 1: Probability and IT
Mon 5/12, 2-3:30 in Room 110 - 182 George Street

   Tom Dean		Measure concentration and the entropy method
   Mario Micheli	Information-theoretic Markov chain convergence
   Yanchun Wu		Entropy and the Central Limit Theorem

------------------------------------------------------------

** DAY 2: Applications of IT
Wed 5/14, 2-3:30 in Room 104 - 37 Manning Street

  Ye Sun		Information theory and investment 
  Nikos Triandopoulos	Average vs worst case complexity 
  Jigang Wang		Pattern classification 

------------------------------------------------------------

** DAY 3: Algorithms for compression and entropy estimation
Thu 5/15, 2-3:30 in Room 110 - 182 George Street

  Ying Xing		Web navigation and Markov entropy estimation
  Tom Pollard		Lossless image compression with LOCO I
  Ioannis Vergados 	Lempel-Ziv-style entropy estimation

------------------------------------------------------------