Spoken Language Processing (CS 4706), Spring 2005

Time:

MW  2:40-3:55

Place

Mudd 253

Professor: 

Julia Hirschberg

Office Hours: 

MW 5-6, CEPSR 705

Email: 

julia [at] cs.columbia.edu

Phone: 

212-939-7114

Teaching Assistant:

Agus Gravano

Office Hours:

MW 4-5, CEPSR 7LW1

Email:

agus [at] cs.columbia.edu

Phone:

212-939-7147

Announcements || Academic Integrity || Description
 
Resources || Requirements || Syllabus || Readings

Description:

This course introduces students to research in spoken language in computational linguistics, aka natural language processing (NLP). We will study the different `meanings' that can be conveyed by the way that speakers produce sentences, techniques for analyzing spoken language, methods of developing speech technologies, and applications of such technologies in the real world, such as text-to-speech systems, speech recognizers, spoken dialogue systems, and detectors for various types of emotional speech.  NB: This course can be counted as a PhD elective in Advanced AI .  It is a requirement for the MS NLP Track.  There are no official prerequisites for this course and no prior knowledge of NLP will be assumed.

Readings:

Required readings:  Acoustic & Auditory Phonetics by Keith Johnson  and selected chapters from  Speech and Language Processing by Jurafsky and Martin.  Both are available from Morningside Bookstore and will also be on reserve in the library.  Other course readings will be available either on the web or in-class handouts. NB: '*' means that the reading is optional/can be chosen for class report.

Course Requirements:

Midterm and final; 4 lab homeworks (can be done on your own pc); 1 10m class presentation on a paper relevant to one of the classes (one of the optional papers, marked with `*’ or one you choose).

Grading:
40% Homeworks
10% Class Presentation
20% Midterm Exam
30% Final Exam

Academic Integrity:

Copying or paraphrasing someone's work (code included), or permitting your own work to be copied or paraphrased, even if only in part, is not allowed, and will result in an automatic grade of 0 for the entire assignment or exam in which the copying or paraphrasing was done. Your grade should reflect your own work. If you believe you are going to have trouble completing an assignment, please talk to the professor in advance of the due date.

Announcements:

Resources:

Praat

Wavesurfer

Help using ToBI - ToBI Annotation Environments

Text-to-Speech Links and more...

Text-to-Song synthesis!

Syllabus:

Week

Date

Topic

Readings and Assignments

Reports and HW

1

W Jan 19

Introduction to the Course

 

 

2

M Jan 24

Understanding Speech Variation

Hirschberg03

 

2

W Jan 26

Studying Speech Variation

Pierrehumbert&Hirschberg '90; Hedberg&Sosa02; Syrdal&Jilka04; *Dohertyetal04

 

3

M Jan 31

Analyzing the Speech Signal:  Symbolic Approaches

J&M 4-4.2

 

3

W Feb 2

Analyzing the Speech Signal:  Signal Processing Approaches

Johnson, Ch. 1-2

Damian Carter

4

M Feb 7

Representing Speech Variation

Wilson93; ToBI labeling conventions (and see ToBI examples); *Baumannetal01

Russell Klopfer
HW1: Using Praat (assigned)

4

W Feb 9

Tools for Speech Analysis

Praat tutorial (Agus Gravano and Stefan Benus)

 

5

M Feb 14

Lab: Learning ToBI Labeling

 (Agus Gravano and Stefan Benus, in the Speech Lab)

 

5

W Feb 16

Speech Generation

HLT96-ch5, TTS systems

HW1 due
HW2: ToBI (assigned)

6

M Feb 21

Modeling Pronunciation

J&M, 4.6; *Fackrell&Skut04

Han Cheng Liang

6

W Feb 23

Predicting Accents and Phrasing

J&M, 4.7
Pan99, *Sun02, Koehn00, Rambow01
(with David Elson on Sable)

 

7

M Feb 28

Information Status: Focus and Given/New

Nakatani99, GBrown83, *Bard99, Prince92, Dahan02

HW2 due

David Vespe

7

W Mar 2

Discourse Structure in TTS Systems

J&M, 18.3, 19.4

 

8

M Mar 7

Speech Recognition and Understanding

J&M, 7; HLT96-ch1

 

8

W Mar 9

Midterm

 

 

10

M Mar 21

Speech Disfluencies

Hindle83;*Nakatani&Hirschberg94;Bear92;

*Liuetal03

HW3a: TTS (assigned)

10

W Mar 23

Sentence, Speaker, and Topic Segmentation

Shriberg00, Choi00, *Utiyama01,

Matthew Waymost

11

M Mar 28

Identifying Speech Acts

J&M, 19.5-6 (new version)
*Jurafsky98, Rosset&Lamel04, Nickerson&Chu-Carroll99

HW3a due
HW3b assigned
Dan J Raz

11

W Mar 30

Spoken Dialogue Systems

J&M, 19.1-3,9 (new version)
Walkeretal97, *Bell&Gustafson00,

James Rishe

12

M Apr 4

Turntaking in Spoken Dialogue

J&M, 19.7 (new version)
Conversational Analysis and Turn-Taking
*Sacksetal74,Beattie82

HW3b due
Sarah Friedman

8

W Apr 6

No Class Today

 

HW3c assigned

12

M Apr 11

Dialogue Errors and Corrections

J&M, 19.4 (new version)
Hirschbergetal04, *Goldberg03, Krahmer01

Michael Bales

13

W Apr 13

Speech Data Mining

*Maskeyetal04, Hearst99

Irina Likhtina

14

M Apr 18

Speech Summarization

*Koumpis&Renals05, Hori02, Zechner01

Daniel Vassilev

14

W Apr 20

Emotional Speech

Cowie00, *Pereira00, Schroeder01, *Bosch00, *Burkhardt00, Ang02

*Gobl&Chasaide03

Lauren Wilcox HW3c due

15

M Apr 25

Deceptive Speech

 DePauloetal83, Frank92, *Mehrabian77, Streeteretal71

Payton Ong
HW4: Perception Test (assigned)

15

W Apr 27

Charismatic Speech

Boss76, Tuppen74, Weber47

Chen-Yung Hsu

16

M May 2

Summing Up

 

HW4 due

16

May 3-5

Study Days

 

 

17

W May 11

Final Exam. 1:10pm - 4:00pm.
Mudd 253 (for those bringing their own laptops) and Speech Lab (for the rest).

  Results of the perception tests - Mapping of synthesizers-CUIDs

 

Announcements || Academic Integrity || Description
Links to Resources || Requirements || Syllabus || Text || Thanks