Recent Advances in Computing: Recent Advances in Natural Language Processing

Lexical Semantics

CS 7301 Section 002

Meets every Tuesday and Thursday 11:00–12:15 in room 2.415

Instructor: Vasileios Hatzivassiloglou

Contact: vh (at) hlt.udtallas.edu

Office hours: Tuesday 12:30–1:30 and by appointment

Course Announcement

Draft list of paper presentations

Lecture Group 1: Introduction to Lexical Semantics (Lectures of 1/11/05, 1/13/05, 1/18/05, and 1/20/05)

• Introduction to the topic and course
• Constraints on word meaning
• Polysemy
• Metaphor and metonymy
• Representing word meaning
• Learning word meaning
• Context and disambiguation
• Non-compositional preferences
• Semantic similarity
• Lexical Properties: Orientation, Semantic Strength
• Text mining
• Terminology
• Applications in Bioinformatics

Lecture Group 2: Overview of Statistical Methods in NLP (Lectures of 1/25/05, 1/27/05, and 2/1/05)

• Parameterized Models
• Maximum Likelihood Estimation
• Smoothing
• Bayesian Learning
• Markov Models
• Estimation-Maximization
• Linear and Log-linear Regression
• Singular Value Decomposition
• Classification: Decision Trees, Rule Induction, and Support Vector Machines

Background lecture on Word Association Metrics: Statistics and Information Theory (2/3/05)

• Slides
• Background Reading: Thomas M. Cover and Joy A. Thomas, “Entropy, Relative Entropy, and Mutual Information”, Chapter 2 of Elements of Information Theory, John Wiley & Sons, 1991.

Paper Session 1: Word Association (2/8/05)

• Slides
• Research Paper: Frank Smadja, Kathleen R. McKeown, and Vasileios Hatzivassiloglou, “Translating Collocations for Bilingual Lexicons: A Statistical Approach”, Sections 1–4, Computational Linguistics, 22(1):1–38, 1996.
• Research Paper: Ted Dunning, “Accurate Methods for the Statistics of Surprise and Coincidence”, Computational Linguistics, 19(1):61–74, 1993.

Paper Session 2: Collocations and Terminology (2/10/05)

• Slides
• Research Paper: Frank Smadja, “Retrieving Collocations from Text: Xtract”, Computational Linguistics 19(1):143–177, 1993.
• Research Paper: Béatrice Daille, “Study and Implementation of Combined Techniques for Automatic Extraction of Terminology”, Proceedings of the ACL Workshop The Balancing Act: Combining Symbolic and Statistical Approaches to Language, pages 29–36, 1994.
• Brief discussion of the paper J. Justeson and S. M. Katz, “Technical Terminology: Some Linguistic Properties and an Algorithm for Identification in Text”, Natural Language Engineering 1(1):9–27, 1995 (this paper is not available online).
• Research Paper: Christian Jacquemin, Judith L. Klavans and Evelyne Tzoukermann, “Expansion of Multi-Word Terms for Indexing and Retrieval Using Morphology and Syntax”, Proceedings of the Thirty-Fifth Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics, pages 24–31, Madrid, Spain.

Paper Session 3: Word Similarity and Clustering — Distributional Methods (2/15/05)

• Presented by Marian Olteanu
• Slides: Part 1, Part 2, and Part 3
• Research Paper: Hinrich Schütze, “Dimensions of Meaning”, Proceedings of Supercomputing, 1992.
• Research Paper: Fernando Pereira, Naftali Tishby, and Lillian Lee, “Distributional Clustering of English Words”, Proceedings of the 31st Annual Meeting of the ACL, pages 183–191, Columbus, Ohio, 1993.
• Research Paper: Vasileios Hatzivassiloglou and Kathleen R. McKeown, “Towards the Automatic Identification of Adjectival Scales: Clustering Adjectives According to Meaning”, Proceedings of the 31st Annual Meeting of the ACL, pages 172–182, Columbus, Ohio, 1993.

No lecture on 2/17/05

Paper Session 4: Word Similarity and Clustering — Information Retrieval and Comparisons (2/22/05)

• Presented by Cosmin Adrian Bejan
• Slides: Part 1, Part 2, and Part 3
• Research Paper: Alan F. Smeaton and Ian Quigley, “Experiments on Using Semantic Distances Between Words in Image Caption Retrieval”, Proceedings of the 19th International Conference on Research and Development in Information Retrieval (SIGIR), Zurich, 1996.
• Research Paper: Lillian Lee, “Measures of Distributional Similarity”, Proceedings of the 37th Annual Meeting of the ACL, pages 25–32, College Park, Maryland, 1999
• Research Paper: Egidio Terra and C. L. A. Clarke, “Frequency Estimates for Statistical Word Similarity Measures”, Proceedings of HLT-NAACL 2003, Edmonton, Canada, 2003.

Paper Session 5: Scalar Properties of Words (2/24/05)

• Presented by Mithun Balakrishna
• Slides
• Research Paper: Michael Elhadad, “Generating Adjectives to Express the Speaker’s Argumentative Intent”, Proceedings of the 9th National Conference on Artificial Intelligence (AAAI), Anaheim, California, 1991.
• Research Paper: Viktor Raskin and Sergei Nirenburg, “Adjectival Modification in Text Meaning Representation”, Proceedings of the 16th Conference on Computational Linguistics (COLING), pages 842–847, Copenhagen, Denmark, 1996.
• Research Paper: Daniel Marcu and Graeme Hirst, “A Uniform Treatment of Pragmatic Inferences in Simple and Complex Utterances and Sequences of Utterances”, Proceedings of the 33rd Annual Meeting of the ACL, pages 144–150, Cambridge, Massachusetts, 1995.

Paper Session 6: Scalar Implicature (3/1/05)

• Presented by Ovidiu Christenel Fortu
• Slides
• Research Paper: Robyn Carston, “Informativeness, Relevance, and Scalar Implicature”, in Robyn Carston and Seiji Uchida (editors), Relevance Theory: Applications and Implications, John Benjamins, Amsterdam, 1998.
• Research Paper: Ira A. Noveck, “When Children Are More Logical Than Adults: Experimental Investigations of Scalar Implicature”, Cognition 78:165–188, 2001.
• Research Paper: Anna Papafragou and Julien Mussolino, “Scalar Implicatures: Experiments at the Semantics-Pragmatics Interface”, Cognition 86:253–282, 2003.

Paper Session 7: Semantic Orientation and Subjectivity (3/3/05)

• Presented by Gabriel Nicolae
• Slides: Part 1, Part 2, and Part 3
• Research Paper: Vasileios Hatzivassiloglou and Kathleen R. McKeown, “Predicting the Semantic Orientation of Adjectives”, Proceedings of the 35th Annual Meeting of the ACL and 8th Conference of the European Chapter of the ACL, pages 174–181, Madrid, Spain, July 1997.
• Research Paper: Janyce M. Wiebe, “Learning Subjective Adjectives from Corpora”, Proceedings of the 17th National Conference on Artificial Intelligence (AAAI), Austin, Texas, 2000.
• Research Paper: Ellen Riloff, Janyce Wiebe, and Theresa Wilson, “Learning Subjective Nouns using Extraction Pattern Bootstrapping”, Proceedings of the 7th Conference on Computational Natural Language Learning (CoNLL), pages 25–32, Edmonton, Canada, 2003.

Paper Session 8: Document-level Semantic Orientation and Argumentation (3/15/05)

• Presented by Marta Tatu
• Slides
• Research Paper: Peter D. Turney, “Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews”, Proceedings of the 40th Annual Meeting of the ACL, pages 417–424, Philadelphia, Pennsylvania, July 2002.
• Research Paper: Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan, “Thumbs Up? Sentiment Classification using Machine Learning Techniques”, Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 79–86, Philadelphia, Pennsylvania, July 2002.
• Research Paper: Simone Teufel and Marc Moens, “Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status”, Computational Linguistics, 28(4):409–445, 2002.

Paper Session 9: Ontologies and Lexical Databases (3/17/05)

• Presented by Cristina Nicolae
• Slides: Part 1, Part 2, Part 3, and Part 4
• Research Summary Paper: George A. Miller, “WordNet: A Lexical Database for English”, Communications of the ACM, 38(11):39–41, November 1995.
• Research Summary Paper: Douglas B. Lenat, “Cyc: A Large-Scale Investment in Knowledge Infrastructure”, Communications of the ACM, 38(11):33–38, November 1995.
• For a comment by Vaughan Pratt on Cyc’s abilities circa 1994, see http://boole.stanford.edu/cyc.html.
• Research Paper: Kevin Knight and Steve K. Luk, “Building a Large-Scale Knowledge Base for Machine Translation”, Proceedings of the 12th National Conference on Artificial Intelligence (AAAI), Seattle, 1994.
• Research Paper: Dieter Fensel, Frank van Harmelen, Ian Horrocks, Deborah L. McGuinness, and Peter F. Patel-Schneider, “OIL: An Ontology Infrastructure for the Semantic Web”, IEEE Intelligent Systems, pages 38–45, March/April 2001.

Paper Session 10: Ontology Construction and Distance Measurement (3/22/05)

• Presented by Cosmin Adrian Bejan and Ovidiu Christenel Fortu
• Slides: Part 1, Part 2, and Part 3
• Research Paper: Gilles Bisson, Claire Nédellec, and Dolores Cañamero, “Designing Clustering Methods for Ontology Building: The Mo’k Workbench”, Proceedings of the First Workshop on Ontology Learning (Workshop at the 14th European Conference on Artificial Intelligence (ECAI)), pages 13–18, 2000.
• Research Paper: Roberto Navigli, Paola Velardi, and Aldo Gangemi, “Ontology Learning and its Application to Automated Terminology Translation”, IEEE Intelligent Systems, pages 22–31, January/February 2003.
• Research Paper: Philip Resnik, “Using Information Content to Evaluate Semantic Similarity in a Taxonomy”, Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI), Montréal, Canada, August 1995.
• Research Paper: Alexander Budanitsky and Graeme Hirst, “Semantic Distance in WordNet: An Experimental, Application-Oriented Evaluation of Five Measures”, Proceedings of the Workshop on WordNet and Other Lexical Resources at the Second Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL), Pittsburgh, June 2001.

Paper Session 11: Word and Phrase Alignment (3/24/05)

• Presented by Marta Tatu and Mithun Balakrishna
• Slides
• Research Paper: Frank Smadja, Kathleen R. McKeown, and Vasileios Hatzivassiloglou, “Translating Collocations for Bilingual Lexicons: A Statistical Approach”, Sections 2 and 5–7, Computational Linguistics, 22(1):1–38, 1996. (Only the alignment method and results).
• Research Paper: Pascale Fung, “A Pattern Matching Method for Finding Noun and Proper Noun Translations from Noisy Parallel Corpora”, Proceedings of the 33rd Annual Meeting of the ACL, pages 236–243, Cambridge, Massachusetts, June 1995.
• Research Paper: Ralf D. Brown, “Automated Dictionary Extraction for “Knowledge-Free” Example-Based Translation”, Proceedings of the Seventh International Conference on Theoretical and Methodological Issues in Machine Translation, pages 111–118, Santa Fe, July 1997.
• Research Paper: Regina Barzilay and Kathleen R. McKeown, “Extracting Paraphrases from a Parallel Corpus”, Proceedings of the 39th Annual Meeting of the ACL, Toulouse, France, 2001.

Paper Session 12: Word Sense Disambiguation (3/29/05)

• Presented by Marian Olteanu
• Slides: Part 1, Part 2, and Part 3
• Research Paper: Yorick Wilks and Mark Stevenson, “Word Sense Disambiguation using Optimised Combinations of Knowledge Sources”, Proceedings of the 17th International Conference on Computational Linguistics and the 36th Annual Meeting of the Association for Computational Linguistics (COLING-ACL), pages 1398–1402, Montréal, Canada, 1998.
• Research Paper: Rada Mihalcea and Dan I. Moldovan, “A Method for Word Sense Disambiguation of Unrestricted Text”, Proceedings of the 37th Annual Meeting of the ACL, pages 152–158, College Park, Maryland, June 1999.
• Research Paper: Radu Florian and David Yarowsky, “Modeling Consensus: Classifier Combination for Word Sense Disambiguation”, Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 25–32, Philadelphia, July 2002.

Lectures on Natural Language Processing in Bioinformatics and Medical Informatics (3/31/05 and 4/5/05)

• Slides

Paper Session 13: Term Disambiguation and Relationship Mining in Bioinformatics (4/7/05)

• Presented by Cristina Nicolae and Gabriel Nicolae
• Slides
• Research Paper: Vasileios Hatzivassiloglou, Pablo A. Duboué, and Andrey Rzhetsky, “Disambiguating Proteins, Genes, and RNA in Text: A Machine Learning Approach”, Bioinformatics 17(S1):97–106, 2001.
• Research Paper: Christian Blaschke, Migual A. Andrade, Christos Ouzounis, and Alfonso Valencia, “Automatic Extraction of Biological Information from Scientific Text: Protein–Protein Interactions”, Proceedings of the 7th International Conference on Intelligent Systems for Molecular Biology (ISMB), pages 60–67, 1999.
• Research Paper: Soumya Ray and Mark Craven, “Representing Sentence Structure in Hidden Markov Models for Information Extraction”, Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI), Seattle, 2001.