COMS W4115
Programming Languages and Translators
Lecture 4: Regular Expressions and Lexical Analysis
February 4, 2013

Lecture Outline

  1. Regular expressions
  2. Lex regular expressions
  3. Specifying a lexical analyzer with Lex
  4. Example Lex programs
  5. Creating a lexical processor with Lex
  6. Lex history

1. Regular Expressions

2. Lex Regular Expressions

3. Specifying a Lexical Analyzer with Lex

4. Example Lex programs

Example 1: Lex program to print all words in an input stream

Example 2: Lex program to print number of words, numbers, and lines in a file

Example 3: Lex program for some typical programming language tokens

5. Creating a Lexical Processor with Lex

6. Lex History

7. Practice Problems

  1. Write a Lex program that copies a file, replacing each nonempty sequence of whitespace consisting of blanks, tabs, and newlines by a single blank.
  2. Write a Lex program that converts a file of English text to "Pig Latin." Assume the file is a sequence of words separated by whitespace. If a word begins with a consonant, move the consonant to the end of the word and add "ay". (E.g., "pig" gets mapped into "igpay".) If a word begins with a vowel, just add "ay" to the end. (E.g., "art" gets mapped to "artay".)

8. Reading Assignment



aho@cs.columbia.edu