Chapter 6

• Formal languages
• Mathematical Induction

A language is a set of strings

$$L = \{a, b, aa, ab, ba, bb\}$$

Is the string $ba$ in this language? Is $ba \in L$?

How would you write an algorithm to decide if string of characters, $w$, is in $L$?

A language is a set of strings

$$L = \{a, b, aa, ab, ba, bb\}$$

Is the string $ba$ in this language? Is $ba \in L$?

How would you write an algorithm to decide if string of characters, $w$, is in $L$?

if (w.equals("a"))
    return true;
else if (w.equals("b"))
    return true;
else if (w.equals("aa"))
    return true;
 . . .
else
    return false;

$$L = \{a, b, aa, ab, ba, bb\}$$

Is there another way?

$$L = \{a, b, aa, ab, ba, bb\}$$

Is there another way?

if (w[0] is 'a' or 'b')
    if (w.length() == 1)
       return true;
    if (w[1] is 'a' or 'b') 
        if (w.length() == 2)
	    return true;
} 
return false;

$$L = \{a, b, aa, ab, ba, bb\}$$

Is there another way?

if (w[0] is 'a' or 'b')
    if (w.length() == 1)
       return true;
    if (w[1] is 'a' or 'b') 
        if (w.length() == 2)
	    return true;
} 
return false;

Now, what if

$$L = \{a, b, aa, ab, ba, bb, aaa, aab, aba, abb, baa, bab, bba, bbb\}$$

More nested if's, or…..

$$L = \{a, b, aa, ab, ba, bb, aaa, aab, aba, abb, baa, bab, bba, bbb\}$$

More nested if's, or….. Yes! Recursion!

isInL(w):
    if (w[0] is 'a' or 'b')
        if (w.length() == 1)
            return true;
        if (w[1] is 'a' or 'b') 
            if (w.length() == 2)
              return true;
} 
return false;

isInL(w):
    if (w[0] is 'a' or 'b')
        return isInL(w.substring(1));
} 
return false;

What is missing?

$$L = \{a, b, aa, ab, ba, bb, aaa, aab, aba, abb, baa, bab, bba, bbb\}$$

isInL(w):
    if (w.length() == 0)
        return true;
    if (w[0] is 'a' or 'b')
        return isInL(w.substring(1));
} 
return false;

But only if $\lambda \in L$. $\lambda$ is the empty string.

What else is in $L$ now that we didn't write?

$$L = \{a, b, aa, ab, ba, bb, aaa, aab, aba, abb, baa, bab, bba, bbb\}$$

isInL(w):
    if (w.length() == 0)
        return true;
    if (w[0] is 'a' or 'b')
        return isInL(w.substring(1));
} 
return false;

But only if $\lambda \in L$. $\lambda$ is the empty string.

What else is in $L$ now that we didn't write?

$$L = \{\lambda, a, b, aa, ab, ba, bb, aaa, aab, aba, abb, baa, bab, bba, bbb, \ldots \}$$

$$L = \{\lambda, a, b, aa, ab, ba, bb, aaa, aab, aba, abb, baa, bab, bba, bbb, \ldots \}$$

So, $L$ has infinite number of strings.

Specify with a grammar.

$$<s> = \lambda | a <s> | b <s>$$

Looks very much like the recursive procedure.

Often start with partial enumeration of $L$, then define the grammar, then write the recognition code.

$$L = \{ \lambda, a, b, aa, bb, aaa, aba, bbb, bab \}$$

What would the grammar be?

$$L = \{ \lambda, a, b, aa, bb, aaa, aba, bbb, bab \}$$

What would the grammar be?

$$<pal> = \lambda | a <pal> a | b <pal> b$$

How about all letters?

$$L = \{ \lambda, a, b, aa, bb, aaa, aba, bbb, bab, \ldots \}$$

$$<pal> = \lambda | <ltr> <pal> <ltr> \\ <ltr> = a | b | c | d | \ldots$$

Is $abc \in L$?

$$L = \{ \lambda, a, b, aa, bb, aaa, aba, bbb, bab, \ldots \}$$

$$<pal> = \lambda | <ltr> <pal> <ltr> \\ <ltr> = a | b | c | d | \ldots$$

Would like to do soemthing like

$$<pal> = \lambda | <ltr1> <pal> <ltr2> \\ <ltr1> = a | b | c | d | \ldots\\ <ltr2> = a | b | c | d | \ldots\\ $$ and $$<ltr1> == <ltr2>$$ but not standard grammar notation.

$$L = \{ \lambda, a, b, aa, bb, aba, aaa, bab, bbb, \ldots, zzz \}$$

$$<pal> = \lambda | a <pal> a | b <pal> b | \cdots | z <pal> z \\ $$

Write a recognition algorithm.

$$<pal> = \lambda | a <pal> a | b <pal> b | \cdots | z <pal> z \\ $$

Write a recognition algorithm.

pal(w):
    if (w.length() == 0)
        return true;
    if (w[0] == w[last])
        return pal(w.substring(1,last-1));
    return false;

$$L = \{w | w \mbox{ is of the form } a^n b^n \mbox{ where } n \ge 0 \}$$

What is a grammar for this?

Think about the palindromes.

$$L = \{w | w \mbox{ is of the form } a^n b^n \mbox{ where } n \ge 0 \}$$

What is a grammar for this?

Think about the palindromes. Recurse in the middle.

$$L = \{w | w \mbox{ is of the form } a^n b^n \mbox{ where } n \ge 0 \}$$

What is a grammar for this?

Think about the palindromes. Recurse in the middle.

$$<anbn> = \lambda | a <anbn> b$$

And now the recognition algorithm.

$$L = \{w | w \mbox{ is of the form } a^n b^n \mbox{ where } n \ge 0 \}$$

$$<anbn> = \lambda | a <anbn> b$$

And now the recognition algorithm.

anbn(w):
    if (w.length() == 0)
        return true;
    if (w[0] == 'a' and w[last] == 'b') 
        return anbn(w.substring(1,last-1));
    return false;

What about the language

$$L = \{w | w \mbox{ is of the form } a^{2n} b^n \mbox{ where } n \ge 0 \}$$

What about the language

$$L = \{w | w \mbox{ is of the form } a^{2n} b^n \mbox{ where } n \ge 0 \}$$

$$<a2nbn> = \lambda | aa <a2nbn> b$$

What about the language

$$L = \{w | w \mbox{ is of the form } a^{2n} b^n \mbox{ where } n \ge 0 \}$$

$$<a2nbn> = \lambda | aa <a2nbn> b$$

a2nbn(w):
    if (w.length() == 0)
        return true;
    if (w[0] == 'a' and w[1] == 'a' and w[last] == 'b') 
        return a2nbn(w.substring(2,last-1));
    return false;

What about the language

$$L = \{w | w \mbox{ is of the form } a^{2n} b^n \mbox{ or } a^n b^{2n} \mbox{ where } n \ge 0 \}$$

$$<a2nb2n> = \lambda | aa <a2nb2n> b | a <a2nb2n> bb$$

Check it. Can you think of any strings that this grammar can generate that are not in $L$?

What about the language

$$L = \{w | w \mbox{ is of the form } a^{2n} b^n \mbox{ or } a^n b^{2n} \mbox{ where } n \ge 0 \}$$

Must make choice at beginning of generation. $$<a2nb2n> = \lambda | aa <a2nbn> b | a <anb2n> bb\\ <a2nbn> = \lambda | aa <a2nbn> b\\ <anb2n> = \lambda | a <anb2n> bb\\ $$

What about the language

$$L = \{w | w \mbox{ is of the form } a^{2n} b^n \mbox{ or } a^n b^{2n} \mbox{ where } n \ge 0 \}$$

Must make choice at beginning of generation. $$<a2nb2n> = \lambda | aa <a2nbn> b | a <anb2n> bb\\ <a2nbn> = \lambda | aa <a2nbn> b\\ <anb2n> = \lambda | a <anb2n> bb\\ $$

a2nb2n(w):
    if (w.length() == 0)
        return true;
    if (w[0] == 'a' and w[1] == 'a' and w[last] == 'b') 
        return a2nbn(w.substring(2,last-1));
    if (w[0] == 'a' and w[last-1] == 'b' and w[last] == 'b')
        return anb2n(w.substring(1,last-2));
    return false;

Infix to Postfix:

• Fully parenthesize
• Move operator to corresponding right parenthesis

$$(3 + 2) * 4 - 10 / 2 = (((3 + 2) * 4) - (10 / 2)) = 3 \; 2 + 4 * 10\; 2 / -$$

$$L = \{ w | w \mbox{ is a valid postfix expression} \}$$

Infix to Postfix:

• Fully parenthesize
• Move operator to correponsing right parenthesis

$$(3 + 2) * 4 - 10 / 2 = (((3 + 2) * 4) - (10 / 2)) = 3 \; 2 + 4 * 10\; 2 / -$$

Grammar:

$$<pfe> = <identifier> | <pfe> <pfe> <operator>\\ <identifier> = a | b | \dots | z\\ <operator> = + | - | * | /$$

$$(3 + 2) * 4 - 10 / 2 = (((3 + 2) * 4) - (10 / 2)) = 3 \; 2 + 4 * 10\; 2 / -$$

Grammar:

$$<pfe> = <identifier> | <pfe> <pfe> <operator>\\ <identifier> = a | b | \dots | z\\ <operator> = + | - | * | /$$

Do number 3. Write a grammar for numbers in scientific notation, like 1.5e-4 and -6.21e20.

Do number 3. Write a grammar for numbers in scientific notation, like 1.5e-4 and -6.21e20.

<n> = <s> <d> . <ds> e <s> <ds>

Do number 3. Write a grammar for numbers in scientific notation, like 1.5e-4 and -6.21e20.

$$\begin{align*} <n> &= <s> <d> . <ds> e <s> <ds>\\ <s> &= + \;|\; - \;|\; \lambda\\ <d> &= 0 \;|\; 1 \;|\; \cdots \;|\; 9\\ <ds> &= <d> <ds> \;|\; <d> \end{align*}$$

Remember this? We will use it to prove things about recursive programs.

We want to prove some property $P(n)$ is true for all $n > n_{min}$.

Prove $P(n+1)$ assuming $P(n)$ is true, and assume $P(0)$ is true.

Basis:

Prove $P(0)$ or $P(1)$, or $P(n_{min})$.

Inductive Hypothesis

$P(k)$ is assumed true.

Inductive Step

Prove $P(k) \implies P(k+1)$. If $P(k)$ is true, then $P(k+1)$ is true.

Basis: Prove $P(0)$ or $P(1)$, or $P(n_{min})$.
Inductive Hypothesis: $P(k)$ is assumed true.
Inductive Step: Prove $P(k) \implies P(k+1)$. If $P(k)$ is true, then $P(k+1)$ is true.

Let's try using this to prove the property $P(n)$ that the function $$\mbox{factorial}(n) = \begin{cases} 1, n =0\\ n \mbox{ factorial}(n-1), n > 0 \end{cases}$$ is equal to $$\mbox{factorial}(n) = n (n-1) (n-2) \cdots 1$$

Definition: $$\mbox{factorial}(n) = \begin{cases} 1, n =1\\ n \mbox{ factorial}(n-1), n > 1 \end{cases}$$

Property $P(n)$: $$\mbox{factorial}(n) = n (n-1) (n-2) \cdots 1$$

Basis:
$P(n=1)$: $\mbox{factorial}(1) = 1$
By definition, $\mbox{factorial}(1) = 1$.
Base case proved!

Definition: $$\mbox{factorial}(n) = \begin{cases} 1, n =1\\ n \mbox{ factorial}(n-1), n > 1 \end{cases}$$

Property $P(n)$: $$\mbox{factorial}(n) = n (n-1) (n-2) \cdots 1$$

Inductive Hypothesis:
$P(k)$ is true, or $$\mbox{factorial}(k) = k(k-1)(k-2)\cdots 1$$

Definition: $$\mbox{factorial}(n) = \begin{cases} 1, n =1\\ n \mbox{ factorial}(n-1), n > 1 \end{cases}$$

Property $P(n)$: $$\mbox{factorial}(n) = n (n-1) (n-2) \cdots 1$$

Inductive Step:
Prove $P(k+1)$ using the Inductive Hypothesis that $P(k)$ is true. $$\mbox{factorial}(k+1) = (k+1)\mbox{ factorial}(k) \mbox{, by definition}$$ Using the Inductive Hypothesis, we get $$\mbox{factorial}(k+1) = (k+1) k(k-1)(k-2)\cdots 1$$ which is what we wanted to prove. So we are done.

Here is the Inductive Step again. We started with the function's definition, and used the Inductive Hypothesis for $P(k)$ to prove $P(k+1)$.

Inductive Step:
$$\begin{align*} \mbox{factorial}(k+1) &= (k+1)\mbox{ factorial}(k),\;\;\;\;\; \mbox{ by definition}\\ \mbox{factorial}(k+1) &= (k+1) k(k-1)(k-2)\cdots 1, \;\; \mbox{ by Ind. Hyp.} \end{align*}$$ which is what we wanted to prove.

An alternative is to start with the statement of $P(k+1)$ and use the Inductive Hypothesis to replace part of the equation and end up with the definition of the function.

Inductive Step:
$$\begin{align*} \mbox{factorial}(k+1) &= (k+1) k(k-1)(k-2)\cdots 1, \;\;\mbox{ by } P(k+1)\\ \mbox{factorial}(k+1) &= (k+1) [k(k-1)(k-2)\cdots 1]\\ \mbox{factorial}(k+1) &= (k+1) \mbox{ factorial}(k),\;\;\; \mbox{ by Ind. Hyp.} \end{align*}$$ which is exactly our definition of the factorial function, so it is true.

How many moves does a solution for $n$ disks take?

Remember the solution?

solve(3,A,B,C) =  solve(2,A,C,B), solve(1,A,B,C), solve(2,C,B,A)

So

nMoves(3) = nMoves(2) + nMoves(1) + nMoves(2)
          = 2 nMoves(2) + nMoves(1)

and

nMoves(2) = nMoves(1) + nMoves(1) + nMoves(1)
          = 2 nMoves(1) + nMoves(1)

How many moves does a solution for $n$ disks take?

In general,

nMoves(n) = nMoves(n-1) + nMoves(1) + nMoves(n-1)
          = 2 nMoves(n-1) + nMoves(1)

$$f(n) = \begin{cases} 1, & n = 1\\ 2 f(n-1) + 1, & n > 1 \end{cases}$$

Can we come up with a simpler, nonrecursive way to calculate this?

$$f(n) = \begin{cases} 1, & n = 1\\ 2 f(n-1) + 1, & n > 1 \end{cases}$$

$$f(1) = 1\\ f(2) = 2 f(1) + 1 = 2 + 1\\ f(3) = 2 f(2) +1 = 2(2+1) + 1 = 2(2+2) - 2 + 1 = 2^3 - 1\\ f(4) = 2 f(3) + 1 = 2(2^3 -1) + 1 = 2^4 -1$$

$$f(n) = 2^n - 1$$

Oh yeah? Prove it.

$$f(n) = \begin{cases} 1, & n = 1\\ 2 f(n-1) + 1, & n > 1 \end{cases}$$

$$f(n) = 2^n - 1$$

Basis

Is $f(1) = 2^1 - 1$ true? Yes, because $f(1) = 1$.

Inductive Hypothesis

Assume $f(k) = 2^k - 1$

Inductive Step

Prove $f(k+1) = 2^{k+1} - 1$ assuming $f(k) = 2^k - 1$.

$$f(n) = \begin{cases} 1, & n = 1\\ 2 f(n-1) + 1, & n > 1 \end{cases}$$

$$f(n) = 2^n - 1$$

Inductive Step

Prove $f(k+1) = 2^{k+1} - 1$ assuming $f(k) = 2^k - 1$.

$$\begin{align*} f(k+1) &= 2^{k+1} - 1, \;\;\mbox{What we want to prove}\\ &= 2^k 2^1 - 1\\ &= 2(2^k) -2 +2 - 1\\ &= 2(2^k-1) + 1\\ &= 2f(k) + 1,\;\; \mbox{by the Ind. Hyp.} \end{align*}$$ which is true because this is how we defined $f(n)$.

Another way is to start with the recurrence relation:

Inductive Step

Prove $f(k+1) = 2^{k+1} - 1$ assuming $f(k) = 2^k - 1$.

$$\begin{align*} f(k+1) &= 2f(k) + 1, \;\;\mbox{by definition}\\ &= 2 (2^k - 1) + 1, \;\; \mbox{by the Ind. Hyp.}\\ &= 2^{k+1} -1 \end{align*}$$ which is the property we want to prove.