2013-02-08
- A More Methodical Explanation of Recursion
- The Y Combinator
- Typing the Y Combinator

========================================================================
>>> A More Methodical Explanation of Recursion

[Note: This explanation is similar to the one you can find in "The
Little Schemer", by Dan Friedman and Matthias Felleisen.]

The explanation that we have now for how to derive the `make-recursive'
definition is fine -- after all, we did manage to get it working.  But
this explanation was done from a kind of an operation point of view: we
knew a certain trick that can make things work and we pushed things
around until we got it working like we wanted.  Instead of doing this,
we can re-approach the problem from a more declarative point of view.

So, start again from the same broken code that we had (using the
broken-scope language):

  (define fact
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))

This is as broken as it was when we started: the occurrence of `fact' in
the body of the function is free, which means that this code is
meaningless.  To avoid the compilation error that we get when we run
this code, we can substitute *anything* for that `fact' -- it's even
better to use a replacement that will lead to a runtime error:

  (define fact
    (lambda (n) (if (zero? n) 1 (* n (777 (- n 1)))))) ;***

This function will not work in a similar way to the original one -- but
there is one case where it *does* work: when the input value is 0 (since
then we do not reach the bogus application).  We note this by calling
this function `fact0':

  (define fact0                ;***
    (lambda (n) (if (zero? n) 1 (* n (777 (- n 1))))))

Now that we have this function defined, we can use it to write `fact1'
which is the factorial function for arguments of 0 or 1:

  (define fact0
    (lambda (n) (if (zero? n) 1 (* n (777 (- n 1))))))
  (define fact1
    (lambda (n) (if (zero? n) 1 (* n (fact0 (- n 1))))))

And remember that this is actually just shorthand for:

  (define fact1
    (lambda (n)
      (if (zero? n)
        1
        (* n ((lambda (n)
                (if (zero? n)
                  1
                  (* n (777 (- n 1)))))
              (- n 1))))))

We can continue in this way and write `fact2' that will work for n<=2:

  (define fact2
    (lambda (n) (if (zero? n) 1 (* n (fact1 (- n 1))))))

or, in full form:

  (define fact2
    (lambda (n)
      (if (zero? n)
        1
        (* n ((lambda (n)
                (if (zero? n)
                  1
                  (* n ((lambda (n)
                          (if (zero? n)
                            1
                            (* n (777 (- n 1)))))
                        (- n 1)))))
              (- n 1))))))

If we continue this way, we *will* get the true factorial function, but
the problem is that to handle *any* possible integer argument, it will
have to be an infinite definition!  Here is what it is supposed to look
like:

  (define fact0 (lambda (n) (if (zero? n) 1 (* n (777 (- n 1))))))
  (define fact1 (lambda (n) (if (zero? n) 1 (* n (fact0 (- n 1))))))
  (define fact2 (lambda (n) (if (zero? n) 1 (* n (fact1 (- n 1))))))
  (define fact3 (lambda (n) (if (zero? n) 1 (* n (fact2 (- n 1))))))
  ...

The true factorial function is `fact-infinity', with an infinite size.
So, we're back at the original problem...

To help make things more concise, we can observe the repeated pattern in
the above, and extract a function that abstracts this pattern.  This
function is the same as the `fact-core' that we have seen previously:

  (define fact-core
    (lambda (fact)
      (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
  (define fact0 (fact-core 777))
  (define fact1 (fact-core fact0))
  (define fact2 (fact-core fact1))
  (define fact3 (fact-core fact2))
  ...

which is actually:

  (define fact-core
    (lambda (fact)
      (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
  (define fact0 (fact-core 777))
  (define fact1 (fact-core (fact-core 777)))
  (define fact2 (fact-core (fact-core (fact-core 777))))
  ...
  (define fact
    (fact-core (fact-core (fact-core (... (fact-core 777) ...)))))

Do this a little differently -- rewrite `fact0' as:

  (define fact0
    ((lambda (mk) (mk 777))
     fact-core))

Similarly, `fact1' is written as:

  (define fact1
    ((lambda (mk) (mk (mk 777)))
     fact-core))

and so on, until the real factorial, which is still infinite at this
stage:

  (define fact
    ((lambda (mk) (mk (mk (... (mk 777) ...))))
     fact-core))

Now, look at that (lambda (mk) ...) -- it is an infinite expression, but
for every actual application of the resulting factorial function we only
need a finite number of `mk' applications.  We can guess how many, and
as soon as we hit an application of 777 we know that our guess is too
small.  So instead of 777, we can try to use the maker function to
create and use the next.

To make things more explicit, here is the expression that is our
`fact0', without the definition form:

  ((lambda (mk) (mk 777))
   fact-core)

This function has a very low guess -- it works for 0, but with 1 it will
run into the 777 application.  At this point, we want to somehow invoke
`mk' again to get the next level -- and since 777 *does* get applied, we
can just replace it with `mk':

  ((lambda (mk) (mk mk))
   fact-core)

The resulting function works just the same for an input of 0 because it
does not attempt a recursive call -- but if we give it 1, then instead
of running into the error of applying 777:

  (* n (777 (- n 1)))

we get to apply `fact-core' there:

  (* n (fact-core (- n 1)))

and this is still wrong, because `fact-core' expects a function as an
input.  To see what happens more clearly, write `fact-core' explicitly:

  ((lambda (mk) (mk mk))
   (lambda (fact)
     (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))

The problem is in what we're going to pass into `fact-core' -- its
`fact' argument will not be the factorial function, but the `mk'
function constructor.  Renaming the `fact' argument as `mk' will make
this more obvious (but not change the meaning):

  ((lambda (mk) (mk mk))
   (lambda (mk)
     (lambda (n) (if (zero? n) 1 (* n (mk (- n 1)))))))

It should now be obvious that this application of `mk' will not work,
instead, we need to apply it on some function and *then* apply the
result on n-1.  To get what we had before, we can use 777 as a bogus
function:

  ((lambda (mk) (mk mk))
   (lambda (mk)
     (lambda (n) (if (zero? n) 1 (* n ((mk 777) (- n 1)))))))

This will allow one recursive call -- so the definition works for both
inputs of 0 and 1 -- but not more.  But that 777 is used as a maker
function now, so instead, we can just use `mk' itself again:

  ((lambda (mk) (mk mk))
   (lambda (mk)
     (lambda (n) (if (zero? n) 1 (* n ((mk mk) (- n 1)))))))

And this is a *working* version of the real factorial function, so make
it into a (non-magical) definition:

  (define fact
    ((lambda (mk) (mk mk))
     (lambda (mk)
       (lambda (n) (if (zero? n) 1 (* n ((mk mk) (- n 1))))))))

But we're not done -- we "broke" into the factorial code to insert that
(mk mk) application -- that's why we dragged in the actual value of
`fact-core'.  We now need to fix this.  The expression on that last line

  (lambda (n) (if (zero? n) 1 (* n ((mk mk) (- n 1)))))

is close enough -- it is (fact-core (mk mk)).  So we can now try to
rewrite our `fact' as:

  (define fact-core
    (lambda (fact)
      (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
  (define fact
    ((lambda (mk) (mk mk))
     (lambda (mk) (fact-core (mk mk)))))

... and would fail in a familiar way!  If it's not familiar enough, just
rename all those `mk's as `x's:

  (define fact-core
    (lambda (fact)
      (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
  (define fact
    ((lambda (x) (x x))
     (lambda (x) (fact-core (x x)))))

We've run into the eagerness of our language again, as we did before.
The solution is the same -- the (x x) is the factorial function, so
protect it as we did before, and we have a working version:

  (define fact-core
    (lambda (fact)
      (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
  (define fact
    ((lambda (x) (x x))
     (lambda (x) (fact-core (lambda (n) ((x x) n))))))

The rest should not be surprising now...  Abstract the recursive making
bit in a new `make-recursive' function:

  (define fact-core
    (lambda (fact)
      (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
  (define (make-recursive f)
    ((lambda (x) (x x))
     (lambda (x) (f (lambda (n) ((x x) n))))))
  (define fact (make-recursive fact-core))

and now we can do the first reduction inside `make-recursive' and write
the `fact-core' expression explicitly:

  #lang pl broken
  (define (make-recursive f)
    ((lambda (x) (f (lambda (n) ((x x) n))))
     (lambda (x) (f (lambda (n) ((x x) n))))))
  (define fact
    (make-recursive
     (lambda (fact)
       (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))))

and this is the same code we had before.

========================================================================
>>> The Y Combinator

Our `make-recursive' function is usually called the "fixpoint operator"
or the "Y combinator".

It looks really simple when using the lazy version (remember: our
version is the eager one):

  (define Y
    (lambda (f)
      ((lambda (x) (f (x x)))
       (lambda (x) (f (x x))))))

(Note that if we *do* allow a recursive definition for Y itself, then
the definition can follow the definition that we've seen:

  (define (Y f) (f (Y f)))

)

And this all comes from the loop generated by:

  ((lambda (x) (x x)) (lambda (x) (x x)))

((lambda (x) (x x)) (lambda (x) (x x))), which is also called `Omega'
(usually the (lambda (x) (x x)) part by itself is called `omega' and
then (omega omega) is `Omega'), is also the idea behind many deep
mathematical facts.  As an example for what it does, follow the next
rule:

  I will say the next sentence twice:
    "I will say the next sentence twice".

(Note the usage of colon for the first and quotes for the second -- what
is the equivalent of that in the lambda expression?)

By itself, this just gets you stuck in an infinite loop, as Omega does,
and the Y combinator adds `F' to that to get an infinite chain of
applications -- which is similar to:

  I will say the next sentence twice:
    "I will hop on one foot and then say the next sentence twice".

========================================================================
>> The main property of Y

`fact-core' is a function that given any limited factorial, will
generate a factorial that is good for one more integer input.  Start
with `777', which is a factorial that is good for nothing (because it's
not a function), and you can get `fact0' as

  fact0 == (fact-core 777)

and that's a good factorial function only for an input of 0.  Use that
with `fact-core' again, and you get

  fact1 == (fact-core fact0) == (fact-core (fact-core 777))

which is the factorial function when you only look at input values of 0
or 1.  In a similar way

  fact2 == (fact-core fact1)

is good for 0..2 -- and we can continue as much as we want, except that
we need to have an infinite number of applications -- in the general
case, we have:

  fact-n == (fact-core (fact-core (fact-core ... 777)))

which is good for 0..n.  The *real* factorial would be the result of
running `fact-core' on itself infinitely, it *is* `fact-infinity'.  In
other words (here `fact' is the *real* factorial):

  fact = fact-infinity == (fact-core (fact-core ...infinitely...))

but note that since this is really infinity, then

  fact = (fact-core (fact-core ...infinitely...))
       = (fact-core fact)

so we get an equation:

  fact = (fact-core fact)

and a solution for this is going to be the real factorial.  The solution
is the "fixed-point" of the `fact-core' function, in the same sense that
0 is the fixed point of the `sin' function because

  0 = (sin 0)

And the Y combinator does just that -- it has this property:

  (make-recursive f) = (f (make-recursive f))

or, using the more common name:

  (Y f) = (f (Y f))

This property encapsulates the real magical power of Y.  You can see how
it works -- because:       (Y f)  =    (f (Y f))
you can also say that:  (f (Y f)) = (f (f (Y f)))
so we get:

  (Y f) = (f (Y f)) = (f (f (Y f))) = (f (f (f (Y f)))) = ...
        = (f (f (f ...)))

and we can conclude that

  (Y fact-core) = (fact-core (fact-core ...infinitely...))
                = fact

========================================================================
>>> Typing the Y Combinator

Typing the Y combinator is always a tricky issue.  For example, in
standard ML you must write a new type definition to do this:

  datatype 'a t = T of 'a t -> 'a
  val y = fn f => (fn (T x) => (f (fn a => x (T x) a)))
                    (T (fn (T x) => (f (fn a => x (T x) a))))

[

Can you find a pattern in the places where T is used?
-- Roughly speaking, that type definition is

  ;; `t' is the type name, `T' is the constructor (aka the variant)
  (define-type (t a)  ; we don't really have polymorphic types
    [T ((t a) -> a)])

First note that the two `fn a => ...' parts are the same as our
protection, so ignoring that we get:

  val y = fn f => (fn (T x) => (f (x (T x))))
                    (T (fn (T x) => (f (x (T x)))))

if you now replace `T' with `Quote', things make more sense:

  val y = fn f => (fn (Quote x) => (f (x (Quote x))))
                    (Quote (fn (Quote x) => (f (x (Quote x)))))

and with our syntax, this would be:

  (define (Y f)
    ((lambda (qx)
       (cases qx
         [(Quote x) (f (x (Quote x)))]))
     (Quote
      (lambda (qx)
        (cases qx
          [(Quote x) (f (x (Quote x)))])))))

it's not really quotation -- but the analogy should help: it uses
`Quote' to distinguish functions as values that are applied (the `x's)
from functions that are passed as arguments.

]

In OCaml, this looks a little different:

  # type 'a t = T of ('a t -> 'a) ;;
  type 'a t = T of ('a t -> 'a)
  # let y f = (fun (T x) -> x (T x))
              (T (fun (T x) -> fun z -> f (x (T x)) z)) ;;
  val y : (('a -> 'b) -> 'a -> 'b) -> 'a -> 'b = <fun>
  # let fact = y (fun fact n -> if n<1 then 1 else n* fact(n-1)) ;;
  val fact : int -> int = <fun>
  # fact 5 ;;
  - : int = 120

but OCaml also lets you start with a `-rectypes' command line argument,
which will make it infer a type itself:

  # let y f = (fun x -> x x) (fun x -> fun z -> f (x x) z) ;;
  val y : (('a -> 'b) -> 'a -> 'b) -> 'a -> 'b = <fun>
  # let fact = y (fun fact n -> if n<1 then 1 else n* fact(n-1)) ;;
  val fact : int -> int = <fun>
  # fact 5 ;;
  - : int = 120

It is also possible to write this expression in Typed Racket, but we
will need to write a proper type definition.  First of all, the type of
Y should be straightforward: it is a fixpoint operation, so it takes a
`T -> T' function and produces its fixpoint.  The fixpoint itself is
some `T' (such that applying the function on it results in itself).  So
this gives us:

  (: make-recursive : (T -> T) -> T)

However, in our case `make-recursive' computes a *functional* fixpoint,
for unary `S -> T' functions, so we should narrow down the type

  (: make-recursive : ((S -> T) -> (S -> T)) -> (S -> T))

Now, in the body of `make-recursive' we need to add a type for the `x'
argument which is behaving in a weird way: it is used both as a function
and as its own argument.  (Remember -- I will say the next sentence
twice: "I will say the next sentence twice".)  We need a recursive type
definition for that:

  (define-type (Tau S T) = (Rec this (this -> (S -> T))))

This type is tailored for our use of `x': given a type `T', `x' is a
function that will consume *itself* (hence the `Rec') and spit out the
value that the `f' argument consumes -- an `S -> T' function.

The resulting full version of the code:

  (: make-recursive : (All (S T) ((S -> T) -> (S -> T)) -> (S -> T)))
  (define-type (Tau S T) = (Rec this (this -> (S -> T))))
  (define (make-recursive f)
    ((lambda: ([x : (Tau S T)]) (f (lambda (z) ((x x) z))))
     (lambda: ([x : (Tau S T)]) (f (lambda (z) ((x x) z))))))

  (: fact : Number -> Number)
  (define fact (make-recursive
                (lambda: ([fact : (Number -> Number)])
                  (lambda: ([n : Number])
                    (if (zero? n)
                      1
                      (* n (fact (- n 1))))))))

  (fact 5)

========================================================================