2010-02-16 - The Core of `make-recursive' - A More Methodical Explanation of Recursion - The Y Combinator - Typing the Y Combinator - Lambda Calculus -- Schlac - Church Numerals ======================================================================== >>> The Core of `make-recursive' As in Scheme, being able to express recursive functions is a fundamental property of the language. It means that we can have loops in our language, and that's the essence of making a language powerful enough to be TM-equivalent -- able to express undecisable problems, where we don't know whether there is an answer or not. The core of what makes this possible is the expression that we have seen in our derivation: ((lambda (x) (x x)) (lambda (x) (x x))) which reduces to itself, and therefore has no value: trying to evaluate it gets stuck in an infinite loop. (This expression is often called "Omega".) This is the key for creating a loop -- we use it to make recursion possible. Looking at our final `make-recursive' definition and ignoring for a moment the "protection" that we need against being stuck prematurely in an infinite loop: (define (make-recursive f) ((lambda (x) (x x)) (lambda (x) (f (x x))))) we can see that this is almost the same as the Omega expression -- the only difference is that application of `f'. Indeed, this expression (the result of (make-recursive F) for some `F') reduces in a similar way to Omega: ((lambda (x) (x x)) (lambda (x) (F (x x)))) ((lambda (x) (F (x x))) (lambda (x) (F (x x)))) (F ((lambda (x) (F (x x))) (lambda (x) (F (x x))))) (F (F ((lambda (x) (F (x x))) (lambda (x) (F (x x)))))) (F (F (F ((lambda (x) (F (x x))) (lambda (x) (F (x x))))))) ... which means that the actual value of this expression is: (F (F (F ...forever...))) This definition would be sufficient if we had a lazy language, but to get things working in a strict one we need to bring back the protection. This makes things a little different -- if we use (protect f) to be a shorthand for the protection trick, (rewrite (protect f) => (lambda (x) (f x))) then we have: (define (make-recursive f) ((lambda (x) (x x)) (lambda (x) (f (protect (x x)))))) which makes the (make-recursive F) evaluation reduce to (F (protect (F (protect (F (protect (...forever...))))))) and this is still the same result (as long as `F' is a single-argument function). ======================================================================== >>> A More Methodical Explanation of Recursion [Note: This explanation is similar to the one you can find in "The Little Schemer", by Dan Friedman and Matthias Felleisen.] The explanation that we have now for how to derive the `make-recursive' definition is fine -- after all, we did manage to get it working. But this explanation was done from a kind of an operation point of view: we knew a certain trick that can make things work and we pushed things around until we got it working like we wanted. Instead of doing this, we can re-approach the problem from a more declarative point of view. So, start again from the same broken code that we had (using the broken-scope language): (define fact (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))) This is as broken as it was when we started: the occurrence of `fact' in the body of the function is free, which means that this code is meaningless. To avoid the compilation error that we get when we run this code, we can substitute *anything* for that `fact' -- it's even better to use a replacement that will lead to a runtime error: (define fact (lambda (n) (if (zero? n) 1 (* n (777 (- n 1)))))) ;*** This function will not work in a similar way to the original one -- but there is one case where it *does* work: when the input value is 0 (since then we do not reach the bogus application). We note this by calling this function `fact0': (define fact0 ;*** (lambda (n) (if (zero? n) 1 (* n (777 (- n 1)))))) Now that we have this function defined, we can use it to write `fact1' which is the factorial function for arguments of 0 or 1: (define fact0 (lambda (n) (if (zero? n) 1 (* n (777 (- n 1)))))) (define fact1 (lambda (n) (if (zero? n) 1 (* n (fact0 (- n 1)))))) And remember that this is actually just shorthand for: (define fact1 (lambda (n) (if (zero? n) 1 (* n ((lambda (n) (if (zero? n) 1 (* n (777 (- n 1))))) (- n 1)))))) We can continue in this way and write `fact2' that will work for n<=2: (define fact2 (lambda (n) (if (zero? n) 1 (* n (fact1 (- n 1)))))) or, in full form: (define fact2 (lambda (n) (if (zero? n) 1 (* n ((lambda (n) (if (zero? n) 1 (* n ((lambda (n) (if (zero? n) 1 (* n (777 (- n 1))))) (- n 1))))) (- n 1)))))) If we continue this way, we *will* get the true factorial function, but the problem is that to handle *any* possible integer argument, it will have to be an infinite definition! Here is what it is supposed to look like: (define fact0 (lambda (n) (if (zero? n) 1 (* n (777 (- n 1)))))) (define fact1 (lambda (n) (if (zero? n) 1 (* n (fact0 (- n 1)))))) (define fact2 (lambda (n) (if (zero? n) 1 (* n (fact1 (- n 1)))))) (define fact3 (lambda (n) (if (zero? n) 1 (* n (fact2 (- n 1)))))) ... The true factorial function is `fact-infinity', with an infinite size. So, we're back at the original problem... To help make things more concise, we can observe the repeated pattern in the above, and exctract a function that abstracts this pattern. This function is the same as the `fact-core' that we have seen previously: (define fact-core (lambda (fact) (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))) (define fact0 (fact-core 777)) (define fact1 (fact-core fact0)) (define fact2 (fact-core fact1)) (define fact3 (fact-core fact2)) ... which is actually: (define fact-core (lambda (fact) (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))) (define fact0 (fact-core 777)) (define fact1 (fact-core (fact-core 777))) (define fact2 (fact-core (fact-core (fact-core 777)))) ... (define fact (fact-core (fact-core (fact-core (... (fact-core 777) ...))))) Do this a little differently -- rewrite `fact0' as: (define fact0 ((lambda (mk) (mk 777)) fact-core)) Similarly, `fact1' is written as: (define fact1 ((lambda (mk) (mk (mk 777))) fact-core)) and so on, until the real factorial, which is still infinite at this stage: (define fact ((lambda (mk) (mk (mk (... (mk 777) ...)))) fact-core)) Now, look at that (lambda (mk) ...) -- it is an infinite expression, but for every actual application of the resulting factorial function we only need a finite number of `mk' applications. We can guess how many, and as soon as we hit an application of 777 we know that our guess is too small. So instead of 777, we can try to use the maker function to create and use the next. To make things more explicit, here is the expression that is our `fact0', without the definition form: ((lambda (mk) (mk 777)) fact-core) This function has a very low guess -- it works for 0, but with 1 it will run into the 777 application. At this point, we want to somehow invoke `mk' again to get the next level -- and since 777 *does* get applied, we can just replace it with `mk': ((lambda (mk) (mk mk)) fact-core) The resulting function works just the same for an input of 0 because it does not attempt a recursive call -- but if we give it 1, then instead of running into the error of applying 777: (* n (777 (- n 1))) we get to apply `fact-core' there: (* n (fact-core (- n 1))) and this is still wrong, because `fact-core' expects a function as an input. To see what happens more clearly, write `fact-core' explicitly: ((lambda (mk) (mk mk)) (lambda (fact) (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))) The problem is in what we're going to pass into `fact-core' -- its `fact' argument will not be the fuctorial function, but the `mk' function constructor. Renaming the `fact' argument as `mk' will make this more obvious (but not change the meaning): ((lambda (mk) (mk mk)) (lambda (mk) (lambda (n) (if (zero? n) 1 (* n (mk (- n 1))))))) It should now be obvious that this application of `mk' will not work, instead, we need to apply it on some function and *then* apply the result on n-1. To get what we had before, we can use 777 as a bogus function: ((lambda (mk) (mk mk)) (lambda (mk) (lambda (n) (if (zero? n) 1 (* n ((mk 777) (- n 1))))))) This will allow one recursive call -- so the definition works for both inputs of 0 and 1 -- but not more. But that 777 is used as a maker function now, so instead, we can just use `mk' itself again: ((lambda (mk) (mk mk)) (lambda (mk) (lambda (n) (if (zero? n) 1 (* n ((mk mk) (- n 1))))))) And this is a *working* version of the real factorial function, so make it into a (non-magical) definition: (define fact ((lambda (mk) (mk mk)) (lambda (mk) (lambda (n) (if (zero? n) 1 (* n ((mk mk) (- n 1)))))))) But we're not done -- we "broke" into the factorial code to insert that (mk mk) application -- that's why we dragged in the actual value of `fact-core'. We now need to fix this. The expression on that last line (lambda (n) (if (zero? n) 1 (* n ((mk mk) (- n 1))))) is close enough -- it is (fact-core (mk mk)). So we can now try to rewrite our `fact' as: (define fact-core (lambda (fact) (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))) (define fact ((lambda (mk) (mk mk)) (lambda (mk) (fact-core (mk mk))))) ... and would fail in a familiar way! If it's not familiar enough, just rename all those `mk's as `x's: (define fact-core (lambda (fact) (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))) (define fact ((lambda (x) (x x)) (lambda (x) (fact-core (x x))))) We've run into the eagerness of our language again, as we did before. The solution is the same -- the (x x) is the factorial function, so protect it as we did before, and we have a working version: (define fact-core (lambda (fact) (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))) (define fact ((lambda (x) (x x)) (lambda (x) (fact-core (lambda (n) ((x x) n)))))) The rest should not be surprising now... Abstract the recursive making bit in a new `make-recursive' function: (define fact-core (lambda (fact) (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))) (define (make-recursive f) ((lambda (x) (x x)) (lambda (x) (f (lambda (n) ((x x) n)))))) (define fact (make-recursive fact-core)) and now we can do the first reduction inside `make-recursive' and write the `fact-core' expression explicitly: #lang pl broken (define (make-recursive f) ((lambda (x) (f (lambda (n) ((x x) n)))) (lambda (x) (f (lambda (n) ((x x) n)))))) (define fact (make-recursive (lambda (fact) (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))) and this is the same code we had before. ======================================================================== >>> The Y Combinator Our `make-recursive' function is usually called the "fixpoint operator" or the "Y combinator". It looks really simple when using the lazy version (remember: our version is the eager one): (define Y (lambda (f) ((lambda (x) (f (x x))) (lambda (x) (f (x x)))))) And this all comes from the loop generated by: ((lambda (x) (x x)) (lambda (x) (x x))) ((lambda (x) (x x)) (lambda (x) (x x))), which is also called `Omega' (usually (lambda (x) (x x)) is called `omega' and (omega omega) is `Omega'), is also the idea behind many deep mathematical facts. As an example for what it does, follow the next rule: I will say the next sentence twice: "I will say the next sentence twice". (Note the usage of colon for the first and quotes for the second -- what is the equivalent of that in the lambda expression?) ======================================================================== >> The Main Property of Y `fact-core' is a function that given any limited factorial, will generate a factorial that is good for one more integer input. Start with `777', which is a factorial that is good for nothing (because it's not a function), and you can get `fact0' as fact0 == (fact-core 777) and that's a good factorial function only for an input of 0. Use that with `fact-core' again, and you get fact1 == (fact-core fact0) == (fact-core (fact-core 777)) which is the factorial function when you only look at input values of 0 or 1. In a similar way fact2 == (fact-core fact1) is good for 0..2 -- and we can continue as much as we want, except that we need to have an infinite number of applications -- in the general case, we have: fact-n == (fact-core (fact-core (fact-core ... 777))) which is good for 0..n. The *real* factorial would be the result of running `fact-core' on itself infinitely, it *is* `fact-infinity'. In other words (here `fact' is the *real* factorial): fact = fact-infinity == (fact-core (fact-core ...infinitely...)) but note that since this is really infinity, then fact = (fact-core (fact-core ...infinitely...)) = (fact-core fact) so we get an equation: fact = (fact-core fact) and a solution for this is going to be the real factorial. The solution is the "fixed-point" of the `fact-core' function, in the same sense that 0 is the fixed point of the `sin' function because 0 = (sin 0) And the Y combinator does just that -- it has this property: (make-recursive f) = (f (make-recursive f)) or, using the more common name: (Y f) = (f (Y f)) This property encapsulates the real magical power of Y. You can see how it works -- because: (Y f) = (f (Y f)) you can also say that: (f (Y f)) = (f (f (Y f))) so we get: (Y f) = (f (Y f)) = (f (f (Y f))) = (f (f (f (Y f)))) = ... = (f (f (f ...))) and we can conclude that (Y fact-core) = (fact-core (fact-core ...infinitely...)) = fact ======================================================================== >>> Typing the Y Combinator Typing the Y combinator is always a tricky issue. For example, in standard ML you must write a new type definition to do this: datatype 'a t = T of 'a t -> 'a val y = fn f => (fn (T x) => (f (fn a => x (T x) a))) (T (fn (T x) => (f (fn a => x (T x) a)))) (Can you find a pattern in the places where T is used?) In OCaml, this looks a little different: # type 'a t = T of ('a t -> 'a) ;; type 'a t = T of ('a t -> 'a) # let y f = (fun (T x) -> x (T x)) (T (fun (T x) -> fun z -> f (x (T x)) z)) ;; val y : (('a -> 'b) -> 'a -> 'b) -> 'a -> 'b = # let fact = y (fun fact n -> if n<1 then 1 else n* fact(n-1)) ;; val fact : int -> int = # fact 5 ;; - : int = 120 but OCaml also lets you start with a `-rectypes' command line argument, which will make it infer a type itself: # let y f = (fun x -> x x) (fun x -> fun z -> f (x x) z) ;; val y : (('a -> 'b) -> 'a -> 'b) -> 'a -> 'b = # let fact = y (fun fact n -> if n<1 then 1 else n* fact(n-1)) ;; val fact : int -> int = # fact 5 ;; - : int = 120 It is also possible to write this expression in typed-scheme, but we will need to write a proper type definition. First of all, the type of Y should be straightforward: it is a fixpoint operation, so it takes a `T -> T' function and produces its fixpoint. The fixpoint itself is some `T' (such that applying the function on it results in itself). So this gives us: (: make-recursive : ((T -> T) -> T)) However, in our case `make-recursive' computes a *functional* fixpoint, for unary `S -> T' functions, so we should narrow down the type (: make-recursive : (((S -> T) -> (S -> T)) -> (S -> T))) Now, in the body of `make-recursive' we need to add a type for the `x' arugment which is behaving in a weird way: it is used both as a function and as its own argument. (Remember -- I will say the next sentence twice: "I will say the next sentence twice".) We need a recursive type definition for that: (define-type (Tau S T) = (Rec this (this -> (S -> T)))) This type is tailored for our use of `x': given a type `T', `x' is a function that will consume *itself* (hence the `Rec') and spit out the value that the `f' argument consumes -- an `S -> T' function. The resulting full version of the code: (: make-recursive : (All (S T) (((S -> T) -> (S -> T)) -> (S -> T)))) (define-type (Tau S T) = (Rec this (this -> (S -> T)))) (define (make-recursive f) ((lambda: ([x : (Tau S T)]) (f (lambda: ([z : S]) ((x x) z)))) (lambda: ([x : (Tau S T)]) (f (lambda: ([z : S]) ((x x) z)))))) (: fact : (Number -> Number)) (define fact (make-recursive (lambda: ([fact : (Number -> Number)]) (lambda: ([n : Number]) (if (zero? n) 1 (* n (fact (- n 1)))))))) (fact 5) ======================================================================== >>> Lambda Calculus -- Schlac We know that many constructs that are usually thought of as primitives are not really needed -- we can implement them ourselves given enough tools. The question is how far can we go? The answer: as far as we want. For example: (define foo((lambda(f)((lambda(x)(x x))(lambda(x)(f(x x)))))(lambda(f )(lambda(x)(((x(lambda(x)(lambda(x y) y))(lambda(x y)x))(x(lambda(x)( lambda(x y)y))(lambda(x y)x))(((x(lambda(p)(lambda(s)(s(p(lambda(x y) y))(lambda(f x)(f((p(lambda(x y)y))f x))))))(lambda(s)(s(lambda(f x)x )(lambda(f x)x))))(lambda(x y)x))(lambda(x)(lambda(x y)y))(lambda(x y )x)))(lambda(f x)(f x))((f((x(lambda(p)(lambda(s)(s(p(lambda(x y)y))( lambda(f x)(f((p(lambda(x y)y))f x))))))(lambda(y s)(s(lambda(f x)x)( lambda(f x)x))))(lambda(x y)x)))(lambda(n)(lambda(f x)(f(n f x))))(f( (((x(lambda(p)(lambda(s)(s(p(lambda(x y)y))(lambda(f x)(f((p(lambda(x y)y))f x))))))(lambda(s)(s(lambda(f x)x)(lambda(f x)x))))(lambda(x y) x))(lambda(p)(lambda(s)(s(p(lambda (x y)y))(lambda(f x)(f((p(lambda(x y)y))f x))))))(lambda(s)(s(lambda(f x)x)(lambda(f x)x))))(lambda(x y) x))))))))) We begin with a very minimal language, which is based on the Lambda Calculus. In this language we get a very minimal set of constructs and values. In DrScheme, this we will use the Schlac language level (for "Scheme as Lambda Calculus"). This language uses a Scheme-like syntax, but don't be confused -- it is *very* different from Scheme. The only constructs that are available in this language are: lambda expressions of at least one argument, function application (again, at least one argument), and simple definition forms which are similar to the ones in the "Broken define" language -- definitions are used as shorthand, and cannot be used for recursive function definition. The BNF is therefore: ::= | ( ...) | (lambda ( ...) ) | (define ) Since this language has no primitive values (other than functions), Scheme numbers and booleans are also considered identifiers, and have no built-in value that come with the language. In addition, all functions and function calls are curried, so (lambda (x y z) (z y x)) is actually shorthand for (lambda (x) (lambda (y) (lambda (z) ((z y) x)))) The rules for evaluation are simple, there is one very important rule for evaluation which is called "beta reduction": ((lambda (x) E1) E2) --> E1[E2/x] where substitution in this context requires being careful so you won't capture names. This requires you to be able to do another kind of transformation which is called "alpha conversion", which basically says that you can rename identifiers as long as you keep the same binding structure (eg, a valid renaming does not change the de-bruijn form of the expression). There is one more rule that can be used, "eta conversion" which says that `(lambda (x) (f x))' is the same as `f' (we used this rule above when deriving the Y combinator). One last difference between Schlac and Scheme is that Schlac is a *lazy* language. This will be important since we do not have any built-in special forms like `if'. Here is a Schlac definition for the identity function: (define identity (lambda (x) x)) and there is not much that we can do with this now: > identity # > (identity identity) # > (identity identity identity) # (In the last expression, note that `(id id id)' is shorthand for `((id id) id)', and since `(id id)' is the identity, applying that on `id' returns it again.) ======================================================================== >>> Church Numerals So far, it seems like it is impossible to do anything useful in this `language, since all we have are functions and applications. We know how to write the identity function, but what about other values? For example, can you write code that evaluates to zero? ** What's zero? I only know how to write functions! (Turing Machine programmer: "What's a function? I only know how to write 0s and 1s!") The first thing we therefore need is to be able to *encode* numbers as functions. For zero, we will use a function of two arguments that simply returns its second value: (define 0 (lambda (f) (lambda (x) x))) or, more concisely (define 0 (lambda (f x) x)) This is the first step in an encoding that is known as "Church Numerals": an encoding of natural numbers as functions. The number zero is encoded as a function that takes in a function and a second value, and applies the function zero times on the argument (which is really what the above definition is doing). Following this view, the number one is going to be a function of two arguments, that applies the first on the second one time: (define 1 (lambda (f x) (f x))) and note that `1' is just like the identity function (as long as you give it a function as its first input, but this is always the case in Schlac). The next number on the list is two -- which applies the first argument on the second one twice: (define 2 (lambda (f x) (f (f x)))) We can go on doing this, but what we really want is a way to perform arbitrary arithmetic. The first requirement for that is an `add1' function that increments its input (an encoded natural number) by one. To do this, we write a function that expects an encoded number: (define add1 (lambda (n) ...)) and this function is expected to return an encoded number, which is always a function of `f' and `x': (define add1 (lambda (n) (lambda (f x) ...))) Now, in the body, we need to apply `f' on `x' n+1 times -- but remember that `n' is a function that will do n applications of its first argument on its second: (define add1 (lambda (n) (lambda (f x) ... (n f x) ...))) and all we have left to do now is to apply `f' one more time, yielding this definition for `add1': (define add1 (lambda (n) (lambda (f x) (f (n f x))))) Using this, we can define a few useful numbers: (define 1 (add1 0)) (define 2 (add1 1)) (define 3 (add1 2)) (define 4 (add1 3)) (define 5 (add1 4)) This is all nice theoretically, but how can we make sure that it is correct? Well, Schlac has a few additional special forms that translate Church numerals into Scheme numbers. To try our definitions we use the `->nat' (read: to natural number): (->nat 0) (->nat 5) (->nat (add1 (add1 5))) You can now verify that the identity function is really the same as the number 1: (->nat identity) We can even write a test case, since Schlac contains the `test' special form, but we have to be careful in that -- first of all, we cannot test whether functions are equal (why?) so we must use `->nat', but (test (->nat (add1 (add1 5))) => 7) will not work since `7' is undefined. To overcome this, Schlac has a `back-door' for primitive Scheme values -- just use a quote: (test (->nat (add1 (add1 5))) => '7) We can now define natural number addition -- the idea is that we get two encoded numbers `m' and `n', and what we want to do is begin with `n' and increment it by 1 m times -- and `m' is itself a function that can do that: `(m f x)' is m applications of `f' on `x', so `(m add1 n)' is m applications of `add1' on `n', which is exactly the encoding of m+n: (define + (lambda (m n) (m add1 n))) (->nat (+ 4 5)) We can also define multiplication of `m' and `n' quite easily -- begin with addition -- `(lambda (x) (+ n x))' is a function that expects an `x' and returns x+n -- it's an increment-by-n function. But since all functions and applications are curried, this is actually the same as `(lambda (x) ((+ n) x))' which is the same as `(+ n)'. Now, what we want to do is repeat this operation m times over zero, which will add n to zero m times, which will result in m*n. The definition is therefore: (define * (lambda (m n) (m (+ n) 0))) (->nat (* 4 5)) (->nat (+ 4 (* (+ 2 5) 5))) An alternative approach is to consider (lambda (x) (n f x)) for some encoded number `n' and a function `f' -- this function is like f^n (f composed n times with itself). But remember that this is shorthand for (lambda (x) ((n f) x)) and we know that `(lambda (x) (foo x))' is just like `foo' (if it is a function), so this is equivalent to just (n f) So `(n f)' is f^n, and in the same way `(m g)' is g^m -- if we use `(n f)' for `g', we get `(m (n f))' which is n self-compositions of `f', self-composed m times. In other words, `(m (n f))' is a function that is like m*n applications of `f', so we can define multiplication as: (define * (lambda (m n) (lambda (f) (m (n f))))) which is the same as (define * (lambda (m n f) (m (n f)))) The same principle can be used to define exponentiation (but now we have to be careful of the order since exponentiation is not commutative): (define ^ (lambda (m n) (n (* m) 1))) (->nat (^ 3 4)) And there is a similar alternative here too -- * a Church numeral `m' is the m-self-composition function, * and `(1 m)' is just like m^1 which is the same as `m' (1=identity) * and `(2 m)' is just like m^2 -- it takes a function `f', self composes it m times, and self composes the result m times -- for a total of f^m*m * and `(3 m)' is similarly f^m*m*m * so `(n m)' is f^(m^n) (note that the first "^" is self-compositions, and the second one is a mathematical exponent) * so `(n m)' is a function that returns m^n self-compositions of an input function, Which means that `(n m)' is the Church numeral for n^m, so we get: (define ^ (lambda (m n) (n m))) which basically says that any number encoding `n' is also the ?^n operation. All of this is was too complicated -- but all of these functions did was increment their inputs in various ways. What about `sub1'? For that, we need to do some more work -- we will need to encode booleans. ========================================================================