PL: Lecture #11  Tuesday, February 12th
(text file)

Fixing an Overlooked Bug

Incidentally, this version fixes a bug we had previously in the substitution version of FLANG:

(run "{with {f {fun {y} {+ x y}}}
        {with {x 7}
          {call f 1}}}")

This bug was due to our naive subst, which doesn’t avoid capturing renames. But note that since that version of the evaluator makes its way from the outside in, there is no difference in semantics for valid programs — ones that don’t have free identifiers.

(Reminder: This was not a dynamically scoped language, just a bug that happened when x wasn’t substituted away before f was replaced with something that refers to x.)

Lexical Scope using Racket Closures

PLAI §11 (without the last part about recursion)

An alternative representation for an environment.

We’ve already seen how first-class functions can be used to implement “objects” that contain some information. We can use the same idea to represent an environment. The basic intuition is — an environment is a mapping (a function) between an identifier and some value. For example, we can represent the environment that maps 'a to 1 and 'b to 2 (using just numbers for simplicity) using this function:

(: my-map : Symbol -> Number)
(define (my-map id)
  (cond [(eq? 'a id) 1]
        [(eq? 'b id) 2]
        [else (error ...)]))

An empty mapping that is implemented in this way has the same type:

(: empty-mapping : Symbol -> Number)
(define (empty-mapping id)
  (error ...))

We can use this idea to implement our environments: we only need to define three things — EmptyEnv, Extend, and lookup. If we manage to keep the contract to these functions intact, we will be able to simply plug it into the same evaluator code with no other changes. It will also be more convenient to define ENV as the appropriate function type for use in the VAL type definition instead of using the actual type:

;; Define a type for functional environments
(define-type ENV = Symbol -> VAL)

Now we get to EmptyEnv — this is expected to be a function that expects no arguments and creates an empty environment, one that behaves like the empty-mapping function defined above. We could define it like this (changing the empty-mapping type to return a VAL):

(define (EmptyEnv) empty-mapping)

but we can skip the need for an extra definition and simply return an empty mapping function:

(: EmptyEnv : -> ENV)
(define (EmptyEnv)
  (lambda (id) (error ...)))

(The un-Rackety name is to avoid replacing previous code that used the EmptyEnv name for the constructor that was created by the type definition.)

The next thing we tackle is lookup. The previous definition that was used is:

(: lookup : Symbol ENV -> VAL)
(define (lookup name env)
  (cases env
    [(EmptyEnv) (error 'lookup "no binding for ~s" name)]
    [(Extend id val rest-env)
    (if (eq? id name) val (lookup name rest-env))]))

How should it be modified now? Easy — an environment is a mapping: a Racket function that will do the searching job itself. We don’t need to modify the contract since we’re still using ENV, except a different implementation for it. The new definition is:

(: lookup : Symbol ENV -> VAL)
(define (lookup name env)
  (env name))

Note that lookup does almost nothing — it simply delegates the real work to the env argument. This is a good hint for the error message that empty mappings should throw —

(: EmptyEnv : -> ENV)
(define (EmptyEnv)
  (lambda (id) (error 'lookup "no binding for ~s" id)))

Finally, Extend — this was previously created by the variant case of the ENV type definition:

[Extend Symbol VAL ENV]

keeping the same type that is implied by this variant means that the new Extend should look like this:

(: Extend : Symbol VAL ENV -> ENV)
(define (Extend id val rest-env)
  ...)

The question is — how do we extend a given environment? Well, first, we know that the result should be mapping — a symbol -> VAL function that expects an identifier to look for:

(: Extend : Symbol VAL ENV -> ENV)
(define (Extend id val rest-env)
  (lambda (name)
    ...))

Next, we know that in the generated mapping, if we look for id then the result should be val:

(: Extend : Symbol VAL ENV -> ENV)
(define (Extend id val rest-env)
  (lambda (name)
    (if (eq? name id)
      val
      ...)))

If the name that we’re looking for is not the same as id, then we need to search through the previous environment, eg: (lookup name rest). But we know what lookup does — it simply delegates back to the mapping function (which is our rest argument), so we can take a direct route:

(: Extend : Symbol VAL ENV -> ENV)
(define (Extend id val rest-env)
  (lambda (name)
    (if (eq? name id)
      val
      (rest-env name))))

(Note that the last line is simply (lookup name rest-env), but we know that we have a functional implementation.)

To see how all this works, try out extending an empty environment a few times and examine the result. For example, the environment that we began with:

(define (my-map id)
  (cond [(eq? 'a id) 1]
        [(eq? 'b id) 2]
        [else (error ...)]))

behaves in the same way (if the type of values is numbers) as

(Extend 'a 1 (Extend 'b 2 (EmptyEnv)))

The new code is now the same, except for the environment code:

#lang pl

#|
The grammar:
  <FLANG> ::= <num>
            | { + <FLANG> <FLANG> }
            | { - <FLANG> <FLANG> }
            | { * <FLANG> <FLANG> }
            | { / <FLANG> <FLANG> }
            | { with { <id> <FLANG> } <FLANG> }
            | <id>
            | { fun { <id> } <FLANG> }
            | { call <FLANG> <FLANG> }

Evaluation rules:
  eval(N,env)                = N
  eval({+ E1 E2},env)        = eval(E1,env) + eval(E2,env)
  eval({- E1 E2},env)        = eval(E1,env) - eval(E2,env)
  eval({* E1 E2},env)        = eval(E1,env) * eval(E2,env)
  eval({/ E1 E2},env)        = eval(E1,env) / eval(E2,env)
  eval(x,env)                = lookup(x,env)
  eval({with {x E1} E2},env) = eval(E2,extend(x,eval(E1,env),env))
  eval({fun {x} E},env)      = <{fun {x} E}, env>
  eval({call E1 E2},env1)
          = eval(Ef,extend(x,eval(E2,env1),env2))
                            if eval(E1,env1) = <{fun {x} Ef}, env2>
          = error!          otherwise
|#

(define-type FLANG
  [Num  Number]
  [Add  FLANG FLANG]
  [Sub  FLANG FLANG]
  [Mul  FLANG FLANG]
  [Div  FLANG FLANG]
  [Id  Symbol]
  [With Symbol FLANG FLANG]
  [Fun  Symbol FLANG]
  [Call FLANG FLANG])

(: parse-sexpr : Sexpr -> FLANG)
;; parses s-expressions into FLANGs
(define (parse-sexpr sexpr)
  (match sexpr
    [(number: n)    (Num n)]
    [(symbol: name) (Id name)]
    [(cons 'with more)
    (match sexpr
      [(list 'with (list (symbol: name) named) body)
        (With name (parse-sexpr named) (parse-sexpr body))]
      [else (error 'parse-sexpr "bad `with' syntax in ~s" sexpr)])]
    [(cons 'fun more)
    (match sexpr
      [(list 'fun (list (symbol: name)) body)
        (Fun name (parse-sexpr body))]
      [else (error 'parse-sexpr "bad `fun' syntax in ~s" sexpr)])]
    [(list '+ lhs rhs) (Add (parse-sexpr lhs) (parse-sexpr rhs))]
    [(list '- lhs rhs) (Sub (parse-sexpr lhs) (parse-sexpr rhs))]
    [(list '* lhs rhs) (Mul (parse-sexpr lhs) (parse-sexpr rhs))]
    [(list '/ lhs rhs) (Div (parse-sexpr lhs) (parse-sexpr rhs))]
    [(list 'call fun arg)
                      (Call (parse-sexpr fun) (parse-sexpr arg))]
    [else (error 'parse-sexpr "bad syntax in ~s" sexpr)]))

(: parse : String -> FLANG)
;; parses a string containing a FLANG expression to a FLANG AST
(define (parse str)
  (parse-sexpr (string->sexpr str)))

;; Types for environments, values, and a lookup function

(define-type VAL
  [NumV Number]
  [FunV Symbol FLANG ENV])

;; Define a type for functional environments
(define-type ENV = Symbol -> VAL)

(: EmptyEnv : -> ENV)
(define (EmptyEnv)
  (lambda (id) (error 'lookup "no binding for ~s" id)))

(: Extend : Symbol VAL ENV -> ENV)
;; extend a given environment cache with a new binding
(define (Extend id val rest-env)
  (lambda (name)
    (if (eq? name id)
      val
      (rest-env name))))

(: lookup : Symbol ENV -> VAL)
;; lookup a symbol in an environment, return its value or throw an
;; error if it isn't bound
(define (lookup name env)
  (env name))

(: NumV->number : VAL -> Number)
;; convert a FLANG runtime numeric value to a Racket one
(define (NumV->number val)
  (cases val
    [(NumV n) n]
    [else (error 'arith-op "expected a number, got: ~s" val)]))

(: arith-op : (Number Number -> Number) VAL VAL -> VAL)
;; gets a Racket numeric binary operator, and uses it within a NumV
;; wrapper
(define (arith-op op val1 val2)
  (NumV (op (NumV->number val1) (NumV->number val2))))

(: eval : FLANG ENV -> VAL)
;; evaluates FLANG expressions by reducing them to values
(define (eval expr env)
  (cases expr
    [(Num n) (NumV n)]
    [(Add l r) (arith-op + (eval l env) (eval r env))]
    [(Sub l r) (arith-op - (eval l env) (eval r env))]
    [(Mul l r) (arith-op * (eval l env) (eval r env))]
    [(Div l r) (arith-op / (eval l env) (eval r env))]
    [(With bound-id named-expr bound-body)
    (eval bound-body
          (Extend bound-id (eval named-expr env) env))]
    [(Id name) (lookup name env)]
    [(Fun bound-id bound-body)
    (FunV bound-id bound-body env)]
    [(Call fun-expr arg-expr)
    (let ([fval (eval fun-expr env)])
      (cases fval
        [(FunV bound-id bound-body f-env)
          (eval bound-body
                (Extend bound-id (eval arg-expr env) f-env))]
        [else (error 'eval "`call' expects a function, got: ~s"
                            fval)]))]))

(: run : String -> Number)
;; evaluate a FLANG program contained in a string
(define (run str)
  (let ([result (eval (parse str) (EmptyEnv))])
    (cases result
      [(NumV n) n]
      [else (error 'run "evaluation returned a non-number: ~s"
                  result)])))

;; tests
(test (run "{call {fun {x} {+ x 1}} 4}")
      => 5)
(test (run "{with {add3 {fun {x} {+ x 3}}}
              {call add3 1}}")
      => 4)
(test (run "{with {add3 {fun {x} {+ x 3}}}
              {with {add1 {fun {x} {+ x 1}}}
                {with {x 3}
                  {call add1 {call add3 x}}}}}")
      => 7)
(test (run "{with {identity {fun {x} x}}
              {with {foo {fun {x} {+ x 1}}}
                {call {call identity foo} 123}}}")
      => 124)
(test (run "{with {x 3}
              {with {f {fun {y} {+ x y}}}
                {with {x 5}
                  {call f 4}}}}")
      => 7)
(test (run "{call {with {x 3}
                    {fun {y} {+ x y}}}
                  4}")
      => 7)
(test (run "{with {f {with {x 3} {fun {y} {+ x y}}}}
              {with {x 100}
                {call f 4}}}")
      => 7)
(test (run "{call {call {fun {x} {call x 1}}
                        {fun {x} {fun {y} {+ x y}}}}
                  123}")
      => 124)

More Closures (on both levels)

Racket closures (= functions) can be used in other places too, and as we have seen, they can do more than encapsulate various values — they can also hold the behavior that is expected of these values.

To demonstrate this we will deal with closures in our language. We currently use a variant that holds the three pieces of relevant information:

[FunV Symbol FLANG ENV]

We can replace this by a functional object, which will hold the three values. First, change the VAL type to hold functions for FunV values:

(define-type VAL
  [NumV Number]
  [FunV (? -> ?)])

And note that the function should somehow encapsulate the same information that was there previously, the question is how this information is going to be done, and this will determine the actual type. This information plays a role in two places in our evaluator — generating a closure in the Fun case, and using it in the Call case:

[(Fun bound-id bound-body)
(FunV bound-id bound-body env)]
[(Call fun-expr arg-expr)
(let ([fval (eval fun-expr env)])
  (cases fval
    [(FunV bound-id bound-body f-env)
      (eval bound-body                  ;***
            (Extend bound-id            ;***
                    (eval arg-expr env)  ;***
                    f-env))]            ;***
    [else (error 'eval "`call' expects a function, got: ~s"
                        fval)]))]

we can simply fold the marked functionality bit of Call into a Racket function that will be stored in a FunV object — this piece of functionality takes an argument value, extends the closure’s environment with its value and the function’s name, and continues to evaluate the function body. Folding all of this into a function gives us:

(lambda (arg-val)
  (eval bound-body (Extend bound-id arg-val env)))

where the values of bound-body, bound-id, and val are known at the time that the FunV is constructed. Doing this gives us the following code for the two cases:

[(Fun bound-id bound-body)
(FunV (lambda (arg-val)
        (eval bound-body (Extend bound-id arg-val env))))]
[(Call fun-expr arg-expr)
(let ([fval (eval fun-expr env)])
  (cases fval
    [(FunV proc) (proc (eval arg-expr env))]
    [else (error 'eval "`call' expects a function, got: ~s"
                        fval)]))]

And now the type of the function is clear:

(define-type VAL
  [NumV Number]
  [FunV (VAL -> VAL)])

And again, the rest of the code is unmodified:

#lang pl

(define-type FLANG
  [Num  Number]
  [Add  FLANG FLANG]
  [Sub  FLANG FLANG]
  [Mul  FLANG FLANG]
  [Div  FLANG FLANG]
  [Id  Symbol]
  [With Symbol FLANG FLANG]
  [Fun  Symbol FLANG]
  [Call FLANG FLANG])

(: parse-sexpr : Sexpr -> FLANG)
;; parses s-expressions into FLANGs
(define (parse-sexpr sexpr)
  (match sexpr
    [(number: n)    (Num n)]
    [(symbol: name) (Id name)]
    [(cons 'with more)
    (match sexpr
      [(list 'with (list (symbol: name) named) body)
        (With name (parse-sexpr named) (parse-sexpr body))]
      [else (error 'parse-sexpr "bad `with' syntax in ~s" sexpr)])]
    [(cons 'fun more)
    (match sexpr
      [(list 'fun (list (symbol: name)) body)
        (Fun name (parse-sexpr body))]
      [else (error 'parse-sexpr "bad `fun' syntax in ~s" sexpr)])]
    [(list '+ lhs rhs) (Add (parse-sexpr lhs) (parse-sexpr rhs))]
    [(list '- lhs rhs) (Sub (parse-sexpr lhs) (parse-sexpr rhs))]
    [(list '* lhs rhs) (Mul (parse-sexpr lhs) (parse-sexpr rhs))]
    [(list '/ lhs rhs) (Div (parse-sexpr lhs) (parse-sexpr rhs))]
    [(list 'call fun arg)
                      (Call (parse-sexpr fun) (parse-sexpr arg))]
    [else (error 'parse-sexpr "bad syntax in ~s" sexpr)]))

(: parse : String -> FLANG)
;; parses a string containing a FLANG expression to a FLANG AST
(define (parse str)
  (parse-sexpr (string->sexpr str)))

;; Types for environments, values, and a lookup function

(define-type VAL
  [NumV Number]
  [FunV (VAL -> VAL)])

;; Define a type for functional environments
(define-type ENV = Symbol -> VAL)

(: EmptyEnv : -> ENV)
(define (EmptyEnv)
  (lambda (id) (error 'lookup "no binding for ~s" id)))

(: Extend : Symbol VAL ENV -> ENV)
;; extend a given environment cache with a new binding
(define (Extend id val rest-env)
  (lambda (name)
    (if (eq? name id)
      val
      (rest-env name))))

(: lookup : Symbol ENV -> VAL)
;; lookup a symbol in an environment, return its value or throw an
;; error if it isn't bound
(define (lookup name env)
  (env name))

(: NumV->number : VAL -> Number)
;; convert a FLANG runtime numeric value to a Racket one
(define (NumV->number val)
  (cases val
    [(NumV n) n]
    [else (error 'arith-op "expected a number, got: ~s" val)]))

(: arith-op : (Number Number -> Number) VAL VAL -> VAL)
;; gets a Racket numeric binary operator, and uses it within a NumV
;; wrapper
(define (arith-op op val1 val2)
  (NumV (op (NumV->number val1) (NumV->number val2))))

(: eval : FLANG ENV -> VAL)
;; evaluates FLANG expressions by reducing them to values
(define (eval expr env)
  (cases expr
    [(Num n) (NumV n)]
    [(Add l r) (arith-op + (eval l env) (eval r env))]
    [(Sub l r) (arith-op - (eval l env) (eval r env))]
    [(Mul l r) (arith-op * (eval l env) (eval r env))]
    [(Div l r) (arith-op / (eval l env) (eval r env))]
    [(With bound-id named-expr bound-body)
    (eval bound-body
          (Extend bound-id (eval named-expr env) env))]
    [(Id name) (lookup name env)]
    [(Fun bound-id bound-body)
    (FunV (lambda (arg-val)
            (eval bound-body (Extend bound-id arg-val env))))]
    [(Call fun-expr arg-expr)
    (let ([fval (eval fun-expr env)])
      (cases fval
        [(FunV proc) (proc (eval arg-expr env))]
        [else (error 'eval "`call' expects a function, got: ~s"
                            fval)]))]))

(: run : String -> Number)
;; evaluate a FLANG program contained in a string
(define (run str)
  (let ([result (eval (parse str) (EmptyEnv))])
    (cases result
      [(NumV n) n]
      [else (error 'run "evaluation returned a non-number: ~s"
                  result)])))

;; tests
(test (run "{call {fun {x} {+ x 1}} 4}")
      => 5)
(test (run "{with {add3 {fun {x} {+ x 3}}}
              {call add3 1}}")
      => 4)
(test (run "{with {add3 {fun {x} {+ x 3}}}
              {with {add1 {fun {x} {+ x 1}}}
                {with {x 3}
                  {call add1 {call add3 x}}}}}")
      => 7)
(test (run "{with {identity {fun {x} x}}
              {with {foo {fun {x} {+ x 1}}}
                {call {call identity foo} 123}}}")
      => 124)
(test (run "{with {x 3}
              {with {f {fun {y} {+ x y}}}
                {with {x 5}
                  {call f 4}}}}")
      => 7)
(test (run "{call {with {x 3}
                    {fun {y} {+ x y}}}
                  4}")
      => 7)
(test (run "{with {f {with {x 3} {fun {y} {+ x y}}}}
              {with {x 100}
                {call f 4}}}")
      => 7)
(test (run "{call {call {fun {x} {call x 1}}
                        {fun {x} {fun {y} {+ x y}}}}
                  123}")
      => 124)

Types of Evaluators

What we did just now is implement lexical environments and closures in the language we implement using lexical environments and closures in our own language (Racket)!

This is another example of embedding a feature of the host language in the implemented language, an issue that we have already discussed.

There are many examples of this, even when the two languages involved are different. For example, if we have this bit in the C implementation of Racket:

// Disclaimer: not real Racket code
Racket_Object *eval_and( int argc, Racket_Object *argv[] )
{
  Racket_Object *tmp;
  if ( argc != 2 )
    signal_racket_error("bad number of arguments");
  else if ( racket_eval(argv[0]) != racket_false &&
            (tmp = racket_eval(argv[1])) != racket_false )
    return tmp;
  else
    return racket_false;
}

then the special semantics of evaluating a Racket and form is being inherited from C’s special treatment of &&. You can see this by the fact that if there is a bug in the C compiler, then it will propagate to the resulting Racket implementation too. A different solution is to not use && at all:

// Disclaimer: not real Racket code
Racket_Object *eval_and( int argc, Racket_Object *argv[] )
{
  Racket_Object *tmp;
  if ( argc != 2 )
    signal_racket_error("bad number of arguments");
  else if ( racket_eval(argv[0]) != racket_false )
    return racket_eval(argv[1]);
  else
    return racket_false;
}

and we can say that this is even better since it evaluates the second expression in tail position. But in this case we don’t really get that benefit, since C itself is not doing tail-call optimization as a standard feature (though some compilers do so under some circumstances).

We have seen a few different implementations of evaluators that are quite different in flavor. They suggest the following taxonomy.

While our substitution-based FLANG evaluator was close to being a syntactic evaluator, we haven’t written any purely syntactic evaluators so far: we still relied on things like Racket arithmetics etc. The most recent evaluator that we have studied, is distinctly a meta evaluator.

With a good match between the evaluated language and the implementation language, writing a meta evaluator can be very easy. With a bad match, though, it can be very hard. With a syntactic evaluator, implementing each semantic feature will be somewhat hard, but in return you don’t have to worry as much about how well the implementation and the evaluated languages match up. In particular, if there is a particularly strong mismatch between the implementation and the evaluated language, it may take less effort to write a syntactic evaluator than a meta evaluator. As an exercise, we can build upon our latest evaluator to remove the encapsulation of the evaluator’s response in the VAL type. The resulting evaluator is shown below. This is a true meta evaluator: it uses Racket closures to implement FLANG closures, Racket function application for FLANG function application, Racket numbers for FLANG numbers, and Racket arithmetic for FLANG arithmetic. In fact, ignoring some small syntactic differences between Racket and FLANG, this latest evaluator can be classified as something more specific than a meta evaluator:

(Put differently, the trivial nature of the evaluator clues us in to the deep connection between the two languages, whatever their syntactic differences may be.)

Feature Embedding

We saw that the difference between lazy evaluation and eager evaluation is in the evaluation rules for with forms, function applications, etc:

eval({with {x E1} E2}) = eval(E2[eval(E1)/x])

is eager, and

eval({with {x E1} E2}) = eval(E2[E1/x])

is lazy. But is the first rule really eager? The fact is that the only thing that makes it eager is the fact that our understanding of the mathematical notation is eager — if we were to take math as lazy, then the description of the rule becomes a description of lazy evaluation.

Another way to look at this is — take the piece of code that implements this evaluation:

(: eval : FLANG -> Number)
;; evaluates FLANG expressions by reducing them to numbers
(define (eval expr)
  (cases expr
    ...
    [(With bound-id named-expr bound-body)
    (eval (subst bound-body
                  bound-id
                  (Num (eval named-expr))))]
    ...))

and the same question applies: is this really implementing eager evaluation? We know that this is indeed eager — we can simply try it and check that it is, but it is only eager because we are using an eager language for the implementation! If our own language was lazy, then the evaluator’s implementation would run lazily, which means that the above applications of the eval and the subst functions would also be lazy, making our evaluator lazy as well.

This is a general phenomena where some of the semantic features of the language we use (math in the formal description, Racket in our code) gets embedded into the language we implement.

Here’s another example — consider the code that implements arithmetics:

(: eval : FLANG -> Number)
;; evaluates FLANG expressions by reducing them to numbers
(define (eval expr)
  (cases expr
    [(Num n) n]
    [(Add l r) (+ (eval l) (eval r))]
    ...))

what if it was written like this:

FLANG eval(FLANG expr) {
  if (is_Num(expr))
    return num_of_Num(expr);
  else if (is_Add(expr))
    return eval(lhs_of_Add(expr)) + eval(rhs_of_Add(expr));
  else if ...
  ...
}

Would it still implement unlimited integers and exact fractions? That depends on the language that was used to implement it: the above syntax suggests C, C++, Java, or some other relative, which usually come with limited integers and no exact fractions. But this really depends on the language — even our own code has unlimited integers and exact rationals only because Racket has them. If we were using a language that didn’t have such features (there are such Scheme implementations), then our implemented language would absorb these (lack of) features too, and its own numbers would be limited in just the same way. (And this includes the syntax for numbers, which we embedded intentionally, like the syntax for identifiers).

The bottom line is that we should be aware of such issues, and be very careful when we talk about semantics. Even the language that we use to communicate (semi-formal logic) can mean different things.


Aside: read “Reflections on Trusting Trust” by Ken Thompson (You can skip to the “Stage II” part to get to the interesting stuff.)

(And when you’re done, look for “XcodeGhost” to see a relevant example, and don’t miss the leaked document on the wikipedia page…)


Here is yet another variation of our evaluator that is even closer to a meta-circular evaluator. It uses Racket values directly to implement values, so arithmetic operations become straightforward. Note especially how the case for function application is similar to arithmetics: a FLANG function application translates to a Racket function application. In both cases (applications and arithmetics) we don’t even check the objects since they are simple Racket objects — if our language happens to have some meaning for arithmetics with functions, or for applying numbers, then we will inherit the same semantics in our language. This means that we now specify less behavior and fall back more often on what Racket does.

We use Racket values with this type definition:

(define-type VAL = (U Number (VAL -> VAL)))

And the evaluation function can now be:

(: eval : FLANG ENV -> VAL)
;; evaluates FLANG expressions by reducing them to values
(define (eval expr env)
  (cases expr
    [(Num n) n]                ;*** return the actual number
    [(Add l r) (+ (eval l env) (eval r env))]
    [(Sub l r) (- (eval l env) (eval r env))]
    [(Mul l r) (* (eval l env) (eval r env))]
    [(Div l r) (/ (eval l env) (eval r env))]
    [(With bound-id named-expr bound-body)
    (eval bound-body
          (Extend bound-id (eval named-expr env) env))]
    [(Id name) (lookup name env)]
    [(Fun bound-id bound-body)
    (lambda ([arg-val : VAL]) ;*** return the racket function
      ;; note that this requires input type specifications since
      ;; typed racket can't guess the right one
      (eval bound-body (Extend bound-id arg-val env)))]
    [(Call fun-expr arg-expr)
    ((eval fun-expr env)      ;*** trivial like the arithmetics!
      (eval arg-expr env))]))

Note how the arithmetics implementation is simple — it’s a direct translation of the FLANG syntax to Racket operations, and since we don’t check the inputs to the Racket operations, we let Racket throw type errors for us. Note also how function application is just like the arithmetic operations: a FLANG application is directly translated to a Racket application.

However, this does not work quite as simply in Typed Racket. The whole point of typechecking is that we never run into type errors — so we cannot throw back on Racket errors since code that might produce them is forbidden! A way around this is to perform explicit checks that guarantee that Racket cannot run into type errors. We do this with the following two helpers that are defined inside eval:

  (: evalN : FLANG -> Number)
  (define (evalN e)
    (let ([n (eval e env)])
      (if (number? n)
        n
        (error 'eval "got a non-number: ~s" n))))
  (: evalF : FLANG -> (VAL -> VAL))
  (define (evalF e)
    (let ([f (eval e env)])
      (if (function? f)
        f
        (error 'eval "got a non-function: ~s" f))))

Note that Typed Racket is “smart enough” to figure out that in evalF the result of the recursive evaluation has to be either Number or (VAL -> VAL); and since the if throws out on numbers, we’re left with (VAL -> VAL) functions, not just any function.

#lang pl

(define-type FLANG
  [Num  Number]
  [Add  FLANG FLANG]
  [Sub  FLANG FLANG]
  [Mul  FLANG FLANG]
  [Div  FLANG FLANG]
  [Id  Symbol]
  [With Symbol FLANG FLANG]
  [Fun  Symbol FLANG]
  [Call FLANG FLANG])

(: parse-sexpr : Sexpr -> FLANG)
;; parses s-expressions into FLANGs
(define (parse-sexpr sexpr)
  (match sexpr
    [(number: n)    (Num n)]
    [(symbol: name) (Id name)]
    [(cons 'with more)
    (match sexpr
      [(list 'with (list (symbol: name) named) body)
        (With name (parse-sexpr named) (parse-sexpr body))]
      [else (error 'parse-sexpr "bad `with' syntax in ~s" sexpr)])]
    [(cons 'fun more)
    (match sexpr
      [(list 'fun (list (symbol: name)) body)
        (Fun name (parse-sexpr body))]
      [else (error 'parse-sexpr "bad `fun' syntax in ~s" sexpr)])]
    [(list '+ lhs rhs) (Add (parse-sexpr lhs) (parse-sexpr rhs))]
    [(list '- lhs rhs) (Sub (parse-sexpr lhs) (parse-sexpr rhs))]
    [(list '* lhs rhs) (Mul (parse-sexpr lhs) (parse-sexpr rhs))]
    [(list '/ lhs rhs) (Div (parse-sexpr lhs) (parse-sexpr rhs))]
    [(list 'call fun arg)
                      (Call (parse-sexpr fun) (parse-sexpr arg))]
    [else (error 'parse-sexpr "bad syntax in ~s" sexpr)]))

(: parse : String -> FLANG)
;; parses a string containing a FLANG expression to a FLANG AST
(define (parse str)
  (parse-sexpr (string->sexpr str)))

;; Types for environments, values, and a lookup function

;; Values are plain Racket values, no new VAL wrapper;
;; (but note that this is a recursive definition)
(define-type VAL = (U Number (VAL -> VAL)))

;; Define a type for functional environments
(define-type ENV = (Symbol -> VAL))

(: EmptyEnv : -> ENV)
(define (EmptyEnv)
  (lambda (id) (error 'lookup "no binding for ~s" id)))

(: Extend : Symbol VAL ENV -> ENV)
;; extend a given environment cache with a new binding
(define (Extend id val rest-env)
  (lambda (name)
    (if (eq? name id)
      val
      (rest-env name))))

(: lookup : Symbol ENV -> VAL)
;; lookup a symbol in an environment, return its value or throw an
;; error if it isn't bound
(define (lookup name env)
  (env name))

(: eval : FLANG ENV -> VAL)
;; evaluates FLANG expressions by reducing them to values
(define (eval expr env)
  (: evalN : FLANG -> Number)
  (define (evalN e)
    (let ([n (eval e env)])
      (if (number? n)
        n
        (error 'eval "got a non-number: ~s" n))))
  (: evalF : FLANG -> (VAL -> VAL))
  (define (evalF e)
    (let ([f (eval e env)])
      (if (function? f)
        f
        (error 'eval "got a non-function: ~s" f))))
  (cases expr
    [(Num n) n]
    [(Add l r) (+ (evalN l) (evalN r))]
    [(Sub l r) (- (evalN l) (evalN r))]
    [(Mul l r) (* (evalN l) (evalN r))]
    [(Div l r) (/ (evalN l) (evalN r))]
    [(With bound-id named-expr bound-body)
    (eval bound-body
          (Extend bound-id (eval named-expr env) env))]
    [(Id name) (lookup name env)]
    [(Fun bound-id bound-body)
    (lambda ([arg-val : VAL])
      (eval bound-body (Extend bound-id arg-val env)))]
    [(Call fun-expr arg-expr)
    ((evalF fun-expr)
      (eval arg-expr env))]))

(: run : String -> VAL) ; no need to convert VALs to numbers
;; evaluate a FLANG program contained in a string
(define (run str)
  (eval (parse str) (EmptyEnv)))

;; tests
(test (run "{call {fun {x} {+ x 1}} 4}")
      => 5)
(test (run "{with {add3 {fun {x} {+ x 3}}}
              {call add3 1}}")
      => 4)
(test (run "{with {add3 {fun {x} {+ x 3}}}
              {with {add1 {fun {x} {+ x 1}}}
                {with {x 3}
                  {call add1 {call add3 x}}}}}")
      => 7)
(test (run "{with {identity {fun {x} x}}
              {with {foo {fun {x} {+ x 1}}}
                {call {call identity foo} 123}}}")
      => 124)
(test (run "{with {x 3}
              {with {f {fun {y} {+ x y}}}
                {with {x 5}
                  {call f 4}}}}")
      => 7)
(test (run "{call {with {x 3}
                    {fun {y} {+ x y}}}
                  4}")
      => 7)
(test (run "{with {f {with {x 3} {fun {y} {+ x y}}}}
              {with {x 100}
                {call f 4}}}")
      => 7)
(test (run "{call {call {fun {x} {call x 1}}
                        {fun {x} {fun {y} {+ x y}}}}
                  123}")
      => 124)

Recursion, Recursion, Recursion

PLAI §9

There is one major feature that is still missing from our language: we have no way to perform recursion (therefore no kind of loops). So far, we could only use recursion when we had names. In FLANG, the only way we can have names is through with which not good enough for recursion.

To discuss the issue of recursion, we switch to a “broken” version of (untyped) Racket — one where a define has a different scoping rules: the scope of the defined name does not cover the defined expression. Specifically, in this language, this doesn’t work:

#lang pl broken
(define (fact n)
  (if (zero? n) 1 (* n (fact (- n 1)))))
(fact 5)

In our language, this translation would also not work (assuming we have if etc):

{with {fact {fun {n}
              {if {= n 0} 1 {* n {call fact {- n 1}}}}}}
  {call fact 5}}

And similarly, in plain Racket this won’t work if let is the only tool you use to create bindings:

(let ([fact (lambda (n)
              (if (zero? n) 1 (* n (fact (- n 1)))))])
  (fact 5))

In the broken-scope language, the define form is more similar to a mathematical definition. For example, when we write:

(define (F x) x)
(define (G y) (F y))
(G F)

it is actually shorthand for

(define F (lambda (x) x))
(define G (lambda (y) (F y)))
(G F)

we can then replace defined names with their definitions:

(define F (lambda (x) x))
(define G (lambda (y) (F y)))
((lambda (y) (F y)) (lambda (x) x))

and this can go on, until we get to the actual code that we wrote:

((lambda (y) ((lambda (x) x) y)) (lambda (x) x))

This means that the above fact definition is similar to writing:

fact := (lambda (n)
          (if (zero? n) 1 (* n (fact (- n 1)))))
(fact 5)

which is not a well-formed definition — it is meaningless (this is a formal use of the word “meaningless”). What we’d really want, is to take the equation (using = instead of :=)

fact = (lambda (n)
        (if (zero? n) 1 (* n (fact (- n 1)))))

and find a solution which will be a value for fact that makes this true.

If you look at the Racket evaluation rules handout on the web page, you will see that this problem is related to the way that we introduced the Racket define: there is a hand-wavy explanation that talks about knowing things.

The big question is: can we define recursive functions without Racket’s magical define form?

Note: This question is a little different than the question of implementing recursion in our language — in the Racket case we have no control over the implementation of the language. As it will eventually turn out, implementing recursion in our own language will be quite easy when we use mutation in a specific way. So the question that we’re now facing can be phrased as either “can we get recursion in Racket without Racket’s magical definition forms?” or “can we get recursion in our interpreter without mutation?”.

Recursion without the Magic

PLAI §22.4 (we go much deeper)

Note: This explanation is similar to the one you can find in “The Why of Y”, by Richard Gabriel.

To implement recursion without the define magic, we first make an observation: this problem does not come up in a dynamically-scoped language. Consider the let-version of the problem:

#lang pl dynamic
(let ([fact (lambda (n)
              (if (zero? n) 1 (* n (fact (- n 1)))))])
  (fact 5))

This works fine — because by the time we get to evaluate the body of the function, fact is already bound to itself in the current dynamic scope. (This is another reason why dynamic scope is perceived as a convenient approach in new languages.)

Regardless, the problem that we have with lexical scope is still there, but the way things work in a dynamic scope suggest a solution that we can use now. Just like in the dynamic scope case, when fact is called, it does have a value — the only problem is that this value is inaccessible in the lexical scope of its body.

Instead of trying to get the value in via lexical scope, we can imitate what happens in the dynamically scoped language by passing the fact value to itself so it can call itself (going back to the original code in the broken-scope language):

(define (fact self n)  ;***
  (if (zero? n) 1 (* n (self (- n 1)))))
(fact fact 5)          ;***

except that now the recursive call should still send itself along:

(define (fact self n)
  (if (zero? n) 1 (* n (self self (- n 1))))) ;***
(fact fact 5)

The problem is that this required rewriting calls to fact — both outside and recursive calls inside. To make this an acceptable solution, calls from both places should not change. Eventually, we should be able to get a working fact definition that uses just

(lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))

The first step in resolving this problem is to curry the fact definition.

(define (fact self)                  ;***
  (lambda (n)                        ;***
    (if (zero? n)
      1
      (* n ((self self) (- n 1)))))) ;***
((fact fact) 5)                      ;***

Now fact is no longer our factorial function — it’s a function that constructs it. So call it make-fact, and bind fact to the actual factorial function.

(define (make-fact self)            ;***
  (lambda (n)
    (if (zero? n) 1 (* n ((self self) (- n 1))))))
(define fact (make-fact make-fact))  ;***
(fact 5)                            ;***

We can try to do the same thing in the body of the factorial function: instead of calling (self self), just bind fact to it:

(define (make-fact self)
  (lambda (n)
    (let ([fact (self self)])      ;***
      (if (zero? n)
        1
        (* n (fact (- n 1)))))))    ;***
(define fact (make-fact make-fact))
(fact 5)

This works fine, but if we consider our original goal, we need to get that local fact binding outside of the (lambda (n) ...) — so we’re left with a definition that uses the factorial expression as is. So, swap the two lines:

(define (make-fact self)
  (let ([fact (self self)])  ;***
    (lambda (n)              ;***
      (if (zero? n) 1 (* n (fact (- n 1)))))))
(define fact (make-fact make-fact))
(fact 5)

But the problem is that this gets us into an infinite loop because we’re trying to evaluate (self self) too early. In fact, if we ignore the body of the let and other details, we basically do this:

(define (make-fact self) (self self)) (make-fact make-fact)
--reduce-sugar-->
(define make-fact (lambda (self) (self self))) (make-fact make-fact)
--replace-definition-->
((lambda (self) (self self)) (lambda (self) (self self)))
--rename-identifiers-->
((lambda (x) (x x)) (lambda (x) (x x)))

And this expression has an interesting property: it reduces to itself, so evaluating it gets stuck in an infinite loop.

So how do we solve this? Well, we know that (self self) should be the same value that is the factorial function itself — so it must be a one-argument function. If it’s such a function, we can use a value that is equivalent, except that it will not get evaluated until it is needed, when the function is called. The trick here is the observation that (lambda (n) (add1 n)) is really the same function as add1, except that the add1 part doesn’t get evaluated until the function is called. Applying this trick to our code produces a version that does not get stuck in the same infinite loop:

(define (make-fact self)
  (let ([fact (lambda (n) ((self self) n))]) ;***
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define fact (make-fact make-fact))
(fact 5)

Continuing from here — we know that

(let ([x v]) e)  is the same as  ((lambda (x) e) v)

(remember how we derived fun from a with), so we can turn that let into the equivalent function application form:

(define (make-fact self)
  ((lambda (fact)                    ;***
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))
  (lambda (n) ((self self) n))))    ;***
(define fact (make-fact make-fact))
(fact 5)

And note now that the (lambda (fact) …) expression is everything that we need for a recursive definition of fact — it has the proper factorial body with a plain recursive call. It’s almost like the usual value that we’d want to define fact as, except that we still have to abstract on the recursive value itself. So lets move this code into a separate definition for fact-core:

(define fact-core                  ;***
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define (make-fact self)
  (fact-core                        ;***
  (lambda (n) ((self self) n))))
(define fact (make-fact make-fact))
(fact 5)

We can now proceed by moving the (make-fact make-fact) self application into its own function which is what creates the real factorial:

(define fact-core
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define (make-fact self)
  (fact-core
  (lambda (n) ((self self) n))))
(define (make-real-fact) (make-fact make-fact)) ;***
(define fact (make-real-fact))                  ;***
(fact 5)

Rewrite the make-fact definition using an explicit lambda:

(define fact-core
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define make-fact                    ;***
  (lambda (self)                      ;***
    (fact-core
    (lambda (n) ((self self) n)))))
(define (make-real-fact) (make-fact make-fact))
(define fact (make-real-fact))
(fact 5)

and fold the functionality of make-fact and make-real-fact into a single make-fact function by just using the value of make-fact explicitly instead of through a definition:

(define fact-core
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define (make-real-fact)
  (let ([make (lambda (self)                      ;***
                (fact-core                        ;***
                (lambda (n) ((self self) n))))]) ;***
    (make make)))
(define fact (make-real-fact))
(fact 5)

We can now observe that make-real-fact has nothing that is specific to factorial — we can make it take a “core function” as an argument:

(define fact-core
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define (make-real-fact core)                    ;***
  (let ([make (lambda (self)
                (core                            ;***
                (lambda (n) ((self self) n))))])
    (make make)))
(define fact (make-real-fact fact-core))          ;***
(fact 5)

and call it make-recursive:

(define fact-core
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define (make-recursive core)            ;***
  (let ([make (lambda (self)
                (core
                (lambda (n) ((self self) n))))])
    (make make)))
(define fact (make-recursive fact-core))  ;***
(fact 5)

We’re almost done now — there’s no real need for a separate fact-core definition, just use the value for the definition of fact:

(define (make-recursive core)
  (let ([make (lambda (self)
                (core
                (lambda (n) ((self self) n))))])
    (make make)))
(define fact
  (make-recursive
  (lambda (fact)                                          ;***
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))) ;***
(fact 5)

turn the let into a function form:

(define (make-recursive core)
  ((lambda (make) (make make))              ;***
  (lambda (self)                          ;***
    (core (lambda (n) ((self self) n)))))) ;***
(define fact
  (make-recursive
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))))
(fact 5)

do some renamings to make things simpler — make and self turn to x, and core to f:

(define (make-recursive f)                  ;***
  ((lambda (x) (x x))                        ;***
  (lambda (x) (f (lambda (n) ((x x) n)))))) ;***
(define fact
  (make-recursive
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))))
(fact 5)

or we can manually expand that first (lambda (x) (x x)) application to make the symmetry more obvious (not really surprising because it started with a let whose purpose was to do a self-application):

(define (make-recursive f)
  ((lambda (x) (f (lambda (n) ((x x) n))))  ;***
  (lambda (x) (f (lambda (n) ((x x) n)))))) ;***
(define fact
  (make-recursive
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))))
(fact 5)

And we finally got what we were looking for: a general way to define any recursive function without any magical define tricks. This also work for other recursive functions:

#lang pl broken
(define (make-recursive f)
  ((lambda (x) (f (lambda (n) ((x x) n))))
  (lambda (x) (f (lambda (n) ((x x) n))))))
(define fact
  (make-recursive
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))))
(fact 5)
(define fib
  (make-recursive
  (lambda (fib)
    (lambda (n) (if (<= n 1) n (+ (fib (- n 1)) (fib (- n 2))))))))
(fib 8)
(define length
  (make-recursive
  (lambda (length)
    (lambda (l) (if (null? l) 0 (+ (length (rest l)) 1))))))
(length '(x y z))

A convenient tool that people often use on paper is to perform a kind of a syntactic abstraction: “assume that whenever I write (twice foo) I really meant to write (foo foo)”. This can often be done as plain abstractions (that is, using functions), but in some cases — for example, if we want to abstract over definitions — we just want such a rewrite rule. (More on this towards the end of the course.) The broken-scope language does provide such a tool — rewrite extends the language with a rewrite rule. Using this, and our make-recursive, we can make up a recursive definition form:

(rewrite (define/rec (f x) E)
      => (define f (make-recursive (lambda (f) (lambda (x) E)))))

In other words, we’ve created our own “magical definition” form. The above code can now be written in almost the same way it is written in plain Racket:

#lang pl broken
(define (make-recursive f)
  ((lambda (x) (f (lambda (n) ((x x) n))))
  (lambda (x) (f (lambda (n) ((x x) n))))))
(rewrite (define/rec (f x) E)
      => (define f (make-recursive (lambda (f) (lambda (x) E)))))
;; examples
(define/rec (fact n) (if (zero? n) 1 (* n (fact (- n 1)))))
(fact 5)
(define/rec (fib n) (if (<= n 1) n (+ (fib (- n 1)) (fib (- n 2)))))
(fib 8)
(define/rec (length l) (if (null? l) 0 (+ (length (rest l)) 1)))
(length '(x y z))

Finally, note that make-recursive is limited to 1-argument functions only because of the protection from eager evaluation. In any case, it can be used in any way you want, for example,

(make-recursive (lambda (f) (lambda (x) f)))

is a function that returns itself rather than calling itself. Using the rewrite rule, this would be:

(define/rec (f x) f)

which is the same as:

(define (f x) f)

in plain Racket.

The Core of make-recursive

As in Racket, being able to express recursive functions is a fundamental property of the language. It means that we can have loops in our language, and that’s the essence of making a language powerful enough to be TM-equivalent — able to express undecidable problems, where we don’t know whether there is an answer or not.

The core of what makes this possible is the expression that we have seen in our derivation:

((lambda (x) (x x)) (lambda (x) (x x)))

which reduces to itself, and therefore has no value: trying to evaluate it gets stuck in an infinite loop. (This expression is often called “Omega”.)

This is the key for creating a loop — we use it to make recursion possible. Looking at our final make-recursive definition and ignoring for a moment the “protection” that we need against being stuck prematurely in an infinite loop:

(define (make-recursive f)
  ((lambda (x) (x x)) (lambda (x) (f (x x)))))

we can see that this is almost the same as the Omega expression — the only difference is that application of f. Indeed, this expression (the result of (make-recursive F) for some F) reduces in a similar way to Omega:

((lambda (x) (x x)) (lambda (x) (F (x x))))
((lambda (x) (F (x x))) (lambda (x) (F (x x))))
(F ((lambda (x) (F (x x))) (lambda (x) (F (x x)))))
(F (F ((lambda (x) (F (x x))) (lambda (x) (F (x x))))))
(F (F (F ((lambda (x) (F (x x))) (lambda (x) (F (x x)))))))
...

which means that the actual value of this expression is:

(F (F (F ...forever...)))

This definition would be sufficient if we had a lazy language, but to get things working in a strict one we need to bring back the protection. This makes things a little different — if we use (protect f) to be a shorthand for the protection trick,

(rewrite (protect f) => (lambda (x) (f x)))

then we have:

(define (make-recursive f)
  ((lambda (x) (x x)) (lambda (x) (f (protect (x x))))))

which makes the (make-recursive F) evaluation reduce to

(F (protect (F (protect (F (protect (...forever...)))))))

and this is still the same result (as long as F is a single-argument function).

(Note that protect cannot be implemented as a plain function!)

Denotational Explanation of Recursion

Note: This explanation is similar to the one you can find in “The Little Schemer” called “(Y Y) Works!”, by Dan Friedman and Matthias Felleisen.

The explanation that we have now for how to derive the make-recursive definition is fine — after all, we did manage to get it working. But this explanation was done from a kind of an operational point of view: we knew a certain trick that can make things work and we pushed things around until we got it working like we wanted. Instead of doing this, we can re-approach the problem from a more declarative point of view.

So, start again from the same broken code that we had (using the broken-scope language):

(define fact
  (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))

This is as broken as it was when we started: the occurrence of fact in the body of the function is free, which means that this code is meaningless. To avoid the compilation error that we get when we run this code, we can substitute anything for that fact — it’s even better to use a replacement that will lead to a runtime error:

(define fact
  (lambda (n) (if (zero? n) 1 (* n (777 (- n 1)))))) ;***

This function will not work in a similar way to the original one — but there is one case where it does work: when the input value is 0 (since then we do not reach the bogus application). We note this by calling this function fact0:

(define fact0                ;***
  (lambda (n) (if (zero? n) 1 (* n (777 (- n 1))))))

Now that we have this function defined, we can use it to write fact1 which is the factorial function for arguments of 0 or 1:

(define fact0
  (lambda (n) (if (zero? n) 1 (* n (777 (- n 1))))))
(define fact1
  (lambda (n) (if (zero? n) 1 (* n (fact0 (- n 1))))))

And remember that this is actually just shorthand for:

(define fact1
  (lambda (n)
    (if (zero? n)
      1
      (* n ((lambda (n)
              (if (zero? n)
                1
                (* n (777 (- n 1)))))
            (- n 1))))))

We can continue in this way and write fact2 that will work for n<=2:

(define fact2
  (lambda (n) (if (zero? n) 1 (* n (fact1 (- n 1))))))

or, in full form:

(define fact2
  (lambda (n)
    (if (zero? n)
      1
      (* n ((lambda (n)
              (if (zero? n)
                1
                (* n ((lambda (n)
                        (if (zero? n)
                          1
                          (* n (777 (- n 1)))))
                      (- n 1)))))
            (- n 1))))))

If we continue this way, we will get the true factorial function, but the problem is that to handle any possible integer argument, it will have to be an infinite definition! Here is what it is supposed to look like:

(define fact0 (lambda (n) (if (zero? n) 1 (* n (777 (- n 1))))))
(define fact1 (lambda (n) (if (zero? n) 1 (* n (fact0 (- n 1))))))
(define fact2 (lambda (n) (if (zero? n) 1 (* n (fact1 (- n 1))))))
(define fact3 (lambda (n) (if (zero? n) 1 (* n (fact2 (- n 1))))))
...

The true factorial function is fact-infinity, with an infinite size. So, we’re back at the original problem…

To help make things more concise, we can observe the repeated pattern in the above, and extract a function that abstracts this pattern. This function is the same as the fact-core that we have seen previously:

(define fact-core
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define fact0 (fact-core 777))
(define fact1 (fact-core fact0))
(define fact2 (fact-core fact1))
(define fact3 (fact-core fact2))
...

which is actually:

(define fact-core
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define fact0 (fact-core 777))
(define fact1 (fact-core (fact-core 777)))
(define fact2 (fact-core (fact-core (fact-core 777))))
...
(define fact
  (fact-core (fact-core (fact-core (... (fact-core 777) ...)))))

Do this a little differently — rewrite fact0 as:

(define fact0
  ((lambda (mk) (mk 777))
  fact-core))

Similarly, fact1 is written as:

(define fact1
  ((lambda (mk) (mk (mk 777)))
  fact-core))

and so on, until the real factorial, which is still infinite at this stage:

(define fact
  ((lambda (mk) (mk (mk (... (mk 777) ...))))
  fact-core))

Now, look at that (lambda (mk) ...) — it is an infinite expression, but for every actual application of the resulting factorial function we only need a finite number of mk applications. We can guess how many, and as soon as we hit an application of 777 we know that our guess is too small. So instead of 777, we can try to use the maker function to create and use the next.

To make things more explicit, here is the expression that is our fact0, without the definition form:

((lambda (mk) (mk 777))
fact-core)

This function has a very low guess — it works for 0, but with 1 it will run into the 777 application. At this point, we want to somehow invoke mk again to get the next level — and since 777 does get applied, we can just replace it with mk:

((lambda (mk) (mk mk))
fact-core)

The resulting function works just the same for an input of 0 because it does not attempt a recursive call — but if we give it 1, then instead of running into the error of applying 777:

(* n (777 (- n 1)))

we get to apply fact-core there:

(* n (fact-core (- n 1)))

and this is still wrong, because fact-core expects a function as an input. To see what happens more clearly, write fact-core explicitly:

((lambda (mk) (mk mk))
(lambda (fact)
  (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))

The problem is in what we’re going to pass into fact-core — its fact argument will not be the factorial function, but the mk function constructor. Renaming the fact argument as mk will make this more obvious (but not change the meaning):

((lambda (mk) (mk mk))
(lambda (mk)
  (lambda (n) (if (zero? n) 1 (* n (mk (- n 1)))))))

It should now be obvious that this application of mk will not work, instead, we need to apply it on some function and then apply the result on (- n 1). To get what we had before, we can use 777 as a bogus function:

((lambda (mk) (mk mk))
(lambda (mk)
  (lambda (n) (if (zero? n) 1 (* n ((mk 777) (- n 1)))))))

This will allow one recursive call — so the definition works for both inputs of 0 and 1 — but not more. But that 777 is used as a maker function now, so instead, we can just use mk itself again:

((lambda (mk) (mk mk))
(lambda (mk)
  (lambda (n) (if (zero? n) 1 (* n ((mk mk) (- n 1)))))))

And this is a working version of the real factorial function, so make it into a (non-magical) definition:

(define fact
  ((lambda (mk) (mk mk))
  (lambda (mk)
    (lambda (n) (if (zero? n) 1 (* n ((mk mk) (- n 1))))))))

But we’re not done — we “broke” into the factorial code to insert that (mk mk) application — that’s why we dragged in the actual value of fact-core. We now need to fix this. The expression on that last line

(lambda (n) (if (zero? n) 1 (* n ((mk mk) (- n 1)))))

is close enough — it is (fact-core (mk mk)). So we can now try to rewrite our fact as:

(define fact-core
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define fact
  ((lambda (mk) (mk mk))
  (lambda (mk) (fact-core (mk mk)))))

… and would fail in a familiar way! If it’s not familiar enough, just rename all those mks as xs:

(define fact-core
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define fact
  ((lambda (x) (x x))
  (lambda (x) (fact-core (x x)))))

We’ve run into the eagerness of our language again, as we did before. The solution is the same — the (x x) is the factorial function, so protect it as we did before, and we have a working version:

(define fact-core
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define fact
  ((lambda (x) (x x))
  (lambda (x) (fact-core (lambda (n) ((x x) n))))))

The rest should not be surprising now… Abstract the recursive making bit in a new make-recursive function:

(define fact-core
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))
(define (make-recursive f)
  ((lambda (x) (x x))
  (lambda (x) (f (lambda (n) ((x x) n))))))
(define fact (make-recursive fact-core))

and now we can do the first reduction inside make-recursive and write the fact-core expression explicitly:

#lang pl broken
(define (make-recursive f)
  ((lambda (x) (f (lambda (n) ((x x) n))))
  (lambda (x) (f (lambda (n) ((x x) n))))))
(define fact
  (make-recursive
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1))))))))

and this is the same code we had before.

The Y Combinator

Our make-recursive function is usually called the fixpoint operator or the Y combinator.

It looks really simple when using the lazy version (remember: our version is the eager one):

(define Y
  (lambda (f)
    ((lambda (x) (f (x x)))
    (lambda (x) (f (x x))))))

Note that if we do allow a recursive definition for Y itself, then the definition can follow the definition that we’ve seen:

(define (Y f) (f (Y f)))

And this all comes from the loop generated by:

((lambda (x) (x x)) (lambda (x) (x x)))

This expression, which is also called Omega (the (lambda (x) (x x)) part by itself is usually called omega and then (omega omega) is Omega), is also the idea behind many deep mathematical facts. As an example for what it does, follow the next rule:

I will say the next sentence twice:
  "I will say the next sentence twice".

(Note the usage of colon for the first and quotes for the second — what is the equivalent of that in the lambda expression?)

By itself, this just gets you stuck in an infinite loop, as Omega does, and the Y combinator adds F to that to get an infinite chain of applications — which is similar to:

I will say the next sentence twice:
  "I will hop on one foot and then say the next sentence twice".

The main property of Y

fact-core is a function that given any limited factorial, will generate a factorial that is good for one more integer input. Start with 777, which is a factorial that is good for nothing (because it’s not a function), and you can get fact0 as

fact0 == (fact-core 777)

and that’s a good factorial function only for an input of 0. Use that with fact-core again, and you get

fact1 == (fact-core fact0) == (fact-core (fact-core 777))

which is the factorial function when you only look at input values of 0 or 1. In a similar way

fact2 == (fact-core fact1)

is good for 02 — and we can continue as much as we want, except that we need to have an infinite number of applications — in the general case, we have:

fact-n == (fact-core (fact-core (fact-core ... 777)))

which is good for 0n. The real factorial would be the result of running fact-core on itself infinitely, it is fact-infinity. In other words (here fact is the real factorial):

fact = fact-infinity == (fact-core (fact-core ...infinitely...))

but note that since this is really infinity, then

fact = (fact-core (fact-core ...infinitely...))
    = (fact-core fact)

so we get an equation:

fact = (fact-core fact)

and a solution for this is going to be the real factorial. The solution is the fixed-point of the fact-core function, in the same sense that 0 is the fixed point of the sin function because

0 = (sin 0)

And the Y combinator does just that — it has this property:

(make-recursive f) = (f (make-recursive f))

or, using the more common name:

(Y f) = (f (Y f))

This property encapsulates the real magical power of Y. You can see how it works: since (Y f) = (f (Y f)), we can add an f application to both sides, giving us (f (Y f)) = (f (f (Y f))), so we get:

(Y f) = (f (Y f)) = (f (f (Y f))) = (f (f (f (Y f)))) = ...
      = (f (f (f ...)))

and we can conclude that

(Y fact-core) = (fact-core (fact-core ...infinitely...))
              = fact

Yet another explanation for Y

Here’s another explanation of how the Y combinator works. Remember that our fact-core function was actually a function that generates a factorial function based on some input, which is supposed to be the factorial function:

(define fact-core
  (lambda (fact)
    (lambda (n) (if (zero? n) 1 (* n (fact (- n 1)))))))

As we’ve seen, you can apply this function on a version of factorial that is good for inputs up to some n, and the result will be a factorial that is good for those values up to n+1. The question is what is the fixpoint of fact-core? And the answer is that if it maps factₙ factorial to factₙ₊₁, then the input will be equal to the output on the infinitieth fact, which is that actual factorial. Since Y is a fixpoint combinator, it gives us exactly that answer:

(define the-real-factorial (Y fact-core))

Typing the Y Combinator

Typing the Y combinator is a tricky issue. For example, in standard ML you must write a new type definition to do this:

datatype 'a t = T of 'a t -> 'a
val y = fn f => (fn (T x) => (f (fn a => x (T x) a)))
                  (T (fn (T x) => (f (fn a => x (T x) a))))

Can you find a pattern in the places where T is used? — Roughly speaking, that type definition is

;; `t' is the type name, `T' is the constructor (aka the variant)
(define-type (RecType a)  ; we don't really have polymorphic types
  [T ((RecType a) -> a)])

First note that the two fn a => ... parts are the same as our protection, so ignoring that we get:

val y = fn f => (fn (T x) => (f (x (T x))))
                  (T (fn (T x) => (f (x (T x)))))

if you now replace T with Quote, things make more sense:

val y = fn f => (fn (Quote x) => (f (x (Quote x))))
                  (Quote (fn (Quote x) => (f (x (Quote x)))))

and with our syntax, this would be:

(define (Y f)
  ((lambda (qx)
    (cases qx
      [(Quote x) (f (x (Quote x)))]))
  (Quote
    (lambda (qx)
      (cases qx
        [(Quote x) (f (x (Quote x)))])))))

it’s not really quotation — but the analogy should help: it uses Quote to distinguish functions as values that are applied (the xs) from functions that are passed as arguments.

In OCaml, this looks a little different:

# type 'a t = T of ('a t -> 'a) ;;
type 'a t = T of ('a t -> 'a)
# let y f = (fun (T x) -> x (T x))
            (T (fun (T x) -> fun z -> f (x (T x)) z)) ;;
val y : (('a -> 'b) -> 'a -> 'b) -> 'a -> 'b = <fun>
# let fact = y (fun fact n -> if n<1 then 1 else n* fact(n-1)) ;;
val fact : int -> int = <fun>
# fact 5 ;;
- : int = 120

but OCaml has also a -rectypes command line argument, which will make it infer the type by itself:

# let y f = (fun x -> x x) (fun x -> fun z -> f (x x) z) ;;
val y : (('a -> 'b) -> 'a -> 'b) -> 'a -> 'b = <fun>
# let fact = y (fun fact n -> if n<1 then 1 else n* fact(n-1)) ;;
val fact : int -> int = <fun>
# fact 5 ;;
- : int = 120

It is also possible to write this expression in Typed Racket, but we will need to write a proper type definition. First of all, the type of Y should be straightforward: it is a fixpoint operation, so it takes a T -> T function and produces its fixpoint. The fixpoint itself is some T (such that applying the function on it results in itself). So this gives us:

(: make-recursive : (T -> T) -> T)

However, in our case make-recursive computes a functional fixpoint, for unary S -> T functions, so we should narrow down the type

(: make-recursive : ((S -> T) -> (S -> T)) -> (S -> T))

Now, in the body of make-recursive we need to add a type for the x argument which is behaving in a weird way: it is used both as a function and as its own argument. (Remember — I will say the next sentence twice: “I will say the next sentence twice”.) We need a recursive type definition for that:

(define-type (Tau S T) = (Rec this (this -> (S -> T))))

This type is tailored for our use of x: given a type T, x is a function that will consume itself (hence the Rec) and spit out the value that the f argument consumes — an S -> T function.

The resulting full version of the code:

(: make-recursive : (All (S T) ((S -> T) -> (S -> T)) -> (S -> T)))
(define-type (Tau S T) = (Rec this (this -> (S -> T))))
(define (make-recursive f)
  ((lambda ([x : (Tau S T)]) (f (lambda (z) ((x x) z))))
  (lambda ([x : (Tau S T)]) (f (lambda (z) ((x x) z))))))

(: fact : Number -> Number)
(define fact (make-recursive
              (lambda ([fact : (Number -> Number)])
                (lambda ([n : Number])
                  (if (zero? n)
                    1
                    (* n (fact (- n 1))))))))

(fact 5)