Lecture #3, Tuesday, January 14th ================================= - Tail calls - Note on Types - Side-note: Names are important - BNF, Grammars, the AE Language ------------------------------------------------------------------------ # Tail calls You should generally know what tail calls are, but here's a quick review of the subject. A function call is said to be in tail position if there is no context to "remember" when you're calling it. Very roughly, this means that function calls that are not nested in argument expressions of another call are tail calls. This definition is something that depends on a context, for example, in an expression like (if (huh?) (foo (add1 (* x 3))) (foo (/ x 2))) both calls to `foo` are tail calls, but they're tail calls of *this* expression and therefore apply to *this* context. It might be that this code is inside another call, as in (blah (if (huh?) (foo (add1 (* x 3))) (foo (/ x 2))) something-else) and the `foo` calls are now not in tail position. The main feature of all Scheme implementations including Racket wrt tail calls is that calls that are in tail position of a function are said to be "eliminated". That means that if we're in an `f` function, and we're about to call `g` in tail position and therefore whatever `g` returns would be the result of `f` too, then when Racket does the call to `g` it doesn't bother keeping the `f` context --- it won't remember that it needs to "return" to `f` and will instead return straight to its caller. In other words, when you think about a conventional implementation of function calls as frames on a stack, Racket will get rid of a stack frame when it can. Another way to see this is to use DrRacket's stepper to step through a function call. The stepper is generally an alternative debugger, where instead of visualizing stack frames it assembles an expression that represents these frames. Now, in the case of tail calls, there is no room in such a representation to keep the call --- and the thing is that in Racket that's perfectly fine since these calls are not kept on the call stack. Note that there are several names for this feature: * "Tail recursion". This is a common way to refer to the more limited optimization of *only* tail-recursive functions into loops. In languages that have tail calls as a feature, this is too limited, since they also optimize cases of mutual recursion, or any case of a tail call. * "Tail call optimization". In some languages, or more specifically in some compilers, you'll hear this term. This is fine when tail calls are considered only an "optimization" --- but in Racket's case (as well as Scheme), it's more than just an optimization: it's a *language feature* that you can rely on. For example, a tail-recursive function like `(define (loop) (loop))` *must* run as an infinite loop, not just optimized to one when the compiler feels like it. * "Tail call elimination". This is the so far the most common proper name for the feature: it's not just recursion, and it's not an optimization. ### When should you use tail calls? #################################### Often, people who are aware of tail calls will try to use them *always*. That's not always a good idea. You should generally be aware of the tradeoffs when you consider what style to use. The main thing to remember is that tail-call elimination is a property that helps reducing *space* use (stack space) --- often reducing it from linear space to constant space. This can obviously make things faster, but usually the speedup is just a constant factor since you need to do the same number of iterations anyway, so you just reduce the time spent on space allocation. Here is one such example that we've seen: (define (list-length-1 list) (if (null? list) 0 (+ 1 (list-length-1 (rest list))))) ;; versus (define (list-length-helper list len) (if (null? list) len (list-length-helper (rest list) (+ len 1)))) (define (list-length-2 list) (list-length-helper list 0)) In this case the first (recursive) version version consumes space linear to the length of the list, whereas the second version needs only constant space. But if you consider only the asymptotic runtime, they are both O(length(l)). A second example is a simple implementation of `map`: (define (map-1 f l) (if (null? l) l (cons (f (first l)) (map-1 f (rest l))))) ;; versus (define (map-helper f l acc) (if (null? l) (reverse acc) (map-helper f (rest l) (cons (f (first l)) acc)))) (define (map-2 f l) (map-helper f l '())) In this case, both the asymptotic space and the runtime consumption are the same. In the recursive case we have a constant factor for the stack space, and in the iterative one (the tail-call version) we also have a similar factor for accumulating the reversed list. In this case, it is probably better to keep the first version since the code is simpler. In fact, Racket's stack space management can make the first version run faster than the second --- so optimizing it into the second version is useless. ------------------------------------------------------------------------ # Note on Types Types can become interesting when dealing with higher-order functions. For example, `map` receives a function and a list of some type, and applies the function over this list to accumulate its output, so its type is: ;; map : (A -> B) (Listof A) -> (Listof B) Actually, `map` can use more than a single list, it will apply the function on the first element in all lists, then the second and so on. So the type of `map` with two lists can be described as: ;; map : (A B -> C) (Listof A) (Listof B) -> (Listof C) Here's a hairy example --- what is the type of this function: (define (foo x y) (map map x y)) Begin by what we know --- both `map`s, call them `map1` and `map2`, have the double- and single-list types of `map` respectively, here they are, with different names for types: ;; the first `map', consumes a function and two lists map1 : (A B -> C) (Listof A) (Listof B) -> (Listof C) ;; the second `map', consumes a function and one list map2 : (X -> Y) (Listof X) -> (Listof Y) Now, we know that `map2` is the first argument to `map1`, so the type of `map1`s first argument should be the type of `map2`: (A B -> C) = (X -> Y) (Listof X) -> (Listof Y) From here we can conclude that A = (X -> Y) B = (Listof X) C = (Listof Y) If we use these equations in `map1`'s type, we get: map1 : ((X -> Y) (Listof X) -> (Listof Y)) (Listof (X -> Y)) (Listof (Listof X)) -> (Listof (Listof Y)) Now, `foo`'s two arguments are the 2nd and 3rd arguments of `map1`, and its result is `map1`s result, so we can now write the type of `foo`: ;; foo : (Listof (X -> Y)) ;; (Listof (Listof X)) ;; -> (Listof (Listof Y)) (define (foo x y) (map map x y)) This should help you understand why, for example, this will cause a type error: (foo (list add1 sub1 add1) (list 1 2 3)) and why this is valid: (foo (list add1 sub1 add1) (map list (list 1 2 3))) ------------------------------------------------------------------------ ## Side-note: Names are important An important "discovery" in computer science is that we *don't* need names for every intermediate sub-expression --- for example, in almost any language we can write the equivalent of: s = (-b + sqrt(b^2 - 4*a*c)) / (2*a) instead of x = b * b y = 4 * a y = y * c x = x - y x = sqrt(x) y = -b x = y + x y = 2 * a s = x / y Such languages are put in contrast to assembly languages, and were all put under the generic label of "high level languages". (Here's an interesting idea --- why not do the same for function values?) ------------------------------------------------------------------------ # BNF, Grammars, the AE Language Getting back to the theme of the course: we want to investigate programming languages, and we want to do that *using* a programming language. The first thing when we design a language is to specify the language. For this we use BNF (Backus-Naur Form). For example, here is the definition of a simple arithmetic language: ::= | + | - Explain the different parts. Specifically, this is a mixture of low-level (concrete) syntax definition with parsing. We use this to derive expressions in some language. We start with ``, which should be one of these: * a number `` * an ``, the text "`+`", and another `` * the same but with "`-`" `` is a terminal: when we reach it in the derivation, we're done. `` is a non-terminal: when we reach it, we have to continue with one of the options. It should be clear that the `+` and the `-` are things we expect to find in the input --- because they are not wrapped in `<>`s. We could specify what `` is (turning it into a `` non-terminal): ::= | + | - ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | But we don't --- why? Because in Racket we have numbers as primitives and we want to use Racket to implement our languages. This makes life a lot easier, and we get free stuff like floats, rationals etc. To use a BNF formally, for example, to prove that `1-2+3` is a valid `` expression, we first label the rules: ::= (1) | + (2) | - (3) and then we can use them as formal justifications for each derivation step: + ; (2) + ; (1) - + ; (3) - + 3 ; (num) - + 3 ; (1) - + 3 ; (1) 1 - + 3 ; (num) 1 - 2 + 3 ; (num) This would be one way of doing this. Alternatively, we can can visualize the derivation using a tree, with the rules used at the nodes. These specifications suffer from being ambiguous: an expression can be derived in multiple ways. Even the little syntax for a number is ambiguous --- a number like `123` can be derived in two ways that result in trees that look different. This ambiguity is not a "real" problem now, but it will become one very soon. We want to get rid of this ambiguity, so that there is a single (= deterministic) way to derive all expressions. There is a standard way to resolve that --- we add another non-terminal to the definition, and make it so that each rule can continue to exactly one of its alternatives. For example, this is what we can do with numbers: ::= | ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 Similar solutions can be applied to the `` BNF --- we either restrict the way derivations can happen or we come up with new non-terminals to force a deterministic derivation trees. As an example of restricting derivations, we look at the current grammar: ::= | + | - and instead of allowing an `` on both sides of the operation, we force one to be a number: ::= | + | - Now there is a single way to derive any expression, and it is always associating operations to the right: an expression like `1+2+3` can only be derived as `1+(2+3)`. To change this to left-association, we would use this: ::= | + | - But what if we want to force precedence? Say that our AE syntax has addition and multiplication: ::= | + | * We can do that same thing as above and add new non-terminals --- say one for "products": ::= | + | ::= | * Now we must parse any AE expression as additions of multiplications (or numbers). First, note that if `` goes to `` and that goes to ``, then there is no need for an `` to go to a ``, so this is the same syntax: ::= + | ::= | * Now, if we want to still be able to multiply additions, we can force them to appear in parentheses: ::= + | ::= | * | ( ) Next, note that `` is still ambiguous about additions, which can be fixed by forcing the left hand side of an addition to be a factor: ::= + | ::= | * | ( ) We still have an ambiguity for multiplications, so we do the same thing and add another non-terminal for "atoms": ::= + | ::= * | ::= | ( ) And you can try to derive several expressions to be convinced that derivation is always deterministic now. But as you can see, this is exactly the cosmetics that we want to avoid --- it will lead us to things that might be interesting, but unrelated to the principles behind programming languages. It will also become much much worse when we have a real language rather such a tiny one. Is there a good solution? --- It is right in our face: do what Racket does --- always use fully parenthesized expressions: ::= | ( + ) | ( - ) To prevent confusing Racket code with code in our language(s), we also change the parentheses to curly ones: ::= | { + } | { - } But in Racket *everything* has a value --- including those `+`s and `-`s, which makes this extremely convenient with future operations that might have either more or less arguments than 2 as well as treating these arithmetic operators as plain functions. In our toy language we will not do this initially (that is, `+` and `-` are second order operators: they cannot be used as values). But since we will get to it later, we'll adopt the Racket solution and use a fully-parenthesized prefix notation: ::= | { + } | { - } (Remember that in a sense, Racket code is written in a form of already-parsed syntax...)