PL: Lecture #9  Tuesday, October 9th
(text file)

Dynamic and Lexical Scopes

This seems like it should work, and it even worked on a few examples, except for one which was hard to follow. Seems like we have a bug…

Now we get to a tricky issue that managed to be a problem for lots of language implementors, including the first version of Lisp. Lets try to run the following expression — try to figure out what it will evaluate to:

(run "{with {x 3}
        {with {f {fun {y} {+ x y}}}
          {with {x 5}
            {call f 4}}}}")

We expect it to return 7 (at least I do!), but we get 9 instead… The question is — should it return 9?

What we have arrived to is called dynamic scope. Scope is determined by the dynamic run-time environment (which is represented by our substitution cache). This is almost always undesirable, as I hope to convince you.

Before we start, we define two options for a programming language:

Racket uses lexical scope, our new evaluator uses dynamic, the old substitution-based evaluator was static etc.

As a side-remark, Lisp began its life as a dynamically-scoped language. The artifacts of this were (sort-of) dismissed as an implementation bug. When Scheme was introduced, it was the first Lisp dialect that used strictly lexical scoping, and Racket is obviously doing the same. (Some Lisp implementations used dynamic scope for interpreted code and lexical scope for compiled code!) In fact, Emacs Lisp is the only live dialects of Lisp that is still dynamically scoped by default. To see this, compare a version of the above code in Racket:

(let ((x 3))
  (let ((f (lambda (y) (+ x y))))
    (let ((x 5))
      (f 4))))

and the Emacs Lisp version (which looks almost the same):

(let ((x 3))
  (let ((f (lambda (y) (+ x y))))
    (let ((x 5))
      (funcall f 4))))

which also happens when we use another function on the way:

(defun blah (func val)
  (funcall func val))

(let ((x 3))
  (let ((f (lambda (y) (+ x y))))
    (let ((x 5))
      (blah f 4))))

and note that renaming identifiers can lead to different code — change that val to x:

(defun blah (func x)
  (funcall func x))

(let ((x 3))
  (let ((f (lambda (y) (+ x y))))
    (let ((x 5))
      (blah f 4))))

and you get 8 because the argument name changed the x that the internal function sees!

Consider also this Emacs Lisp function:

(defun return-x ()
  x)

which has no meaning by itself (x is unbound),

(return-x)

but can be given a dynamic meaning using a let:

(let ((x 5)) (return-x))

or a function application:

(defun foo (x)
  (return-x))

(foo 5)

There is also a dynamically-scoped language in the course languages:

#lang pl dynamic

(define x 123)

(define (getx) x)

(define (bar1 x) (getx))
(define (bar2 y) (getx))

(test (getx) => 123)
(test (let ([x 456]) (getx)) => 456)
(test (getx) => 123)
(test (bar1 999) => 999)
(test (bar2 999) => 123)

(define (foo x) (define (helper) (+ x 1)) helper)
(test ((foo 0)) => 124)

;; and *much* worse:
(define (add x y) (+ x y))
(test (let ([+ *]) (add 6 7)) => 42)

Note how bad the last example gets: you basically cannot call any function and know in advance what it will do.

There are some cases where dynamic scope can be useful in that it allows you to “remotely” customize any piece of code. A good example of where this is taken to an extreme is Emacs: originally, it was based on an ancient Lisp dialect that was still dynamically scoped, but it retained this feature even when practically all Lisp dialects moved on to having lexical scope by default. The reason for this is that the danger of dynamic scope is also a way to make a very open system where almost anything can be customized by changing it “remotely”. Here’s a concrete example for a similar kind of dynamic scope usage that makes a very hackable and open system:

#lang pl dynamic

(define tax% 6.25)
(define (with-tax n)
  (+ n (* n (/ tax% 100))))

(with-tax 10) ; how much do we pay?
(let ([tax% 18.0]) (with-tax 10)) ; how much would we pay in Israel?

;; make that into a function
(define il-tax% 18.0)
(define (us-over-il-saving n)
  (- (let ([tax% il-tax%]) (with-tax n))
    (with-tax n)))

(us-over-il-saving 10)
;; can even control that: how much would we save if
;; the tax in israel went down one percent?
(let ([il-tax% (- il-tax% 1)]) (us-over-il-saving 10))

;; or change both: how much savings in NH instead of MA?
(let ((tax% 0.0) (il-tax% tax%)) (us-over-il-saving 1000))

Obviously, this power to customize everything is also the main source of problems with getting no guarantees for code. A common way to get the best of both worlds is to have controllable dynamic scope. For example, Common Lisp also has lexical scope everywhere by default, but some variables can be declared as special, which means that they are dynamically scoped. The main problem with that is that you can’t tell when a variable is special by just looking at the code that uses it, so a more popular approach is the one that is used in Racket: all bindings are always lexically scoped, but there are parameters which are a kind of dynamically scoped value containers — but they are bound to plain (lexically scoped) identifiers. Here’s the same code as above, translated to Racket with parameters:

#lang racket

(define tax% (make-parameter 6.5))  ; create the dynamic container
(define (with-tax n)
  (+ n (* n (/ (tax%) 100))))      ; note how its value is accessed

(with-tax 10) ; how much do we pay?
(parameterize ([tax% 18.0]) (with-tax 10)) ; not a `let'

;; make that into a function
(define il-tax% (make-parameter 18.0))
(define (us-over-il-saving n)
  (- (parameterize ([tax% (il-tax%)]) (with-tax n))
    (with-tax n)))

(us-over-il-saving 10)
(parameterize ([il-tax% (- (il-tax%) 1)]) (us-over-il-saving 10))

The main point here is that the points where a dynamically scoped value is used are under the programmer’s control — you cannot “customize” what - is doing, for example. This gives us back the guarantees that we like to have (= that code works), but of course these points are pre-determined, unlike an environment where everything can be customized including things that are unexpectedly useful.

As a side-note, after many decades of debating this, Emacs has finally added lexical scope in its core language, but this is still determined by a flag — a global lexical-binding variable.

Dynamic versus Lexical Scope

And back to the discussion of whether we should use dynamic or lexical scope: