The FLANG Language
Now for the implementation — we call this new language FLANG.
First, the BNF:
| { + <FLANG> <FLANG> }
| { - <FLANG> <FLANG> }
| { * <FLANG> <FLANG> }
| { / <FLANG> <FLANG> }
| { with { <id> <FLANG> } <FLANG> }
| <id>
| { fun { <id> } <FLANG> }
| { call <FLANG> <FLANG> }
And the matching type definition:
[Num Number]
[Add FLANG FLANG]
[Sub FLANG FLANG]
[Mul FLANG FLANG]
[Div FLANG FLANG]
[Id Symbol]
[With Symbol FLANG FLANG]
[Fun Symbol FLANG] ; No named-expression
[Call FLANG FLANG])
The parser for this grammar is, as usual, straightforward:
;; parses s-expressions into FLANGs
(define (parse-sexpr sexpr)
(match sexpr
[(number: n) (Num n)]
[(symbol: name) (Id name)]
[(cons 'with more)
(match sexpr
[(list 'with (list (symbol: name) named) body)
(With name (parse-sexpr named) (parse-sexpr body))]
[else (error 'parse-sexpr "bad `with' syntax in ~s" sexpr)])]
[(cons 'fun more)
(match sexpr
[(list 'fun (list (symbol: name)) body)
(Fun name (parse-sexpr body))]
[else (error 'parse-sexpr "bad `fun' syntax in ~s" sexpr)])]
[(list '+ lhs rhs) (Add (parse-sexpr lhs) (parse-sexpr rhs))]
[(list '- lhs rhs) (Sub (parse-sexpr lhs) (parse-sexpr rhs))]
[(list '* lhs rhs) (Mul (parse-sexpr lhs) (parse-sexpr rhs))]
[(list '/ lhs rhs) (Div (parse-sexpr lhs) (parse-sexpr rhs))]
[(list 'call fun arg)
(Call (parse-sexpr fun) (parse-sexpr arg))]
[else (error 'parse-sexpr "bad syntax in ~s" sexpr)]))
We also need to patch up the substitution function to deal with these
things. The scoping rule for the new function form is, unsurprisingly,
similar to the rule of with
, except that there is no extra expression
now, and the scoping rule for call
is the same as for the arithmetic
operators:
{+ E1 E2}[v/x] = {+ E1[v/x] E2[v/x]}
{- E1 E2}[v/x] = {- E1[v/x] E2[v/x]}
{* E1 E2}[v/x] = {* E1[v/x] E2[v/x]}
{/ E1 E2}[v/x] = {/ E1[v/x] E2[v/x]}
y[v/x] = y
x[v/x] = v
{with {y E1} E2}[v/x] = {with {y E1[v/x]} E2[v/x]}
{with {x E1} E2}[v/x] = {with {x E1[v/x]} E2}
{call E1 E2}[v/x] = {call E1[v/x] E2[v/x]}
{fun {y} E}[v/x] = {fun {y} E[v/x]}
{fun {x} E}[v/x] = {fun {x} E}
And the matching code:
;; substitutes the second argument with the third argument in the
;; first argument, as per the rules of substitution; the resulting
;; expression contains no free instances of the second argument
(define (subst expr from to)
(cases expr
[(Num n) expr]
[(Add l r) (Add (subst l from to) (subst r from to))]
[(Sub l r) (Sub (subst l from to) (subst r from to))]
[(Mul l r) (Mul (subst l from to) (subst r from to))]
[(Div l r) (Div (subst l from to) (subst r from to))]
[(Id name) (if (eq? name from) to expr)]
[(With bound-id named-expr bound-body)
(With bound-id
(subst named-expr from to)
(if (eq? bound-id from)
bound-body
(subst bound-body from to)))]
[(Call l r) (Call (subst l from to) (subst r from to))]
[(Fun bound-id bound-body)
(if (eq? bound-id from)
expr
(Fun bound-id (subst bound-body from to)))]))
Now, before we start working on an evaluator, we need to decide on what exactly do we use to represent values of this language. Before we had functions, we had only number values and we used Racket numbers to represent them. Now we have two kinds of values — numbers and functions. It seems easy enough to continue using Racket numbers to represent numbers, but what about functions? What should be the result of evaluating
? Well, this is the new toy we have: it should be a function value,
which is something that can be used just like numbers, but instead of
arithmetic operations, we can call
these things. What we need is a way
to avoid evaluating the body expression of the function — delay it
— and instead use some value that will contain this delayed expression
in a way that can be used later.
To accommodate this, we will change our implementation strategy a
little: we will use our syntax objects for numbers ((Num n)
instead of
just n
), which will be a little inconvenient when we do the arithmetic
operations, but it will simplify life by making it possible to evaluate
functions in a similar way: simply return their own syntax object as
their values. The syntax object has what we need: the body expression
that needs to be evaluated later when the function is called, and it
also has the identifier name that should be replaced with the actual
input to the function call. This means that evaluating:
now yields
and a number (Num 5)
evaluates to (Num 5)
.
In a similar way, (Fun 'x (Num 2))
evaluates to (Fun 'x (Num 2))
.
Why would this work? Well, because call
will be very similar to with
— the only difference is that its arguments are ordered a little
differently, being retrieved from the function that is applied and the
argument.
The formal evaluation rules are therefore treating functions like numbers, and use the syntax object to represent both values:
eval({+ E1 E2}) = eval(E1) + eval(E2)
eval({- E1 E2}) = eval(E1) - eval(E2)
eval({* E1 E2}) = eval(E1) * eval(E2)
eval({/ E1 E2}) = eval(E1) / eval(E2)
eval(id) = error!
eval({with {x E1} E2}) = eval(E2[eval(E1)/x])
eval(FUN) = FUN ; assuming FUN is a function expression
eval({call E1 E2})
= eval(B[eval(E2)/x]) if eval(E1) = {fun {x} B}
= error! otherwise
Note that the last rule could be written using a translation to a with
expression:
= eval({with {x E2} B}) if eval(E1) = {fun {x} B}
= error! otherwise
And alternatively, we could specify with
using call
and fun
:
There is a small problem in these rules which is intuitively seen by the
fact that the evaluation rule for a call
is expected to be very
similar to the one for arithmetic operations. We now have two kinds of
values, so we need to check the arithmetic operation’s arguments too:
if eval(E1), eval(E2) evaluate to numbers N1, N2
otherwise error!
...
The corresponding code is:
;; evaluates FLANG expressions by reducing them to *expressions* but
;; only expressions that stand for values: only `Fun`s and `Num`s
(define (eval expr)
(cases expr
[(Num n) expr] ;*** change here
[(Add l r) (arith-op + (eval l) (eval r))]
[(Sub l r) (arith-op - (eval l) (eval r))]
[(Mul l r) (arith-op * (eval l) (eval r))]
[(Div l r) (arith-op / (eval l) (eval r))]
[(With bound-id named-expr bound-body)
(eval (subst bound-body
bound-id
(eval named-expr)))] ;*** no `(Num ...)'
[(Id name) (error 'eval "free identifier: ~s" name)]
[(Fun bound-id bound-body) expr] ;*** similar to `Num'
[(Call (Fun bound-id bound-body) arg-expr) ;*** nested pattern
(eval (subst bound-body ;*** just like `with'
bound-id
(eval arg-expr)))]
[(Call something arg-expr)
(error 'eval "`call' expects a function, got: ~s" something)]))
Note that the Call
case is doing the same thing we do in the With
case. In fact, we could have just generated a With
expression and
evaluate that instead:
[(Call (Fun bound-id bound-body) arg-expr)
(eval (With bound-id arg-expr bound-body))]
...
The arith-op
function is in charge of checking that the input values
are numbers (represented as FLANG numbers), translating them to plain
numbers, performing the Racket operation, then re-wrapping the result in
a Num
. Note how its type indicates that it is a higher-order function.
;; gets a Racket numeric binary operator, and uses it within a FLANG
;; `Num' wrapper (note the H.O. type, and note the hack of the `val`
;; name which is actually an AST that represents a runtime value)
(define (arith-op op val1 val2)
(Num (op (Num->number val1) (Num->number val2))))
It uses the following function to convert FLANG numbers to Racket
numbers. (Note that else
is almost always a bad idea since it can
prevent the compiler from showing you places to edit code — but this
case is an exception since we never want to deal with anything other
than Num
s.) The reason that this function is relatively trivial is
that we chose the easy way and represented numbers using Racket numbers,
but we could have used strings or anything else.
;; convert a FLANG number to a Racket one
(define (Num->number e)
(cases e
[(Num n) n]
[else (error 'arith-op "expected a number, got: ~s" e)]))
We can also make things a little easier to use if we make run
convert
the result to a number:
;; evaluate a FLANG program contained in a string
(define (run str)
(let ([result (eval (parse str))])
(cases result
[(Num n) n]
[else (error 'run "evaluation returned a non-number: ~s"
result)])))
Adding few simple tests we get:
#lang pl
#|
The grammar:
<FLANG> ::= <num>
| { + <FLANG> <FLANG> }
| { - <FLANG> <FLANG> }
| { * <FLANG> <FLANG> }
| { / <FLANG> <FLANG> }
| { with { <id> <FLANG> } <FLANG> }
| <id>
| { fun { <id> } <FLANG> }
| { call <FLANG> <FLANG> }
Evaluation rules:
subst:
N[v/x] = N
{+ E1 E2}[v/x] = {+ E1[v/x] E2[v/x]}
{- E1 E2}[v/x] = {- E1[v/x] E2[v/x]}
{* E1 E2}[v/x] = {* E1[v/x] E2[v/x]}
{/ E1 E2}[v/x] = {/ E1[v/x] E2[v/x]}
y[v/x] = y
x[v/x] = v
{with {y E1} E2}[v/x] = {with {y E1[v/x]} E2[v/x]} ; if y =/= x
{with {x E1} E2}[v/x] = {with {x E1[v/x]} E2}
{call E1 E2}[v/x] = {call E1[v/x] E2[v/x]}
{fun {y} E}[v/x] = {fun {y} E[v/x]} ; if y =/= x
{fun {x} E}[v/x] = {fun {x} E}
eval:
eval(N) = N
eval({+ E1 E2}) = eval(E1) + eval(E2) \ if both E1 and E2
eval({- E1 E2}) = eval(E1) - eval(E2) \ evaluate to numbers
eval({* E1 E2}) = eval(E1) * eval(E2) / otherwise error!
eval({/ E1 E2}) = eval(E1) / eval(E2) /
eval(id) = error!
eval({with {x E1} E2}) = eval(E2[eval(E1)/x])
eval(FUN) = FUN ; assuming FUN is a function expression
eval({call E1 E2}) = eval(B[eval(E2)/x])
if eval(E1)={fun {x} B}, otherwise error!
|#
(define-type FLANG
[Num Number]
[Add FLANG FLANG]
[Sub FLANG FLANG]
[Mul FLANG FLANG]
[Div FLANG FLANG]
[Id Symbol]
[With Symbol FLANG FLANG]
[Fun Symbol FLANG]
[Call FLANG FLANG])
(: parse-sexpr : Sexpr -> FLANG)
;; parses s-expressions into FLANGs
(define (parse-sexpr sexpr)
(match sexpr
[(number: n) (Num n)]
[(symbol: name) (Id name)]
[(cons 'with more)
(match sexpr
[(list 'with (list (symbol: name) named) body)
(With name (parse-sexpr named) (parse-sexpr body))]
[else (error 'parse-sexpr "bad `with' syntax in ~s" sexpr)])]
[(cons 'fun more)
(match sexpr
[(list 'fun (list (symbol: name)) body)
(Fun name (parse-sexpr body))]
[else (error 'parse-sexpr "bad `fun' syntax in ~s" sexpr)])]
[(list '+ lhs rhs) (Add (parse-sexpr lhs) (parse-sexpr rhs))]
[(list '- lhs rhs) (Sub (parse-sexpr lhs) (parse-sexpr rhs))]
[(list '* lhs rhs) (Mul (parse-sexpr lhs) (parse-sexpr rhs))]
[(list '/ lhs rhs) (Div (parse-sexpr lhs) (parse-sexpr rhs))]
[(list 'call fun arg)
(Call (parse-sexpr fun) (parse-sexpr arg))]
[else (error 'parse-sexpr "bad syntax in ~s" sexpr)]))
(: parse : String -> FLANG)
;; parses a string containing a FLANG expression to a FLANG AST
(define (parse str)
(parse-sexpr (string->sexpr str)))
(: subst : FLANG Symbol FLANG -> FLANG)
;; substitutes the second argument with the third argument in the
;; first argument, as per the rules of substitution; the resulting
;; expression contains no free instances of the second argument
(define (subst expr from to)
(cases expr
[(Num n) expr]
[(Add l r) (Add (subst l from to) (subst r from to))]
[(Sub l r) (Sub (subst l from to) (subst r from to))]
[(Mul l r) (Mul (subst l from to) (subst r from to))]
[(Div l r) (Div (subst l from to) (subst r from to))]
[(Id name) (if (eq? name from) to expr)]
[(With bound-id named-expr bound-body)
(With bound-id
(subst named-expr from to)
(if (eq? bound-id from)
bound-body
(subst bound-body from to)))]
[(Call l r) (Call (subst l from to) (subst r from to))]
[(Fun bound-id bound-body)
(if (eq? bound-id from)
expr
(Fun bound-id (subst bound-body from to)))]))
(: Num->number : FLANG -> Number)
;; convert a FLANG number to a Racket one
(define (Num->number e)
(cases e
[(Num n) n]
[else (error 'arith-op "expected a number, got: ~s" e)]))
(: arith-op : (Number Number -> Number) FLANG FLANG -> FLANG)
;; gets a Racket numeric binary operator, and uses it within a FLANG
;; `Num' wrapper
(define (arith-op op val1 val2)
(Num (op (Num->number val1) (Num->number val2))))
(: eval : FLANG -> FLANG)
;; evaluates FLANG expressions by reducing them to *expressions* but
;; only expressions that stand for values: only `Fun`s and `Num`s
(define (eval expr)
(cases expr
[(Num n) expr]
[(Add l r) (arith-op + (eval l) (eval r))]
[(Sub l r) (arith-op - (eval l) (eval r))]
[(Mul l r) (arith-op * (eval l) (eval r))]
[(Div l r) (arith-op / (eval l) (eval r))]
[(With bound-id named-expr bound-body)
(eval (subst bound-body
bound-id
(eval named-expr)))]
[(Id name) (error 'eval "free identifier: ~s" name)]
[(Fun bound-id bound-body) expr]
[(Call (Fun bound-id bound-body) arg-expr)
(eval (subst bound-body
bound-id
(eval arg-expr)))]
[(Call something arg-expr)
(error 'eval "`call' expects a function, got: ~s" something)]))
(: run : String -> Number)
;; evaluate a FLANG program contained in a string
(define (run str)
(let ([result (eval (parse str))])
(cases result
[(Num n) n]
[else (error 'run "evaluation returned a non-number: ~s"
result)])))
;; tests
(test (run "{call {fun {x} {+ x 1}} 4}")
=> 5)
(test (run "{with {add3 {fun {x} {+ x 3}}}
{call add3 1}}")
=> 4)
(test (run "{with {add3 {fun {x} {+ x 3}}}
{with {add1 {fun {x} {+ x 1}}}
{with {x 3}
{call add1 {call add3 x}}}}}")
=> 7)
There is still a problem with this version. First a question — if
call
is similar to arithmetic operations (and to with
in what it
actually does), then how come the code is different enough that it
doesn’t even need an auxiliary function?
Second question: what should happen if we evaluate these code snippets:
{fun {y}
{+ x y}}}}
{call {call add 8} 9}}")
(run "{with {identity {fun {x} x}}
{with {foo {fun {x} {+ x 1}}}
{call {call identity foo} 123}}}")
(run "{call {call {fun {x} {call x 1}}
{fun {x} {fun {y} {+ x y}}}}
123}")
Third question, what will happen if we do the above?
What we’re missing is an evaluation of the function expression, in case
it’s not a literal fun
form. The following fixes this:
;; evaluates FLANG expressions by reducing them to *expressions* but
;; only expressions that stand for values: only `Fun`s and `Num`s
(define (eval expr)
(cases expr
[(Num n) expr]
[(Add l r) (arith-op + (eval l) (eval r))]
[(Sub l r) (arith-op - (eval l) (eval r))]
[(Mul l r) (arith-op * (eval l) (eval r))]
[(Div l r) (arith-op / (eval l) (eval r))]
[(With bound-id named-expr bound-body)
(eval (subst bound-body
bound-id
(eval named-expr)))]
[(Id name) (error 'eval "free identifier: ~s" name)]
[(Fun bound-id bound-body) expr]
[(Call fun-expr arg-expr)
(define fval (eval fun-expr)) ;*** need to evaluate this!
(cases fval
[(Fun bound-id bound-body)
(eval (subst bound-body bound-id (eval arg-expr)))]
[else (error 'eval "`call' expects a function, got: ~s"
fval)])]))
The complete code is:
flang.rkt D ;; The Flang interpreter
#lang pl
#|
The grammar:
<FLANG> ::= <num>
| { + <FLANG> <FLANG> }
| { - <FLANG> <FLANG> }
| { * <FLANG> <FLANG> }
| { / <FLANG> <FLANG> }
| { with { <id> <FLANG> } <FLANG> }
| <id>
| { fun { <id> } <FLANG> }
| { call <FLANG> <FLANG> }
Evaluation rules:
subst:
N[v/x] = N
{+ E1 E2}[v/x] = {+ E1[v/x] E2[v/x]}
{- E1 E2}[v/x] = {- E1[v/x] E2[v/x]}
{* E1 E2}[v/x] = {* E1[v/x] E2[v/x]}
{/ E1 E2}[v/x] = {/ E1[v/x] E2[v/x]}
y[v/x] = y
x[v/x] = v
{with {y E1} E2}[v/x] = {with {y E1[v/x]} E2[v/x]} ; if y =/= x
{with {x E1} E2}[v/x] = {with {x E1[v/x]} E2}
{call E1 E2}[v/x] = {call E1[v/x] E2[v/x]}
{fun {y} E}[v/x] = {fun {y} E[v/x]} ; if y =/= x
{fun {x} E}[v/x] = {fun {x} E}
eval:
eval(N) = N
eval({+ E1 E2}) = eval(E1) + eval(E2) \ if both E1 and E2
eval({- E1 E2}) = eval(E1) - eval(E2) \ evaluate to numbers
eval({* E1 E2}) = eval(E1) * eval(E2) / otherwise error!
eval({/ E1 E2}) = eval(E1) / eval(E2) /
eval(id) = error!
eval({with {x E1} E2}) = eval(E2[eval(E1)/x])
eval(FUN) = FUN ; assuming FUN is a function expression
eval({call E1 E2}) = eval(B[eval(E2)/x])
if eval(E1)={fun {x} B}, otherwise error!
|#
(define-type FLANG
[Num Number]
[Add FLANG FLANG]
[Sub FLANG FLANG]
[Mul FLANG FLANG]
[Div FLANG FLANG]
[Id Symbol]
[With Symbol FLANG FLANG]
[Fun Symbol FLANG]
[Call FLANG FLANG])
(: parse-sexpr : Sexpr -> FLANG)
;; parses s-expressions into FLANGs
(define (parse-sexpr sexpr)
(match sexpr
[(number: n) (Num n)]
[(symbol: name) (Id name)]
[(cons 'with more)
(match sexpr
[(list 'with (list (symbol: name) named) body)
(With name (parse-sexpr named) (parse-sexpr body))]
[else (error 'parse-sexpr "bad `with' syntax in ~s" sexpr)])]
[(cons 'fun more)
(match sexpr
[(list 'fun (list (symbol: name)) body)
(Fun name (parse-sexpr body))]
[else (error 'parse-sexpr "bad `fun' syntax in ~s" sexpr)])]
[(list '+ lhs rhs) (Add (parse-sexpr lhs) (parse-sexpr rhs))]
[(list '- lhs rhs) (Sub (parse-sexpr lhs) (parse-sexpr rhs))]
[(list '* lhs rhs) (Mul (parse-sexpr lhs) (parse-sexpr rhs))]
[(list '/ lhs rhs) (Div (parse-sexpr lhs) (parse-sexpr rhs))]
[(list 'call fun arg)
(Call (parse-sexpr fun) (parse-sexpr arg))]
[else (error 'parse-sexpr "bad syntax in ~s" sexpr)]))
(: parse : String -> FLANG)
;; parses a string containing a FLANG expression to a FLANG AST
(define (parse str)
(parse-sexpr (string->sexpr str)))
(: subst : FLANG Symbol FLANG -> FLANG)
;; substitutes the second argument with the third argument in the
;; first argument, as per the rules of substitution; the resulting
;; expression contains no free instances of the second argument
(define (subst expr from to)
(cases expr
[(Num n) expr]
[(Add l r) (Add (subst l from to) (subst r from to))]
[(Sub l r) (Sub (subst l from to) (subst r from to))]
[(Mul l r) (Mul (subst l from to) (subst r from to))]
[(Div l r) (Div (subst l from to) (subst r from to))]
[(Id name) (if (eq? name from) to expr)]
[(With bound-id named-expr bound-body)
(With bound-id
(subst named-expr from to)
(if (eq? bound-id from)
bound-body
(subst bound-body from to)))]
[(Call l r) (Call (subst l from to) (subst r from to))]
[(Fun bound-id bound-body)
(if (eq? bound-id from)
expr
(Fun bound-id (subst bound-body from to)))]))
(: Num->number : FLANG -> Number)
;; convert a FLANG number to a Racket one
(define (Num->number e)
(cases e
[(Num n) n]
[else (error 'arith-op "expected a number, got: ~s" e)]))
(: arith-op : (Number Number -> Number) FLANG FLANG -> FLANG)
;; gets a Racket numeric binary operator, and uses it within a FLANG
;; `Num' wrapper
(define (arith-op op val1 val2)
(Num (op (Num->number val1) (Num->number val2))))
(: eval : FLANG -> FLANG)
;; evaluates FLANG expressions by reducing them to *expressions* but
;; only expressions that stand for values: only `Fun`s and `Num`s
(define (eval expr)
(cases expr
[(Num n) expr]
[(Add l r) (arith-op + (eval l) (eval r))]
[(Sub l r) (arith-op - (eval l) (eval r))]
[(Mul l r) (arith-op * (eval l) (eval r))]
[(Div l r) (arith-op / (eval l) (eval r))]
[(With bound-id named-expr bound-body)
(eval (subst bound-body
bound-id
(eval named-expr)))]
[(Id name) (error 'eval "free identifier: ~s" name)]
[(Fun bound-id bound-body) expr]
[(Call fun-expr arg-expr)
(define fval (eval fun-expr))
(cases fval
[(Fun bound-id bound-body)
(eval (subst bound-body bound-id (eval arg-expr)))]
[else (error 'eval "`call' expects a function, got: ~s"
fval)])]))
(: run : String -> Number)
;; evaluate a FLANG program contained in a string
(define (run str)
(let ([result (eval (parse str))])
(cases result
[(Num n) n]
[else (error 'run "evaluation returned a non-number: ~s"
result)])))
;; tests
(test (run "{call {fun {x} {+ x 1}} 4}")
=> 5)
(test (run "{with {add3 {fun {x} {+ x 3}}}
{call add3 1}}")
=> 4)
(test (run "{with {add3 {fun {x} {+ x 3}}}
{with {add1 {fun {x} {+ x 1}}}
{with {x 3}
{call add1 {call add3 x}}}}}")
=> 7)
(test (run "{with {add {fun {x}
{fun {y}
{+ x y}}}}
{call {call add 8} 9}}")
=> 17)
(test (run "{with {identity {fun {x} x}}
{with {foo {fun {x} {+ x 1}}}
{call {call identity foo} 123}}}")
=> 124)
(test (run "{call {call {fun {x} {call x 1}}
{fun {x} {fun {y} {+ x y}}}}
123}")
=> 124)