Typing Recursion
We already know that without recursion life can be very boring… So we obviously want to be able to have recursive functions — but the question is how will they interact with our type system. One thing that we have seen is that by just having functions we get recursion. This was achieved by the Y combinator function. It seems like the same should apply to our simple typed language. The core of the Y combinator was using an expression similar to Omega that generates the infinite loop that is needed. In our language:
This expression was impossible to evaluate completely since it never
terminates, but it served as a basis for the Y combinator so we need to
be able to perform this kind of infinite loop. Now, consider the type of
the first x
— it’s used in a call
expression as a function, so its
type must be a function type, say τ₁->τ₂. In addition, its argument is
x
itself so its type is also τ₁ — this means that we have:
and from this we get:
= (τ₁ -> τ₂) -> τ₂
= ((τ₁ -> τ₂) -> τ₂) -> τ₂
= ...
And this is a type that does not exist in our type system, since we can only have finite types. Therefore, we have a proof by contradiction that this expression cannot be typed in our system.
This is closely related to the fact that the typed language we have
described so far is “strongly normalizing”: no matter what program you
write, it will always terminate! To see this, very informally, consider
this language without functions — this is clearly a language where all
programs terminate, since the only way to create a loop is through
function applications. Now add functions and function application — in
the typing rules for the resulting language, each fun
creates a
function type (creates an arrow), and each function application consumes
a function type (deletes one arrow) — since types are finite, the
number of arrows is finite, which means that the number of possible
applications is finite, so all programs must run in finite time.
Note that when we discussed how to type the Y combinator we needed to use a
Rec
constructor — something that the current type system has. Using that, we could have easily solve theτ₁ = τ₁ -> τ₂
equation with(Rec τ₁ (τ₁ -> τ₂))
.
In the our language, therefore, the halting problem doesn’t even exist, since all programs (that are properly typed) are guaranteed to halt. This property is useful in many real-life situations (consider firewall rules, configuration files, devices with embedded code). But the language that we get is very limited as a result — we really want the power to shoot our feet…
Extending Picky with recursion
As we have seen, our language is strongly normalizing, which means that
to get general recursion, we must introduce a new construct (unlike
previously, when we didn’t really need one). We can do this as we
previously did — by adding a new construct to the language, or we can
somehow extend the (sub) language of type descriptions to allow a new
kind of type that can be used to solve the τ₁ = τ₁ -> τ₂
equation. An
example of this solution would be similar to the Rec
type constructor
in Typed Racket: a new type constructor that allows a type to refer to
itself — and using (Rec τ₁ (τ₁ -> τ₂))
as the solution. However,
this complicates things: type descriptions are no longer unique, since
we have Num
, (Rec this Num)
, and (Rec this (Rec that Num))
that
are all equal.
For simplicity we will now take the first route and add rec
— an
explicit recursive binder form to the language (as with with
, we’re
going back to rec
rather than bindrec
to keep things simple).
First, the new BNF:
| <id>
| { + <PICKY> <PICKY> }
| { < <PICKY> <PICKY> }
| { fun { <id> : <TYPE> } : <TYPE> <PICKY> }
| { call <PICKY> <PICKY> }
| { with { <id> : <TYPE> <PICKY> } <PICKY> }
| { rec { <id> : <TYPE> <PICKY> } <PICKY> }
| { if <PICKY> <PICKY> <PICKY> }
<TYPE> ::= Number
| Boolean
| ( <TYPE> -> <TYPE> )
We now need to add a typing judgment for rec
expressions. What should
it look like?
———————————————————————————
Γ ⊢ {rec {x : τ₁ V} E} : τ₂
rec
is similar to all the other local binding forms, like with
, it
can be seen as a combination of a function and an application. So we
need to check the two things that those rules checked — first, check
that the body expression has the right type assuming that the type
annotation given to x
is valid:
———————————————————————————
Γ ⊢ {rec {x : τ₁ V} E} : τ₂
Now, we also want to add the other side — making sure that the τ₁ type annotation is valid:
——————————————————————————————
Γ ⊢ {rec {x : τ₁ V} E} : τ₂
But that will not be possible in general — V
is an expression that
usually includes x
itself — that’s the whole point. The conclusion
is that we should use a similar trick to the one that we used to specify
evaluation of recursive binders — the same environment is used for
both the named expression and for the body expression:
—————————————————————————————————————
Γ ⊢ {rec {x : τ₁ V} E} : τ₂
You can also see now that if this rule adds an arrow type to the Γ type
environment (i.e., τ₁
is an arrow), then it is doing so in a way that
makes it possible to use it over and over, making it possible to run
infinite loops in this language.
Our complete language specification is below.
| <id>
| { + <PICKY> <PICKY> }
| { < <PICKY> <PICKY> }
| { fun { <id> : <TYPE> } : <TYPE> <PICKY> }
| { call <PICKY> <PICKY> }
| { with { <id> : <TYPE> <PICKY> } <PICKY> }
| { rec { <id> : <TYPE> <PICKY> } <PICKY> }
| { if <PICKY> <PICKY> <PICKY> }
<TYPE> ::= Number
| Boolean
| ( <TYPE> -> <TYPE> )
Γ ⊢ n : Number
Γ ⊢ x : Γ(x)
Γ ⊢ A : Number Γ ⊢ B : Number
———————————————————————————————
Γ ⊢ {+ A B} : Number
Γ ⊢ A : Number Γ ⊢ B : Number
———————————————————————————————
Γ ⊢ {< A B} : Boolean
Γ[x:=τ₁] ⊢ E : τ₂
——————————————————————————————————————
Γ ⊢ {fun {x : τ₁} : τ₂ E} : (τ₁ -> τ₂)
Γ ⊢ F : (τ₁ -> τ₂) Γ ⊢ V : τ₁
——————————————————————————————
Γ ⊢ {call F V} : τ₂
Γ ⊢ C : Boolean Γ ⊢ T : τ Γ ⊢ E : τ
———————————————————————————————————————
Γ ⊢ {if C T E} : τ
Γ ⊢ V : τ₁ Γ[x:=τ₁] ⊢ E : τ₂
——————————————————————————————
Γ ⊢ {with {x : τ₁ V} E} : τ₂
Γ[x:=τ₁] ⊢ V : τ₁ Γ[x:=τ₁] ⊢ E : τ₂
—————————————————————————————————————
Γ ⊢ {rec {x : τ₁ V} E} : τ₂
Typing Data
An important concept that we have avoided so far is user-defined types. This issue exists in practically all languages, including the ones we did so far, since a language without the ability to create new user-defined types is a language with a major problem. (As a side note, we did talk about mimicking an object system using plain closures, but it turns out that this is insufficient as a replacement for true user-defined types — you can kind of see that in the Schlac language, where the lack of all types mean that there is no type error.)
In the context of a statically typed language, this issue is even more
important. Specifically, we talked about typing recursive code, but we
should also consider typing recursive data. For example, we will start
with a length
function in an extension of the language that has
empty?
, rest
, and NumCons
and NumEmpty
constructors:
{fun {l : ???} : Number
{if {empty? l}
0
{+ 1 {call length {rest l}}}}}}
{call length {NumCons 1 {NumCons 2 {NumCons 3 {NumEmpty}}}}}}
But adding all of these new functions as built-ins is getting messy: we
want our language to have a form for defining new kinds of data. In this
example — we want to be able to define the NumList
type for lists of
numbers. We therefore extend the language with a new with-type
form
for creating new user-defined types, using variants in a similar way to
our own course language:
[NumCons Number ???]}
{rec {length : ???
{fun {l : ???} : Number
...}}
...}}
We assume here that the NumList
definition provides us with a number
of new built-ins — NumEmpty
and NumCons
constructors, and assume
also a cases
form that can be used to both test a value and access its
components (with the constructors serving as patterns). This makes the
code a little different than what we started with:
[NumCons Number ???]}
{rec {length : ???
{fun {l : ???} : Number
{cases l
[{NumEmpty} 0]
[{NumCons x r} {+ 1 {call length r}}]}}}
{call length {NumCons 1 {NumCons 2 {NumCons 3 {NumEmpty}}}}}}}
The question is what should the ???
be filled with? Clearly, recursive
data types are very common and we need to support them. The scope of
with-type
should therefore be similar to rec
, except that it works
at the type level: the new type is available for its own definition.
This is the complete code now:
[NumCons Number NumList]}
{rec {length : (NumList -> Number)
{fun {l : NumList} : Number
{cases l
[{NumEmpty} 0]
[{NumCons x r} {+ 1 {call length r}}]}}}
{call length {NumCons 1 {NumCons 2 {NumCons 3 {NumEmpty}}}}}}}
(Note that in the course language we can do just that, and in addition,
the Rec
type constructor can be used to make up recursive types.)
An important property that we would like this type to have is for it to
be well founded: that we’d never get stuck in some kind of type-level
infinite loop. To see that this holds in this example, note that some of
the variants are self-referential (only NumCons
here), but there is at
least one that is not (NumEmpty
) — if there wasn’t any simple
variant, then we would have no way to construct instances of this type
to begin with!
[As a side note, if the language has lazy semantics, we could use such types — for example:
{rec {ones : NumList {NumCons 1 ones}}
...}}
Reasoning about such programs requires more than just induction though.]
Judgments for recursive types
If we want to have a language that is basically similar to the course
language, then — as seen above — we’d use a similar cases
expression. How should we type-check such expressions? In this case, we
want to verify this:
[{NumCons x r} {+ 1 {call length r}}]} : Number
Similarly to the judgment for if
expressions, we require that the two
result expressions are numbers. Indeed, you can think about cases
as a
more primitive tool that has the functionality of if
— in other
words, given such user-defined types we could implement booleans as a
new type and and implement if
using cases
. For example, wrap
programs with:
and translate {if E1 E2 E3}
to {cases E1 [{True} E2] [{False} E3]}
.
Continuing with typing cases
, we now have:
————————————————————————————————————————————————————————————
Γ ⊢ {cases l [{NumEmpty} 0]
[{NumCons x r} {+ 1 {call length r}}]} : Number
But this will not work — we have no type for r
here, so we can’t
prove the second subgoal. We need to consider the NumList
type
definition as something that, in addition to the new built-ins, provides
us with type judgments for these built-ins. In the case of the NumCons
variant, we know that using {NumCons x r}
is a pattern that matches
NumList
values that are a result of this variant constructor but it
also binds x
and r
to the values of the two fields, and since all
uses of the constructor are verified, the fields have the declared
types. This means that we need to extend Γ in this rule so we’re able to
prove the two subgoals. Note that we do the same for the NumEmpty
case, except that there are no new bindings there.
Γ[x:=Number; r:=NumList] ⊢ {+ 1 {call length r}} : Number
————————————————————————————————————————————————————————————
Γ ⊢ {cases l [{NumEmpty} 0]
[{NumCons x r} {+ 1 {call length r}}]} : Number
Finally, we need to verify that the value itself — l
— has the
right type: that it is a NumList
.
Γ ⊢ 0 : Number
Γ[x:=Number; r:=NumList] ⊢ {+ 1 {call length r}} : Number
————————————————————————————————————————————————————————————
Γ ⊢ {cases l [{NumEmpty} 0]
[{NumCons x r} {+ 1 {call length r}}]} : Number
But why NumList
and not some other defined type? This judgment needs
to do a little more work: it should inspect all of the variants that are
used in the branches, find the type that defines them, then use that
type as the subgoal. Furthermore, to make the type checker more useful,
it can check that we have complete coverage of the variants, and that no
variant is used twice:
(also need to show that NumEmpty and NumCons are all of
the variants of NumList, with no repetition or extras.)
Γ ⊢ 0 : Number
Γ[x:=Number; r:=NumList] ⊢ {+ 1 {call length r}} : Number
————————————————————————————————————————————————————————————
Γ ⊢ {cases l [{NumEmpty} 0]
[{NumCons x r} {+ 1 {call length r}}]} : Number
Note that how this is different from the version in the textbook — it
has a type-case
expression with the type name mentioned explicitly —
for example: {type-case l NumList {{NumEmpty} 0} ...}
. This is
essentially the same as having each defined type come with its own
cases
expression. Our rule needs to do a little more work, but overall
it is a little easier to use. (And the same goes for the actual
implementation of the two languages.)
In addition to cases
, we should also have typing judgments for the
constructors. These are much simpler, for example:
————————————————————————————————
Γ ⊢ {NumCons x r} : NumList
Alternatively, we could add the constructors as new functions instead of
new special forms — so in the Picky language they’d be used in call
expressions. The with-type
will then create the bindings for its scope
at runtime, and for the typechecker it will add the relevant types to Γ:
(This requires functions of any arity, of course.) Using accessor
functions could be similarly simpler than cases
, but less convenient
for users.
Note about representation: a by-product of our type checker is that
whenever we have a NumList
value, we know that it must be an
instance of either NumEmpty
or NumCons
. Therefore, we could
represent such values as a wrapped value container, with a single bit
that distinguishes the two. This is in contrast to dynamically typed
languages like Racket, where every new type needs to have its own
globally unique tag.
extra “Runaway” instances
Consider this code:
We now know how to type check its validity, but what about the type of
this whole expression? The obvious choice would be NumList
:
There is a subtle but important problem here: the expression evaluates
to a NumList
, but we can no longer use this value, since we’re out of
the scope of the NumList
type definition! In other words, we would
typecheck a program that is pretty much useless.
Even if we were to allow such a value to flow to a different context
with a NumList
type definition, we wouldn’t want the two to be
confused — following the principle of lexical scope, we’d want each
type definition to be unique to its own scope even if it has the same
concrete name. For example, using NumList
as the type of the inner
with-type
here:
{with-type {NumList [NumEmpty] ...}
{NumEmpty}}}
would make it wrong.
(In fact, we might want to have a new type even if the value goes
outside of this scope and back in. The default struct definitions in
Racket have exactly this property — they’re generative — which
means that each “call” to define-struct
creates a new type, so:
(define (foo x)
(struct foo (x))
(foo x))
(list (foo 1) (foo 2)))
returns two instances of two different foo
types!)
One way to resolve this is to just forbid the type from escaping the
scope of its definition — so we would forbid the type of the
expression from being NumList
, which makes
invalid. But that’s not enough — what about returning a compound value
that contains an instance of NumList
? For example — what if we
return a list or a function with a NumList
instance?
{fun {x} {NumEmpty}}} : Num -> NumList??
Obviously, we would need to extend this restriction: the resulting type
should not mention the defined type at all — not even in lists or
functions or anything else. This is actually easy to do: if the overall
expression is type-checked in the surrounding lexical scope, then it is
type-checked in the surrounding type environment (Γ), and that
environment has nothing in it about NumList
(well, nothing about
this NumList
).
Note that this is, very roughly speaking, what our course language does:
define-type
can only define new types when it is used at the
top-level.
This works fine with the above assumption that such a value would be
completely useless — but there are aspects of such values that are
useful. Such types are close to things that are known as “existential
types”, and they are for defining opaque values that you can do nothing
with except pass them around, and only code in a specific lexical
context can actually use them. For example, you could lump together the
value with a function that can work on this value. If it wasn’t for the
define-type
top-level restriction, we could write the following:
(define (foo x)
(define-type FOO [Foo Integer])
(list (Foo 1)
(lambda (f)
(cases f [(Foo n) (* n n)]))))
There is nothing that we can do with resulting Foo
instance (we don’t
even have a way to name it) — but in the result of the above function
we get also a function that could work on such values, even ones from
different calls:
Since such kind of values are related to hiding information, they’re useful (among other things) when talking about module systems (and object systems), where you want to have a local scope for a piece of code with bindings that are not available outside it.
Type soundness
Having a type checker is obviously very useful — but to be able to
rely on it, we need to provide some kind of a formal account of the
kind of guarantees that we get by using one. Specifically, we want to
guarantee that a program that type-checks is guaranteed to never fail
with a type error. Such type errors in Racket result in an exception
— but in C they can result in anything. In our simple Picky
implementation, we still need to check the resulting value in run
:
(let ([result (eval prog (EmptyEnv))])
(cases result
[(NumV n) n]
;; this error is never reached, since we make sure
;; that the program always evaluates to a number above
[else (error 'run "evaluation returned a non-number: ~s"
result)]))
A soundness proof for this would show that checking the result (in
cases
) is not needed. However, the check must be there since Typed
Racket (or any other typechecker) is far from making up and verifying
such a proof by itsef.
In this context we have a specific meaning for “fail with a type error”, but these failures can be very different based on the kind of properties that your type checker verifies. This property of a type system is called soundness: a sound type system is one that will never allow such errors for type-checked code:
For any program
p
, if we can type-checkp : τ
, thenp
will evaluate to a value that is in the typeτ
.
The importance of this can be seen in that it is the only connection between the type system and code execution. Without it, a type system is a bunch of syntactic rules that are completely disconnected from how the program runs. (Note also that — “in the type” — works for the (common) case where types are sets of values.)
But this statement isn’t exactly what we need — it states a property
that is too strong: what if execution gets stuck in an infinite loop?
(That wasn’t needed before we introduced rec
, where we could extend
the conclusion part to: “… then p
will terminate and evaluate to a
value that is in the type τ
”.) We therefore need to revise it:
For any program
p
, if we can type-checkp : τ
, and ifp
terminates and returnsv
, thenv
is in the typeτ
.
But there are still problems with this. Some programs evaluate to a
value, some get stuck in an infinite loop, and some … throw an error.
Even with type checking, there are still cases when we get runtime
errors. For example, in practically all statically typed languages the
length of a list is not encoded in its type, so {first null}
would
throw an error. (It’s possible to encode more information like that in
types, but there is a downside to this too: putting more information in
the type system means that things get less flexible, and it becomes more
difficult to write programs since you’re moving towards proving more
facts about them.)
Even if we were to encode list lengths in the type, we would still have runtime errors: opening a missing file, writing to a read-only file fetching a non-existent URL, etc, so we must find some way to account for these errors. Some “solutions” are:
-
For all cases where an error should be raised, just return some value (of the appropriate type). For example,
(first l)
could return0
if the list is empty;(substring "foo" 10 20)
would return “huh?”, etc. It seems like a dangerous way to resolve the issue, but in fact that’s what most C library calls do: return some bogus value (for example,malloc()
returnsNULL
when there is no available memory), and possibly set some global flag that specifies the exact error. (The main problem with this is that C programmers often don’t check all of these conditions, leading to propagating undetected errors further down — and all of this is a very rich source of security issues.) -
For all cases where an error should be raised, just get stuck into an infinite loop. This approach is obviously impractical — but it is actually popular in some theoretical circles. The reason for that is that theory people will often talk about “domains”, and to express facts about computation on these domains, they’re extended with a “bottom” value that represents a diverging computation. Since this introduction is costly in terms of work that it requires, adding one more such value can lead to more effort than re-using the same “bottom” value.
-
Raise an exception. This works out better than the above two extremes, and it is the approach taken by practically all modern languages.
So, assuming exceptions, we need to further refine what it means for a type system to be sound:
For any program
p
, if we can type-checkp : τ
, and ifp
terminates without exceptions and returnsv
, thenv
is in the typeτ
.
An important thing to note here is that languages can have very different ideas about where to raise an exception. For example, Scheme implementations often have a trivial type-checker and throw runtime exceptions when there is a type error. On the other hand, there are systems that express much more in their type system, leaving much less room for runtime exceptions.
A soundness proof ties together a particular type system with the
statement that it is sound. As such, it is where you tie the knot
between type checking (which happens at the syntactic level) and
execution (dealing with runtime values). These are two things that are
usually separate — we’ve seen throughout the course many examples for
things that could be done only at runtime, and things that should happen
completely on the syntax. eval
is the important semantic function
that connects the two worlds (compile
also did this, when we converted
our evaluator to a compiler) — and in here, it is the soundness proof
that makes the connection.
To demonstrate the kind of differences between the two sides, consider
an if
expression — when it is executed, only one branch is
evaluated, and the other is irrelevant, but when we check its type,
both sides need to be verified. The same goes for a function whose
execution get stuck in an infinite loop: the type checker will not get
into a loop since it is not executing the code, only scans the (finite)
syntax.
The bottom line here is that type soundness is really a claim that the type system provides some guarantees about the runtime behavior of programs, and its proof demonstrates that these guarantees do hold. A fundamental problem with the type system of C and C++ is that it is not sound: these languages have a type system, but it does not provide such runtime guarantees. (In fact, C is even worse in that it really has two type systems: there is the system that C programmers usually interact with, which has a conventional set of type — including even higher-order function types; and there is the machine-level type system, which only talks about various bit lengths of data. For example, using “%s” in a printf() format string will blindly copy characters from the address pointed to by the argument until it reaches a 0 character — even if the actual argument is really a floating point number or a function.)
Note that people often talk about “strongly typed languages”. This term is often meaningless in that different people take it to mean different things: it is sometimes used for a language that “has a static type checker”, or a language that “has a non-trivial type checker”, and sometimes it means that a language has a sound type system. For most people, however, it means some vague idea like “a language like C or Pascal or Java” rather than some concrete definition.