Lectures onType Theory
Chapter 128
Chapter 128Optional

Supercompilation, Driving, and Generalization

Prerequisites. Direct starred prerequisites: Chapter 127. No later core chapter depends on this route.

A specializer can stop an unfolding computation after a fixed number of steps. The number explains neither why the computation is growing nor which earlier state it resembles. In grow(x)=grow(suc(x)), no two calls are equal, so memoization also misses the obstruction. A supercompiler records a process tree, detects structural growth, and replaces the growing state by a reusable generalization.

Driving constructs a process tree

We use the pure call-by-value language of Jónsson and Nordlander. Fix a finite constructor signature with finite arities and a finite set of primitive operations.

Definition 128.1 — SC-CBV state space

Terms are e::=nxge0e1λx.ek(e)e1e2case e of{piei}ilet x=e0 in e1letrec g=v in e,p::=nk(x). The letrec form is source notation, not a seventh core contraction. With a distinguished global fix satisfying G(fix)=λf.f(λn.fix f n), define letrec h=λx.e in e:=(λh.e)(λy.fix(λh.λx.e)y), where x=(x1,,xn), the only free variable of λx.e may be h, and y satisfies yfv(e)fv(e){h,x1,,xn}. This is the exact macro expansion used here; evaluation and residualization expand it before using the core rules. Values are v::=nλx.ek(v). The finite map G supplies each global g’s closed value. Concrete evaluation uses contexts E::=[]Ee(λx.e)Ek(v,E,e)EenEcase E of{piei}let x=E in e and, after expanding letrec, the six left-to-right call-by-value contractions. For a fixed branch list B={piei}i, write C[u;B]=case u ofB. EgcbvEv(G(g)=v),E(λx.e)vcbvEe[v/x],Elet x=v in ecbvEe[v/x],EC[kj(v);B]cbvEej[v/xj],EC[nj;B]cbvEej,En1n2cbvEprim(n1,n2). A configuration is a pair Re, where the one-hole strict context R::=[]Recase R of{piei}iReeR records work that must finish before its focus. A process edge is labelled by a substitution and is not a concrete evaluation step.

The full driver is the ordered partial function DR,G,ρ(e). Here Re plugs e into R, a ranges over obstructed expressions, and ρ records residual names paired with earlier calls and satisfying the allocation condition stated with A4a–A4c. More precisely, an entry for a configuration Q, whose ordered free-variable vector is x, has the form ρ(h)=λx.Q. We abbreviate that assertion by (h,Q)ρ. Whenever fv(Q) occurs in an argument position, it denotes this fixed ordered vector rather than an unordered set. For the branch list B={piei}i, put BxR={piD[]((Rei)[pi/x])}i,BR={piD[](Rei)}i. The complete ordered rule ledger is: R1R3:DR(n)=Rn,DR(x)=Rx,DR(g)=DRapp(g);R4R6:D[](k(e))=k(D[](e)),DR(xe)=RxD[](e),D[](λx.e)=λx.D[](e);R7:DR(n1n2)=D[](Rn),n=prim(n1,n2);R8:DR(e1e2)={D[](e1)D[](e2),e1e2=a,DRe1[](e2),e1=n or a,DR[]e2(e1),otherwise; R9R10:DR((λx.f)e)=DR(let x=e in f),DR(ee)=DR[]e(e);R11R12:DR(let x=n in f)=D[](Rf[n/x]),DR(let x=y in f)=D[](Rf[y/x]), where the last equation requires that y was not introduced by a preceding split. For rule R13, let L=let x=e in f and Q=xstrict(f)linear(f). R13:DR(L)={D[](Rf[e/x]),Q,let x=D[](e) inD[](Rf),otherwise; R14:DR(letrec g=v in e)=D[],G,ρ(Re),G=G{gv}; R15R17:DR(C[x;B])=C[x;BxR],DR(C[kj(e);B])=D[](Rlet xj=e in ej),DR(C[nj;B])=D[](Rej);R18R20:DR(C[a;B])=C[D[](a);BR],DR(C[e;B])=DRcase(e),DR(e)=Re. Here Rcase=RC[[];B] in rule R19. Rules are tried in numerical order. The obstructed grammar is a::=xnaanaaae. The strict-variable analysis is the exact syntax-directed function strict(x)={x},strict(n)=strict(g)=,strict(k(e))=istrict(ei),strict(λx.e)=,strict(fe)=strict(f)strict(e),strict(let x=e in f)=strict(e)(strict(f){x}),strict(letrec g=v in f)=strict(f),strict(e1e2)=strict(e1)strict(e2),strict(ecase)=strict(e)i(strict(ei)fv(pi)), where ecase=case e of{piei}. A term is linear in x when x occurs at most once, except that distinct case branches may each contain x; the case scrutinee and any branch may not both contain it. Let linear(f) collect such variables. Rule R13 is the only rule that spends both strictness and linearity. Removing either side condition can move or duplicate a diverging computation across a call-by-value demand.

The application driver must retain all four alternatives. Put g^=Rg, and write AR,G,ρ(g) for DRapp(g). The alternatives are tried in order: A1:AR,G,ρ(g)=hxif (h,e1)ρ and σe1=g^;x=σ(fv(e1));A2:AR,G,ρ(g)=g^if (h,e1)ρ, e1g^e1;A3:AR,G,ρ(g)=[D[](f)/y]D[](fg)if (h,e1)ρ and e1g^;(fg,f,y)=split(g^,e1). If none applies, choose a residual name h such that hfv(g^)fn(g^)dom(ρ)qrng(ρ)fn(q)vrng(G)fn(v), and set (g,v)G,ρ=ρ{hλx.g^},x=fv(g^),e=D[],G,ρ(Rv). and use the ordered subalternatives. Here Sub(e) is the finite set of strict proper subexpressions of e. A4a:AR,G,ρ(g)=[D[](f)/y]D[](fg)if a selected e1Sub(e) exists;e1g^ and g^e1,and (fg,f,y)=split(g^,e1);A4b:AR,G,ρ(g)=letrec h=λx.e in hxif hfn(e);A4c:AR,G,ρ(g)=eotherwise. Alternative A1 is the only fold. Alternative A2 stops at mutual embedding. Alternative A3 is downward generalization; A2 followed on a later visit by A4a supplies upward generalization. Rule A4b closes a residual recursive binding exactly when the name allocated before unfolding occurs in the driven body. Dropping that preallocation loses recursive back edges. These are the application rules of the SC-CBV algorithm used throughout the remainder of the chapter.

The negative premise in A4a is a progress guard: it requires the selected subexpression to embed the call strictly rather than mutually. Syntactic proper-subexpression status alone does not imply this condition, because embedding forgets variable identities. For pairwise distinct variables x,y,z, the calls g(x,x) and g(y,z) mutually embed. Their equal-head split may have general spine g(zy,x,zz,x), where the two variables are chosen by the fixed enumeration of Z(g(y,z),g(x,x)). This spine has exactly the weight of g(y,z). Without the guard, the recursive call on that spine need not decrease. With the guard, such a candidate falls through to A4b or A4c; clause adequacy is unchanged, while the strict-descent theorem below becomes valid.

For the first-order process-tree drawings, we use the partial projection drive0(Q)={(θj,Qj)}j. Its domain contains a call-by-value beta redex, a constructor case, or an open-variable case at the focus. It is not the full SC-CBV driver: application focus, constructors with nonvalue fields, globals, primitives, let, and expanded letrec remain governed by rules R1–R20 above. The local projection has exactly the following rules.

drive0(R(λx.e)v)={([v/x],Re)}
D-Beta

For the branch family B={ci(xi)ei}iI, put θj=[cj(zj)/y] and Qj=Rθjcase cj(zj) of B. The two case rules are

jI
drive0(Rcase cj(v) of B)={([v/xj],Rej)}
D-Case-Known
y is freejI
drive0(Rcase y of B)={(θj,Qj)}jI
D-Case-Open

Compatibility is rule specific. For a beta edge, δ agrees with a closing value substitution σ on the free variables of the parent and maps the discharged binder x to vσ. For a known-case edge it instead maps every branch binder xi to the corresponding component of vσ. For an open-case edge j, δ agrees with σ away from y, maps zj to the fields of σ(y)=cj(w), and satisfies δ(zj)=w. The edge observation is {QσcbvQδ,for D-Beta and D-Case-Known,Qσ=Qδ,for D-Case-Open. The first line is a concrete contraction, not syntactic equality. In the open-case line, Q retains the now-known case expression: the edge only refines y’s constructor shape. A subsequent D-Case-Known edge performs the contraction. This separation is what makes the equality true.

Driving case y of{NilNil;Cons(a,z)Cons(a,z)} creates two children. Their edge substitutions describe all and only outer constructor shapes that the unknown y may acquire.

Theorem 128.2 — Local driving coverage

If drive0(Q)={(θj,Qj)}jJ, and every open case reached by the displayed derivation has an exhaustive, disjoint constructor branch family for the closed value supplied by σ, then every substitution σ mapping the free variables of Q to closed values makes Qσ closed and selects exactly one compatible child. Conversely, each compatible child denotes the selected call-by-value behavior of Qσ.

Proof of Theorem 128.2 — Local driving coverage

Proof. Proceed by the last driving rule. In D-Beta, substitution is the call-by-value beta step. D-Case-Known selects the constructor’s unique branch. In D-Case-Open, exhaustiveness gives an outer constructor shape for σ(y), and disjointness makes it unique; its fields define the unique compatible extension. Applying that extension to the refined child gives the uncontracted parent instance literally. A subsequent node uses D-Case-Known. These are the three rule families of drive0. No coverage claim is made for the full grammar; R1–R20 supply that separate driver. ◻

Exercise 128.1

★☆☆ Suppose append recurses on its first argument. Drive the first two nodes of append(x,Cons(b,Nil)). Give both substitutions at the open case and the residual expression at each leaf.

A whistle detects growth

The whistle is the test that reports when an earlier configuration embeds in the configuration being driven. The name denotes only an alarm for structural growth: it proves neither semantic equivalence nor that the later configuration may be folded to the earlier one.

Two configurations are variants, written QαQ, when a bijective renaming of free variables maps one to the other. A later variant folds to the earlier node and supplies a residual recursive call.

Lemma 128.3 — Folding equation

If Q=Qπ for a variable renaming π, replacing the subtree rooted at Q by a call to the residual function for Q, with arguments π, preserves the returned value of every closing instance. The residual named call is accounted for separately by the improvement relation of definition 128.6; this lemma asserts no equality of named-call counts.

Proof of Lemma 128.3 — Folding equation

Proof. The residual equation for Q abstracts precisely its free variables. Instantiation by π alpha-renames that equation to the one required at Q. Unfolding the residual call exposes that instantiated body. Induction on a finite evaluation derivation then matches every source contraction with the corresponding contraction of the unfolded equation; alpha-renaming preserves the selected rule and substitution. Both closed instances therefore return the same value. Named residual calls are not counted by this local argument. ◻

Folding does not catch x,S(x),S(S(x)),. The homeomorphic embedding relation on constructor trees is generated by

xy
Emb-Var
n1n2
Emb-Num
sti
sc(t1,,tk)
Emb-Dive
siti(1ik)
c(s1,,sk)c(t1,,tk)
Emb-Couple

Variables embed variables and numerals embed numerals irrespective of their names or values; distinct constructor heads do not couple. Encode a configuration by a tree with a distinguished configuration-root label and ordered children for its context and focus, and use the displayed relation on that tree. Thus xS(x), but S(x)Z.

Theorem 128.4 — Whistle property

Fix a finite constructor signature and a finite global map G. Every infinite sequence of finite configurations whose global names lie in dom(G) contains i<j with QiQj. Consequently an infinite process-tree branch contains a later configuration that raises the embedding whistle.

Proof of Theorem 128.4 — Whistle property

Proof. Let X be a set with a reflexive, transitive relation X. The pair (X,X) is a well-quasi-order when every infinite sequence x0,x1, contains indices i<j with xiXxj. We import the following combinatorial theorem. If (X,X) is a well-quasi-order, then its finite rooted labelled trees are well-quasi-ordered by homeomorphic embedding, where coupling compares root labels by X and corresponding children, and diving compares a tree with a descendant. Kruskal’s proof of this statement occupies physical pages 5–15, printed pages 214–224 .

Let L contain the configuration-root label, the finitely many term and context tags, one label for every member of dom(G), one numeral label, and one variable label. Equality makes the finite set L a well-quasi-order: an infinite label sequence repeats a label by the infinite pigeonhole principle. Encode Q as the ordered L-labelled tree described before the theorem. A structural induction on the imported tree-embedding derivation proves E(Q)E(Q)QQ. The coupling case is Emb-Couple; the descendant case is Emb-Dive; and the two collapsed leaf cases are Emb-Var and Emb-Num. Kruskal therefore supplies i<j with E(Qi)E(Qj), and the displayed implication supplies QiQj. An infinite process-tree branch is such a sequence, so its later member raises the whistle against the earlier member. No termination theorem for a program transformer is imported here.

The finite-label hypothesis cannot be removed. For pairwise distinct nullary labels 0,1,, the one-node trees 0,1, form an infinite sequence with no embedded pair, because neither diving nor equal-root coupling applies. ◻

Exercise 128.2

★☆☆ Derive Cons(x,Nil)Cons(a,Cons(x,Nil)). Show that reversing the judgment fails by identifying the first rule that cannot apply.

Generalization answers the whistle

Stopping at a whistle residualizes the whole growing state. Instead compute a most-specific generalization, the least general tree having both states as substitution instances, in a first-order term algebra. Let TΣ be the finite trees whose internal nodes are the constructors, term-former tags, and context-former tags of the SC-CBV signature. Before encoding a configuration, alpha-rename every binder apart, replace its bound occurrences by de Bruijn indices, and erase its source name from the binder node. Free source variables remain first-order leaves. Substitutions below act only on those free-variable leaves. They are not capture-avoiding substitutions on raw source terms.

For s,tTΣ, the operation msg(s,t)=(g,σ,τ) satisfies gσ=s and gτ=t. Fix an infinite variable set Z(s,t) disjoint from fv(s)fv(t). Equal heads are generalized recursively. At the first unequal pair (u,v), choose the first unused zu,vZ(s,t), store (u,v)zu,v, and reuse zu,v at every repeated disagreement pair. For example, with zZ(suc(x),suc(suc(x))), msg(suc(x),suc(suc(x)))=(suc(z),[x/z],[suc(x)/z]).

Write a uniform term as s(e). If msg(s(e1),s(e2))=(tg,θ1,θ2), the exact split alternatives are split(s(e1),s(e2))={(tg,rng(θ1),dom(θ1)),s=s,(s(x),e1,x),ss, where the components of x are pairwise distinct and lie outside fv(s(e1))fv(s(e2)). The first alternative preserves a nontrivial common spine; the second splits the first term along its spine when root disagreement would make the msg a single variable. The application-driver alternatives use this operation for upward and downward generalization. Rule R13’s strictness condition prevents the reassembly from moving a possibly divergent argument out of a call-by-value position.

The termination argument needs a weight that distinguishes a variable introduced by splitting from the expression it replaces. Give every numeral, global or residual name, bound-variable occurrence, and source-variable occurrence weight two. Give every variable introduced by split weight one. For every other syntax former s, including a vector let treated as one former, put wt(s(t1,,tk))=1+i=1kwt(ti). Alpha-renaming preserves the source-or-split tag. The variable chosen by split has weight one, so replacing a weight-two atom or a composite by that variable strictly decreases weight. Every proper subexpression has smaller weight than its parent. Let κ(t) be the number of case-expression nodes in t. A pattern contains no case node, so applying a constructor-pattern substitution cannot increase κ.

Lemma 128.5 — Generalization witnesses

For s,tTΣ, msg(s,t)=(g,σ,τ) terminates, satisfies gσ=s and gτ=t, and is most specific: if hσ=s and hτ=t, then hδ=g for some δ, up to bijective renaming of the variables chosen from Z(s,t). Moreover, every split selected by A3 or A4a satisfies split(s,t)=(fg,f,y),wt(fg)<wt(s),fif. wt(fi)<wt(s).

Proof of Lemma 128.5 — Generalization witnesses

Proof. Induct on the ordinary node count of s plus that of t, with the finite disagreement table threaded through the recursive calls. A table hit returns its stored variable without a recursive call. Suppose s=c(s1,,sk) and t=c(t1,,tk). For each integer i with 1ik, process the children from left to right. The induction hypothesis gives msg(si,ti)=(gi,σi,τi) at the table produced by the preceding children. Threading makes σi and σj, and likewise τi and τj, agree on every shared domain variable. Hence the following unions are functions: g=c(g1,,gk),σ=i=1kσi,τ=i=1kτi. Then gσ=s and gτ=t by the induction equations for every child. If a common generalization has a variable at its root, map that variable to g. Otherwise its root must be c; it restricts to a common generalization at each child, so the induction factorizations combine through the same root. This treats every equal-head arity.

For unequal heads the chosen variable zu,v with two singleton substitutions has the required equations. Any common generalization must place a variable at that disagreement, so mapping it to zu,v defines δ. Memoized disagreement pairs force the same choice at repeated positions and preserve the factorization. Equal heads recurse only on strict child pairs, and unequal heads or table hits stop; hence the node-count induction also proves termination.

It remains to prove the weight assertion. Every selected split has ts and st: in A3 the first relation is its side condition and the second follows because the earlier A2 test failed; in A4a they are the two displayed side conditions. In the equal-head arm of split, the generalization keeps the common root. Every range member of its first witness substitution is a proper subexpression of s. If every disagreement on the s-side were a weight-one split variable, then the corresponding t-side subterm would also be a variable: no composite tree embeds a variable leaf. Coupling the common nodes and applying Emb-Var at those disagreements would derive st, contrary to strictness. Hence at least one disagreement replaces a weight-two atom or a composite subexpression by a weight-one split variable. Consequently both the generalization and every range member have weight strictly below wt(s).

In the unequal-head arm, s=s0(e1,,ek) is a selected application configuration and therefore has at least one immediate component. The configuration encoding exposes the selected global atom in one such component. The result s0(y1,,yk) replaces every immediate component by a weight-one split variable, while the component containing the global atom has weight at least two. Hence the result has smaller weight, and every ei is a proper subexpression of s. These are exactly the two split arms used by A3 and A4a. ◻

Aggressively folding case x of to an ancestor whose scrutinee is S(x) discards the zero branch. The variant test rejects the fold; embedding raises a whistle; generalization retains the constructor distinction. Embedding is therefore a trigger, not a folding criterion.

Definition 128.6 — SC-CBV improvement

Let H range over one-hole SC-CBV term contexts, let k,k range over N, and let v,v range over closed values. The cost comparison counts only calls of named recursive functions.

Write ecallskv when closed call-by-value evaluation returns v after exactly k such calls; beta, constructor, primitive, let, and case contractions are not counted. Say that e terminates when ecallskv for some k,v. Write closes(H;e,e) when both He and He are closed. For open terms e,e, an operational approximation preserves termination in every closing context, an improvement also does not increase named-call cost, and a strong improvement combines improvement with termination equivalence. Precisely, eopeH. closes(H;e,e)  k,v. Hecallskvk,v. Hecallskv,eimpeH,k,v. closes(H;e,e)  Hecallskvkk,v. Hecallskv,eseeimpe  eope eope. The relation cost equivalence, written ecoste, holds when both eimpe and eimpe. Cost equivalence implies strong improvement in either orientation. The result v is existential because the observation is termination in every closing context; a context that distinguishes observable constructors supplies the corresponding result test. Thus s is Sands’s strong-improvement preorder, not the homeomorphic-embedding relation .

Proof structure.

The contextual quantifiers in the definition provide the nonrecursive algebra, but replacing a recursive body must account for every dynamically reached call. Lemma 128.7 proves the contextual algebra. Lemma 128.8 lifts one body improvement through every recursive call, and lemma 128.9 relates a memo entry whose name satisfies the allocation condition to the corresponding local recursive binding. The clause lemma then proves one complete driver step. The descent lemma supplies the well-founded induction that assembles those local steps in theorem 128.14.

Lemma 128.7 — Local strong-improvement laws

Let e,e,e, and b be SC-CBV terms. Let C be a finite multi-hole term context, let π be a capture-avoiding bijective renaming, let h be a residual function name, and let x be a finite vector of pairwise distinct variables.

  1. Strong improvement is reflexive and transitive.

  2. If ese, then C[e]sC[e], with the replacement made in any fixed collection of holes.

  3. ecosteπ.

  4. If ecbve, then ese. The same conclusion holds for the multi-argument beta-to-let and known-case-to-let equations in the driver.

  5. If hfn(e), then ecostletrec h=λx.b in e.

Proof of Lemma 128.7 — Local strong-improvement laws

Proof. For reflexivity, reuse the given evaluation derivation with the same call count. For transitivity, compose the two existential evaluation witnesses; the inequalities compose by transitivity of , and the two directions of operational approximation compose separately.

For congruence, fix a closing observation context H. Replacing the selected holes turns HC[] into another closing context for the original pair. Apply the assumed improvement and both assumed termination implications to that composite context. This proves all three conjuncts of strong improvement. Repeating the one-hole argument proves the finite multi-hole form.

Renaming transports an evaluation derivation rule by rule. Binders are renamed before substitution, constructor and global tags are unchanged, and the named-call count is unchanged. Applying the inverse renaming transports the derivation back, which proves cost equivalence.

For a contraction ecbve, splice the contraction in front of any terminating derivation for the contractum. Conversely, invert the unique first step of a terminating derivation for the redex. The two derivations have equal named-call cost except at a global unfolding, where the redex derivation has one additional named call. Thus the contractum never uses more calls, and termination is equivalent. Multi-argument beta and the known-constructor case evaluate the same arguments from left to right and then reach the same simultaneous-substitution instance; induction on the argument vector gives the stated equations.

Finally expand letrec. Because h is absent from e, the outer beta substitution changes no occurrence of e. The administrative beta and fix contractions are not named calls, so the two evaluation derivations have equal named-call cost in both directions. ◻

Lemma 128.8 — Local recursive replacement

Let h be a residual function name, let x=(x1,,xn) be pairwise distinct variables, and let b0,b1 be SC-CBV terms satisfying fv(b0)fv(b1){h,x1,,xn}. Put Li(t)=letrec h=λx.bi in t for i{0,1}. If L0(b0)sL0(b1), then L0(t)sL1(t) for every term t.

Proof of Lemma 128.8 — Local recursive replacement

Proof. The decisive case is a dynamic call to h: changing only its first unfolding leaves recursive calls under the old binding. We therefore induct on the named-call cost of a finite evaluation derivation, with derivation height as a secondary measure.

Fix a closing observation context H. For the improvement direction, induct lexicographically on (k,d), where k is the named-call cost of a derivation of HL0(t) and d is its height. Copy a rule that is not a call to h. A premise of equal call cost has smaller height; a premise below any named call has smaller call cost. The induction hypotheses therefore transform every such premise.

At a call hv, the call rule spends one named call and evaluates an instance b0[v/x] under L0. Apply the assumed body improvement in the closing context consisting of the actual arguments, the evaluation prefix, and the continuation. It gives a derivation of b1[v/x] under L0 with no larger cost. That cost is strictly smaller than the cost of the original call node, because the call node itself contributed one. The outer induction transforms the supplied derivation from binding L0 to binding L1. Reattach the call rule, prefix, and continuation. Summing premise costs shows that the complete transformed derivation uses no more named calls. This proves improvement and forward operational approximation.

For reverse operational approximation, start with a terminating derivation for L1(t) in the closing context H, and use the same lexicographic measure. At a call to h, first apply the induction hypothesis to its body derivation; the call node contributes one, so that body has smaller cost. This gives a terminating instance of b1 under L0. The reverse operational-approximation conjunct of the assumed body relation then gives a terminating instance of b0 under L0. Reattach the call rule. The noncall cases copy their final rule as in the forward construction. Thus termination is preserved in the reverse direction. The improvement and two approximation results are the three conjuncts of strong improvement. ◻

Lemma 128.9 — Allocation and call bridge

Let u,u1,q be SC-CBV terms, let h be a residual function name, and let ρ be a finite closing memo environment. Let G be a finite global map, let R be a strict context, let g be a global name, and let v be a closed value with G(g)=v. Suppose u=Rg,u1=Rv, so that ucbvu1 is one named global unfolding. Suppose also that x is the ordered list fv(u), hfv(u)fn(u)fv(u1)fn(u1)dom(ρ), and every free occurrence of h in q is a saturated call hπ(x) for a bijective renaming π of x. Let ρ extend ρ by mapping h to λx.u. Then ρ(q)costletrec h=λx.u1 inρ(q),ucostletrec h=λx.u1 in hx.

Proof of Lemma 128.9 — Allocation and call bridge

Proof. For the second equation, the displayed disjointness condition for h first inserts the unused binding around u. Inside that binding, the left program performs the named global unfolding ucbvu1; the right program performs one named call to h and then beta-reduces to the same u1. Thus every closing evaluation has a mate with the same result and named-call count, in both directions.

For the first equation, enumerate the finitely many free occurrences of h in q. Closing by ρ replaces the occurrence hπ(x) by (λx.u)π(x), which beta-reduces to uπ. Under the displayed letrec, the corresponding call to h unfolds to u1π. The first part of the argument, renamed by π, gives a cost-equivalent replacement at that occurrence. Apply finite-hole congruence from lemma 128.7 to all occurrences. All other variables are closed by the common map ρ, so the resulting terms are exactly the two terms displayed in the statement. ◻

For a residual term q, write ρ(q) for the simultaneous residual-name closing operation: each name h in the finite domain of ρ is replaced by the closed abstraction ρ(h). The allocation condition beside A4a–A4c makes the simultaneous operation capture avoiding; it does not recursively reapply ρ inside its own range.

Lemma 128.10 — Clause adequacy for the full driver

Fix an instance I of one selected equation among R1–R20 or A1–A4c with outer state (RI,GI,ρI,eI). Put LI=RIeI, and write PI[q1,,qm] for its right-hand side with the m recursive driver calls replaced by holes. The j-th recursive call has its own state (RI,j,GI,j,ρI,j,eI,j) and input configuration LI,j=RI,jeI,j. Suppose the range of every ρI,j is closed, (fv(LI,j)fn(LI,j))dom(ρI,j)=, and LI,jsρI,j(qj) for every integer j with 1jm. Suppose also that the range of ρI is closed and (fv(LI)fn(LI))dom(ρI)=. Then LIsρI(PI[q1,,qm]). Here m=0 is permitted. The selected clause includes every side condition printed in its rule, including the strict–linear condition of R13, the split witnesses of A3 and A4a, and the occurrence test of A4b.

Proof of Lemma 128.10 — Clause adequacy for the full driver

Proof. Apply the congruence and transitivity parts of lemma 128.7 to the state-indexed recursive hypotheses. It remains to check the noncongruence step selected by each rule family. All observations below use the named-call cost of definition 128.6; no step-counting or value-equality surrogate is introduced.

Rules R1–R6 and R20 only plug a value, variable, constructor, lambda, or obstructed expression into the recorded context. Rule R7 performs the primitive contraction. Rules R8, R10, R18, and R19 expose exactly the demanded call-by-value evaluation context. Rule R9 replaces multi-argument beta reduction by call-by-value lets; both forms first evaluate the arguments from left to right and then evaluate the same simultaneous substitution instance. Rules R11, R12, R16, and R17 perform the displayed let or case contraction. Rule R15 refines the scrutinee constructor and binds precisely the pattern fields selected by that constructor.

For the substitution arm of R13, strictness of x in f ensures that evaluation of e is demanded before the result of f is returned, and linearity ensures that the substitution creates at most one demand for e. Hence it neither removes nor duplicates a possibly diverging call-by-value computation. The other arm leaves the call-by-value let in the residual term. Rule R14 changes the global environment by the displayed recursive value and then drives the body; unfolding that global has exactly the displayed letrec equation.

For A1, suppose ρ(h)=λx.Q and g^=Qπ. Closing the emitted fold gives ρ(hπ(x))=(λx.Q)π(x)cbvQπ=g^. The beta contraction costs no named call, and alpha-renaming preserves every later call count by lemma 128.7; hence the fold is cost equivalent to the selected configuration. Rule A2 is the identity residualization at mutual embedding. In A3 and A4a, the equations fg[f/y]=g^ supplied by lemma 128.5 and the definition of split reassemble the selected configuration; the recursive hypotheses then apply to D[](fg) and every D[](fi). The recursive case A4b spends the state indices. Write g^=Rg, choose G(g)=v, put x=fv(g^), and let ρ=ρ{hλx.g^},q=D[],G,ρ(Rv). The allocation condition for h and abstraction of the complete ordered free-variable list make the new range entry closed. The recursive hypothesis is therefore the state-correct assertion Rvsρ(q), not the ill-scoped assertion with ρ(q). Put u=Rg, u1=Rv, and L0(t)=letrec h=λx.u1 in t,L1(t)=letrec h=λx.ρ(q) in t. The global rule in the evaluation context gives ucbvu1. Every occurrence of the residual name h emitted while driving q is a saturated call at a renaming of the ordered vector x. The first equation of lemma 128.9 therefore gives ρ(q)sL0(ρ(q)). The unused-binding equation and the recursive hypothesis give the annotated body chain L0(u1)sunused bindingu1srecursive hypothesisρ(q)sallocation bridgeL0(ρ(q)). This is the body premise of lemma 128.8. That lemma yields L0(hx)sL1(hx). The second equation of lemma 128.9 gives ucostL0(hx). Compose this equivalence with the recursive-replacement conclusion. The resulting left side is u=g^, and its right side is exactly the A4b residual under the outer closing map ρ.

In A4c, hfn(e), so removing the unused allocation changes neither evaluation nor named-call cost. These cases exhaust the ordered ledger. Every use of a recursive result is indexed by its own (RI,j,GI,j,ρI,j), and the only change of closing map is the allocation of h treated above. The displayed state-indexed clause follows. ◻

Definition 128.11 — Admissible driver state

An initial state DR,G,ρ(e) is admissible when the constructor and global signatures are finite; constructor arities agree at every occurrence; every case family is finite, exhaustive, and has disjoint patterns; and every global in the transitive syntactic call closure of Re, G, and the range of ρ has one closed value in G. Binders are pairwise distinct after alpha-renaming. Fix an enumeration of residual function names. At a state (R,G,ρ,e), the allocation policy chooses the first name h in that enumeration satisfying hfv(Re)fn(Re)dom(ρ)qrng(ρ)fn(q)vrng(G)fn(v). Every entry of the finite map ρ has the form hλx.Q, where x is the ordered vector fv(Q), and every such abstraction is closed. The selector used by A4a is a fixed deterministic selector from the finite set of strict proper subexpressions e1 satisfying e1g^ and g^e1. The functions strict, linear, msg, and split are exactly the finite syntax-directed functions displayed in this chapter.

Lemma 128.12 — Ledger totality and functionality

Exactly one ordered driver clause applies to every state reachable from an admissible initial state.

Proof of Lemma 128.12 — Ledger totality and functionality

Proof. Inspect the outer form of the focus. Numerals, variables, globals, constructors, lambdas, primitives, applications, lets, letrecs, and cases are the disjoint outer forms in definition 128.1. Within one outer form, the side conditions printed in the ordered ledger are tested in order; the final arm of R8, the second arm of R13, and R20 cover failure of all earlier tests. A reachable global has exactly one value in G by admissibility. For its application ledger, A1–A3 are tested in order; if none applies, the deterministic finite selector decides A4a, and the decidable occurrence test then distinguishes A4b from A4c. Thus at least one clause applies. Ordered selection prevents two clauses from being selected. ◻

Lemma 128.13 — Strict descent of recursive driver calls

For an admissible initial state, there is a lexicographic measure W:reachable driver statesN4 such that every recursive driver call on the right side of a selected R1–R20 or A1–A4c clause has smaller measure than its parent.

Proof of Lemma 128.13 — Strict descent of recursive driver calls

Proof. Fix the admissible initial state S0=DR0,G0,ρ0(e0). Consider the tree of memo-table extensions reachable from S0 without an earlier memoized configuration embedding a later one. It is finitely branching: a term has finitely many subterms and branches, the signatures are finite, and the A4a selector is fixed. It has no infinite branch by theorem 128.4. König’s lemma therefore makes this tree finite. For completeness, if it were infinite, its finite root degree would leave one child with infinitely many descendants; repeating that choice would construct an infinite branch. Let N(S0) be the finite tree’s maximum extension depth.

For a reachable state S=DR,G,ρ(e), put dS0(ρ)=|dom(ρ)dom(ρ0)|. Define WS0(S)=(N(S0)dS0(ρ), κ(Re), wt(Re), wt(e)). Reachability gives dS0(ρ)N(S0), so this is a quadruple of natural numbers.

We verify the rule families. Rules R1–R3 and R20 either stop or delegate to the application ledger. Rules R4–R6 recurse on proper subterms. Their case count cannot increase, and their plugged weight decreases when the case counts agree. Rule R7 replaces a primitive node by its weight-two numeral result. In the first arm of R8, both operands are proper subterms; in its other arms the plugged configuration is unchanged and the focused weight decreases. Rule R9 removes one application-or-lambda node when it changes beta application into the single vector-let former. Rule R10 preserves the plugged application and focuses on its proper head. Rules R11 and R12 replace a weight-two bound-variable occurrence by a weight-two numeral or source variable and remove the positive weight of the let and its right-hand side. In the substitution arm of R13, strictness supplies at least one occurrence of x and linearity supplies at most one. There is therefore exactly one occurrence: substitution adds one copy of e, removes the weight-two occurrence of x, and removes the let former and its separate right-hand side. In the other arm, each of the two recursive calls is on a proper component. Rule R14 removes the letrec former and its stored value from the driven term.

Rules R15–R17 remove the selected case node. Pattern substitution can duplicate constructor syntax but cannot introduce a case node, so the second component strictly decreases. In R18, every recursive input is a proper scrutinee or branch component of the selected case; its case count is strictly smaller than that of the whole case. Rule R19 preserves the plugged case, its case count, and its plugged weight, but decreases the focused fourth component.

Alternatives A1 and A2 make no recursive call. In A3, unequal-head splitting returns proper spine components; equal-head generalization returns a common spine and disagreement ranges that are proper parts of the selected configuration. Alternative A2 has already rejected mutual embedding, so at least one disagreement is strict. The size induction and split-weight assertion in lemma 128.5 therefore decrease the case-count–weight pair for every A3 recursive call: splitting introduces no case node, and its weight strictly decreases when the case count is unchanged.

The common first step of A4a–A4c extends ρ to ρ before driving the unfolded body. Its recursive call therefore decreases the first component regardless of the body’s size. If A4a is then selected, its term e1 is a strict syntactic subterm of that driven body and the progress guard makes e1g^ strict. The split-weight assertion of lemma 128.5 therefore makes every additional A4a call smaller in the case-count–weight pair. Alternatives A4b and A4c make no further recursive call. These are all ordered clauses, and lexicographic order on N4 is well founded. ◻

Theorem 128.14 — SC-CBV termination and correctness

Let DR,G,ρ(e) be an admissible initial driver state such that (fv(Re)fn(Re))dom(ρ)=. Its strictness-guided driving, variant folding, embedding whistle, split and generalization policy, and residualization terminate. Put e=ρ(DR,G,ρ(e)). Then Rese. Consequently, for every H with closes(H;Re,e), (k,v. HRecallskv)(k,v. Hecallskv). Moreover, for every such H, every kN, and every closed value v, HRecallskvkk,v. Hecallskv.

Proof of Theorem 128.14 — SC-CBV termination and correctness

Proof. By lemma 128.12, exactly one ledger clause is selected at each reachable state. Well-founded induction on the measure of lemma 128.13 proves termination: that clause performs finite local work, and every recursive premise has smaller measure.

Use the same induction for correctness and apply lemma 128.10 at its final step. Each recursive call uses its actual (Rj,Gj,ρj); in particular, the A4b premise uses the extended allocation map ρ, while the clause lemma transports its conclusion back to the outer ρ. A variant back edge emits the allocated name and invokes no recursive driver call. Every other clause receives its hypotheses from strict descendants. The clause conclusion at the root is Resρ(DR,G,ρ(e)). Unfolding the three conjuncts of strong improvement gives termination in both directions and the stated bound on named calls. No claim is made for a different evaluator, split policy, or effectful language. ◻

Remark 128.15 — Why the closing names must be disjoint

The disjointness hypothesis cannot be removed. Let h be an ordinary free variable, take R=[] and e=h0, and let ρ contain the residual name h with closed function λx.0. Rule R5 leaves the head variable in place, so closing the residual changes h0 to (λx.0)0. Let Ω be a closed divergent term, and close the source occurrence of h with λx.1 in the observation context H=(λh.case [] of {0Ω;10})(λx.1). Then Hh0 terminates, whereas H(λx.0)0 diverges. Thus the source configuration does not operationally approximate the residual term.

Proposition 128.16 — Residual well-scopedness

Under the hypotheses of theorem 128.14, every residual function abstracts exactly the free variables of its memoized configuration, every fold supplies those variables in that order, and the final residual expression has no free variable outside fv(Re).

Proof of Proposition 128.16 — Residual well-scopedness

Proof. Maintain the invariant that a call DR,G,ρ(e) may emit only variables free in Re, variables in the ordered parameter lists of entries in ρ, and variables bound by the residual syntax surrounding that call. Rules R1–R12 and R20 preserve this set by plugging, congruence, or capture-avoiding substitution. In R13, alpha-renaming the let binder apart makes the substitution arm capture avoiding; the other arm binds x around both residual subterms. Rule R14 alpha-renames g and the parameters of its value apart before extending G, so expansion introduces no free occurrence of either binder. Rules R15–R19 bind every pattern field in its branch and apply the edge substitution to the branch body and recorded context.

For A1, the renaming σ is applied to the ordered list fv(e1), so the fold supplies exactly the parameters of the selected memo entry. Rule A2 introduces no name. In A3 and A4a, the split side condition places y outside the free variables of both inputs; alpha-renaming places the surrounding binders outside y. Moreover, fg[f/y]=g^; substituting the driven witnesses therefore removes every introduced yi. In A4b, the name h satisfies the allocation condition. The recursive binding binds h, and its lambda binds the ordered vector x=fv(g^); every recursive occurrence uses that vector. Rule A4c requires hfn(e), so discarding the allocation leaves no free h. These are all ledger clauses. At the root there is no surrounding residual binder, and applying ρ closes every allocated name. The remaining free variables are therefore contained in fv(Re), as claimed. ◻

The fusion calculation starts from the configuration Q0(xs,ys)=append(map(f,xs),map(f,ys)). Driving the case on xs gives Q0(Nil,ys)=map(f,ys),Q0(Cons(a,as),ys)=Cons(f(a),Q0(as,ys)). The nil branch opens a case on ys. Its cons successor is a variant of Q0(Nil,bs), so folding closes the graph. With residual name h, the complete residual equation is h(xs,ys)=case xs ofNilcase ys of{NilNil,Cons(b,bs)Cons(f(b),h(Nil,bs)),Cons(a,as)Cons(f(a),h(as,ys)). Neither map nor append occurs in this residual program, and each input constructor is inspected once. The fuelled specializer of chapter 127 retains the producer–consumer boundary when both lists are unknown. This comparison concerns these systems and inputs, not a universal ordering.

Sources and seminar

The SC-CBV rule card and examples follow the extended proof report and the later strict-subexpression presentation, except that A4a carries the progress guard stated and justified beside the rule. The printed later rule requires only a strict proper subexpression e1 with e1g^; the variable-sharing example beside the local rule shows why that premise does not establish strict weight descent. The thesis supplies additional examples [Jó08, JN10, JN08]. The only imported proof in this chapter is Kruskal’s tree theorem, at the exact source location recorded in theorem 128.4; the driver descent, strong-improvement algebra, recursive replacement, clause adequacy, and total correctness arguments are proved locally. Turchin’s system, normal-order positive supercompilation, call-by-need variants, and distillation have different observations or control policies and lend no unstated theorem to theorem 128.14.

Suggested first pass.

None of these problems is a prerequisite for a later chapter. Begin with exercise 128.3, then complete exercise 128.5.

Exercise 128.3

★★☆ Build the first five nodes for loop(x)=loop(suc(x)). Mark the first embedding pair, compute its most-specific generalization, and prove both equations.

Exercise 128.4

★★★ Design a folding test that ignores constructor heads. Give a closed input on which it changes the returned constructor. Repair the test with variant equivalence and prove the repaired back-edge equation.

Exercise 128.5 — Practical: whistle visualizer

★★★ Practical project.sc-cbv-whistle-visualizer Implement the finite Kappa constructor-tree model from the tutorial. Emit deterministic fold, whistle, and drive decisions. Maintain the invariant that variant equality is tested before embedding and that every returned generalization witness reconstructs both input trees. Replay the ordered rules R1–R20 and alternatives A1–A4c. The named cases print these exact lines: variant: foldgrowth: whistleunrelated: drivemsg-witness: valid. Change the folding test to ignore constructor heads. Replay the named cases, and explain which premise of theorem 128.14 it violates. The finite model does not certify the full SC-CBV transformer.

Search the book

Type to search the local edition.