Lectures onType Theory
Chapter 29
Chapter 29Optional

Scoped Operations and Explicit Substitution

Prerequisites. Direct starred prerequisites: none. No later core chapter depends on this route.

The transaction from chapter 22 performs a write and may raise an exception. Treat transaction(M) as a scoped operation: effects of M commit on return and roll back on failure, while a computation sequenced after the transaction lies outside the rollback region. Put M:=put(s0+1)=λ_.return(),k:=λ_.raise(e0). The first-order algebraicity equation from the previous chapter would force transaction(M)=k=?transaction(M=k). The two sides do not have the same transactional boundary. On the left, M returns, the write is committed, and only then does k raise outside the transaction; the store visible with the failure is s0+1. On the right, the same raise has been moved into the transaction body, so failure rolls the write back and exposes s0. Equation (23.1) has turned a continuation into part of the scope.

The running example makes the same defect visible without state. Let or be nondeterministic choice and let once select the first solution. With k(x)=or(x,x+1), the intended calculation is once(or(1,5))=k=once1=k=leftunitor(1,2). If once were algebraic, bind would instead be pushed into its argument: once(or(1,5))=k=false alg.once(or(k(1),k(5)))=definition of konce(or(or(1,2),or(5,6)))=once1. The problem is therefore structural, not a missing effect equation. An ordinary algebraic node stores parameters and response branches. A scoped node must additionally say which computation belongs to the scope and which computation follows it. The latter must remain outside when a later bind is performed.

We therefore freeze the syntax and construct its substitution directly.

Exercise 23.1

★☆☆ Let M:=put(s0+1)=λ_.return7,k(x):=return(x+1). Explain why the transactional contract does not distinguish the two programs transaction(M)=kandtransaction(M=k), and then identify the exact feature of k in (23.1) that makes the original counterexample work. Why does agreement on this one successful continuation not restore algebraicity?

The scoped signature

The source calculus separates ordinary algebraic operations from operations that own computations. A general functorial signature has a polynomial presentation that names the parameters and branches used in calculations.

Definition 23.1 — Scoped signature

A scoped signature is a pair (Σ,Γ) of endofunctors on Set. The functor Σ describes ordinary algebraic operations and Γ describes scope creators.

For an elementwise presentation, fix sets O of ordinary operation symbols and S of scoped operation symbols, together with parameter and arity sets Po, Ro(oO),Ps, Qs(sS). The corresponding polynomial functors are ΣX:=oOPo×XRo,ΓX:=sSPs×XQs. Thus Po and Ps are ordinary parameter sets. An ordinary operation has one continuation branch for each response in Ro. A scoped operation owns one scoped computation for each position in Qs.

This polynomial presentation names parameters and branches for nondeterminism, exceptions, local state, and concurrency.

Definition 23.2 — Exact nested syntax

For H:SetSet, put (GH)A:=A+Σ(HA)+Γ(H(HA)). Assume that the transfinite initial chain of this displayed endofunctor G has monic connecting maps and converges to an initial algebra. The scoped syntax endofunctor is the carrier T:=μG. Consequently, for every set A, its constructors have the exact types Var:ATA,Op:Σ(TA)TA,Scope:Γ(T(TA))TA. The functor action of T is written Th:TATB for h:AB. Its constructor clauses are Th(Var(a))=Var(h(a)),Th(Opo(p,k))=Opo(p,λr.Th(k(r))),Th(Scope(v))=Scope(Γ(T(Th))(v)).

The double occurrence T(TA) in the scope constructor is the essential one. A return from a computation inside a scope need not be a final A-value. It may instead be an entire TA-computation representing what should happen after the scope closes. That extra layer is the explicit substitution: the inside return supplies the input to the continuation that follows the scope.

The nested shape is Var a,Op(Σ(TA)),Scope(Γ(T(TA))).

Proposition 23.3 — Existence for the finitely branching running signatures

The nondeterminism signature below has a monic initial chain that converges at ω. The same conclusion holds for the exception signature when the exception set E is finite, and for the local-state signature when the stored value set S is finite. For arbitrary E or S, this proposition makes no ω-convergence claim; the transfinite hypothesis of definition 23.2 remains in force.

Proof of Proposition 23.3 — Existence for the finitely branching running signatures

Proof. Under the stated hypotheses, every response and owned-position set displayed below is finite. Parameter sets may be arbitrary: they occur only as coefficients of sums and products. Hence Σ and Γ are finitary polynomial functors: they preserve injections and colimits of ω-chains of injections. Starting from the empty endofunctor, induction on n shows that every connecting map TnTn+1 is componentwise injective; sums, finite products, finite powers, and the composite Tn(Tn) preserve that property. At the limit, the nested summand uses the diagonal cofinality calculation colimnTn(TnA)colim(m,n)N2Tm(TnA)(colimmTm)((colimnTn)A). For the first isomorphism, every pair (m,n) maps to a diagonal stage (N,N) with Nm,n; for the second, finitarity of each stage moves the inner colimit through Tm, after which the outer colimit is taken. Finitarity of Σ and Γ now gives G(colimnTn)colimnG(Tn), so T=colimnTn is the initial algebra for the once, finite-exception, and finite-local-state signatures. ◻

Three running signatures

For nondeterminism with fail, binary or, and unary once, take ΣX1+X×X,ΓXX. The two summands of Σ are the nullary and binary algebraic nodes; once owns one scoped computation.

For exceptions from a set E, the source gives ΣXE,ΓXX×XE. The ordinary operation is raise: an exception value is a parameter and there is no response branch. A scoped catch node owns one protected computation and one recovery computation for each exception. All those computations return the same intermediate result type; an outside continuation remains separate.

For local state with names N and stored values S, take Pget=N,Rget=S,Pput=N×S,Rput=1, so the ordinary signature and scoped part are ΣXN×XS+(N×S)×X,ΓXN×S×X. A get node chooses one continuation branch from the stored value it receives; a put node carries a name and new value and has one unit-response branch. A local node has two ordinary parameters, a name and its initial value, and exactly one scoped computation. Its interpretation belongs to the source’s later semantic development and is not part of this syntax chapter.

Exercise 23.2

★☆☆ Give polynomial data Po,Ro,Ps,Qs realizing (23.7). Then state the types of the three corresponding elementwise constructors before any smart constructor notation is introduced.

Exercise 23.3

★☆☆ Rewrite (23.8) in polynomial form by taking the scoped-position set to be 1+E. Identify which position is the protected computation and which positions are recovery computations. What uniformity condition on their return type is forced by the single argument X of ΓX?

The explicit substitution hidden in a scope node

The nested constructor is compact but conceals the three pieces that the opening calculation requires. Exposing them does not change the syntax.

Fix a polynomial scoped symbol sS. A scope node returning A may be represented by data p:Ps,X:Set,m:QsTX,k:XTA. We write this representative as Scopes(p;X;m;k). The roles are distinct: componenttyperolepPsordinary parameters, fixed at the operation sitemQsTXcomputations owned by the scopekXTAcontinuation after the scope returns The intermediate set X is not observable syntax. Replacing its names must not create a different node.

Definition 23.4 — Reindexing equation

For h:XY, m:QsTX, and k:YTA, a reindexing changes the intermediate carrier along h while identifying Scopes(p;X;m;kh)=Scopes(p;Y;λq.Th(m(q));k). Equality of elementwise scoped nodes is the congruence generated by these reindexings and ordinary equality of their data. With s,p,A fixed, denote the reindexing class of Scopes(p;X;m;k) by [X,m,k]. These square brackets form a quotient class; they are not the brackets in the explicit-substitution notation t[f].

Equation (23.12) says that an intermediate return can be renamed either before leaving the scope or in the explicit continuation. No handler or categorical representation theorem is being imported.

Lemma 23.5 — Canonical nested representative

For fixed s,p,A, reindexing classes of representatives Scopes(p;X;m;k) are in bijection with families u:QsT(TA). The maps are pack[X,m,k]:=λq.Tk(m(q)),unpack(u):=[TA,u,idTA]. Under the polynomial identification (23.4), these are exactly the data in the s-summand of Γ(T(TA)) from (23.6).

Proof of Lemma 23.5 — Canonical nested representative

Proof. First check that pack respects one generating reindexing. The two representatives in (23.12) map at position q to T(kh)(m(q))=functoriality of TTk(Th(m(q))). Hence pack is well defined on classes.

For u:QsT(TA), pack(unpack(u))(q)=(23.13)T(idTA)(u(q))=functor identityu(q). Conversely, instantiate (23.12) with h=k:XTA and k=idTA: [X,m,k]=(23.12)[TA,λq.Tk(m(q)),idTA]=(23.13)unpack(pack[X,m,k]). Thus the maps are inverse. ◻

This lemma is the bridge between the readable representation (23.10) and the exact nested constructor.

Every stage functor preserves injections, so the least stage containing a representative is a well-founded rank. At a successor stage, unpack the scope and apply the rank hypotheses to its owned computations and continuation.

Lemma 23.6 — Well-founded elementwise recursion

Under the monic and convergent initial-chain hypothesis of definition 23.2, every term for polynomial Σ and Γ has an elementwise representative in which every ordinary branch, every owned computation, and every continuation branch lies at an earlier stage of the initial construction of T. The least such stage is therefore a well-founded rank. Any elementwise definition whose recursive calls are restricted to those displayed components can be constructed by recursion on that rank, provided its scoped clause is invariant under reindexing.

Proof of Lemma 23.6 — Well-founded elementwise recursion

Proof. Use the monic convergent transfinite initial chain of G fixed in definition 23.2. Its first stages are T0A=0,Tα+1A=A+Σ(TαA)+Γ(Tα(TαA)), and a limit stage is the colimit of the preceding stages. The connecting maps are componentwise injective by hypothesis. A transfinite induction shows that every stage functor preserves injections. At a successor, this follows because sums and the polynomial functors Σ,Γ preserve injections and because the nested composite Tα(Tα) does so when Tα does. At a limit λ, the stage is the pointwise colimit of the earlier chain. Suppose classes represented by xTαA and yTβA have equal images under an injection i:AB. Equality in the chain colimit is witnessed at some γ<λ above α and β: the transported elements have equal images under Tγi. Injectivity of Tγi makes the transported representatives equal, hence the original colimit classes are equal. Thus the limit stage also preserves injections. Since the connecting maps are monic, we may identify each stage with its image in the next one. At the convergence ordinal, the colimit is T by the stated construction hypothesis.

Proceed by induction on the least stage containing the term. The same least-stage argument gives the following recursion schema. Given result sets DA and clauses cVar(a):DA,cOp(p,k,(F(k(r)))r):DA,cScope(p,X,m,k,(F(m(q)))q,(F(k(x)))x):DA, with the scoped clause invariant under reindexing, define FA(t) at the least stage of t by the corresponding clause. At a successor stage all displayed recursive arguments lie at an earlier stage; at a limit, the term already has a representative at an earlier stage. The connecting maps are injective, so a term cannot acquire a different constructor when transported to a later stage, and using the unique least stage makes the value independent of a later-stage presentation. Reindexing invariance makes it independent of the chosen elementwise representative. This constructs F; induction on least stage proves that it is the unique family satisfying the three clauses. ◻

Corollary 23.7 — Elementwise induction

Let PA(t) be a family of propositions that respects reindexing. To prove PA(t) for every set A and every t:TA, it suffices to prove variable:PA(Var(a));ordinary:(rRo.PA(k(r)))PA(Opo(p,k));scoped:(qQs.PX(m(q)))(xX.PA(k(x)))PA(Scopes(p;X;m;k)).

Proof of Corollary 23.7 — Elementwise induction

Proof. Use the well-founded rank constructed in lemma 23.6.

Variables and ordinary nodes give the first two induction clauses directly. A scope introduced at stage α+1 has, in the s-summand, parameters p:Ps and a family u:QsTα(TαA). Take X:=TαA, take m(q) to be u(q) followed by the stage inclusion into TX, and take k:XTA to be the stage inclusion. Then every m(q) and every k(x) comes from stage α, so the induction hypotheses give the premises of the scoped clause in (23.14). A term at a limit stage already appears at an earlier stage. Finally, because P respects (23.12), the conclusion is independent of the chosen elementwise representative. ◻

Example 23.8 — The smart constructor for once

For t:TA, the unary scoped operation has the elementwise form once(t):=Scopeonce(;A;λ_.t;Var). Its canonical nested representative has body TVar(t):T(TA), so (23.15) is exactly the source program Scope(Once(fmapreturnt)). The inserted Var is the explicit identity continuation.

Exercise 23.4

★★☆ Let Qs={L,R}, let X={0,1}, and let m(L),m(R):TX and k:XTA be arbitrary. Lemma 23.5 gives the representative [X,m,k]; write its corresponding family in T(TA). Next suppose h:XY and k:YTA satisfy k=kh. Verify directly that [X,m,k] and [Y,λq.Th(m(q)),k] pack to the same family in T(TA).

The failed substitution and the hard repair

For an ordinary algebraic operation, substitution must recurse into every response branch. With elementwise notation, Opo(p,k),k:RoTA, so a substitution f:ATB naturally sends k(r) to k(r)[f]. From this point onward, t[f] is postfix Kleisli substitution: its second argument is a map f:ATB. The notation deliberately resembles ordinary postfix substitution, but the displayed type distinguishes the two uses.

One tempting generic traversal copies that pattern into the owned computation while also preserving the explicitly stored continuation position: Scopes(p;X;m;k)badScopes(p;X;λq.m(q)[f];λx.k(x)[f]). Call this the all-fields traversal. In general, (23.16) is not even typed. Applying the substitution to m(q):TX first requires X=A. Even under that equality, the transformed body has type TB, so the new scope carrier would have to be B, whereas the transformed old continuation is still indexed by x:A. Without a further adapter it has the required domain only when A=B. Thus the clause is typed in the endomorphic homogeneous special case X=A=B; there it duplicates the post-computation by moving it into the scope and retaining it after the scope. This is distinct from false algebraicity, which moves the post-computation into the scoped body and thereby loses it from the outside continuation. For the smart node once(t)=Scopeonce(;A;λ_.t;Var), the three results are correct:once(t;f),false alg.:once(t[f];Var),all-fields traversal:once(t[f];f). The opening calculation exhibits the second result; (23.16) specifies the third. A scoped computation and an outer continuation are different recursive positions.

Definition 23.9 — Explicit substitution

For f:ATB, construct t[f]:TB by the least-stage recursion of lemma 23.6. On a stage-bounded elementwise representative, use the clauses Var(a)[f]:=f(a),Opo(p,k)[f]:=Opo(p,λr.k(r)[f]),Scopes(p;X;m;k)[f]:=Scopes(p;X;m;λx.k(x)[f]). Every recursive call in (23.17) is then on an earlier-stage term; at a limit stage the term already comes from an earlier stage. This transfinite recursion constructs a candidate value from each elementwise representative.

Substitution descends through ordinary branches and through the explicit continuation of a scoped node. It leaves both the ordinary parameters p and the scoped computations m unchanged.

The third line is the repair. It is also the reason explicit substitution was needed: the continuation is present as syntax and can be composed without pretending that it lies inside the scope.

Proof of Lemma 23.10 — Substitution respects reindexing

Proof. For any representative [X,m,k], form the candidate Cf[X,m,k]:=[X,m,λx.k(x)[f]], where the substitutions in the continuation are already defined by the least-stage recursion. We show that Cf is constant on each generating reindexing (23.12). Put kf(y):=k(y)[f]. For the left representative, Cf[X,m,kh]=candidatedefinition[X,m,λx.k(h(x))[f]]=helperdefinition[X,m,kfh]. For the right representative, Cf[Y,λq.Th(m(q)),k]=definition[Y,λq.Th(m(q)),kf]. These two candidates are related by (23.12), now with continuation kf. Hence Cf descends through the generated congruence. On every stage-bounded representative used in definition 23.9, the descended map is exactly the scoped recursive clause in (23.17). Therefore the recursively constructed substitution has the displayed value on every representative and is independent of the chosen representative. ◻

Consequently the candidate is independent of the chosen representative, so equation 23.17 defines substitution on every reindexing class.

The canonical nested clause follows rather than being guessed. Define bf:TATB by bf(u)=u[f]. A canonical representative [TA,u,id] becomes [TA,u,bf] by (23.17). Canonicalizing it with lemma 23.5 gives Scope(v)[f]=Scope(Γ(Tbf)(v)). This is exactly the source implementation Scopesc=f=Scope(fmap(fmap(=f))sc). The inner map traverses the scoped syntax only to reach its returned explicit continuations; it does not move f underneath the scoped boundary as (23.16) does.

Ordinary and genuinely scoped calculations

For nondeterminism, define fail:=Opfail(,absurd0),or(t,u):=Opor(,λb.if b then t else u). Then ordinary substitution is the familiar algebraic calculation: or(Var(1),Var(5))[f]=(23.17)or(Var(1)[f],Var(5)[f])=(23.17)or(f(1),f(5)).

Now make the continuation explicit in the unary scoped node: once(t;k):=Scopeonce(;X;λ_.t;k),t:TX,k:XTA. The ordinary smart constructor is once(t)=once(t;Var). Hence once(t;k)[f]=(23.17)once(t;λx.k(x)[f]),once(t)[f]=identity continuationonce(t;f). The body t is byte-for-byte the same syntax in both lines. Only the explicit continuation changes.

For a concrete comparison, let t=or(Var(1),Var(5)),f(x)=or(Var(x),Var(x+1)). Then the three candidate clauses produce correct:once(or(Var(1),Var(5));f),false algebraicity:once(or(or(Var(1),Var(2)),or(Var(5),Var(6)));Var),all-fields traversal:once(or(or(Var(1),Var(2)),or(Var(5),Var(6)));f). The correct clause keeps the two original branches inside the scope and puts f only in the outside continuation. False algebraicity moves f into the owned computation and loses it outside. The all-fields traversal both moves and retains it, duplicating the post-computation.

Exceptions display both ordinary and scoped clauses. Define raise(e):=Opraise(e,absurd0),catch(M,H;k):=Scopecatch(;X;m;k), where M:TX, H:ETX, and m(inl())=M,m(inr(e))=H(e). For f:ATB, the complete substitution calculations are raise(e)[f]=(23.17)raise(e),catch(M,H;k)[f]=(23.17)catch(M,H;λx.k(x)[f]). The exception value e is an ordinary parameter. The protected computation M and every recovery computation H(e) are owned by the catch scope. Only the continuation after catch is composed with f.

Local state makes all three roles visible at once. Write local(n,s,t;k):=Scopelocal((n,s);X;λ_.t;k). Then local(n,s,t;k)[f]=(23.17)local(n,s,t;λx.k(x)[f]). The name n and initial state s are ordinary parameters, the computation t is owned by the local scope, and k is the only component composed with the later substitution.

Exercise 23.5

★☆☆ Let f(1)=or(Var(10),Var(11)) and f(5)=Var(50). Expand (23.19) completely. How many or-nodes are in the resulting syntax tree, and which of them came from the original term?

Exercise 23.6

★★☆ Let t=or(Var(1),Var(5)),f(x)=or(Var(x),Var(x+1)). Calculate once(t)[f] using (23.20). Then write the ill-behaved endomorphic homogeneous results produced by false algebraicity and by the all-fields traversal (23.16). Identify exactly where the extra or-nodes occur and which candidate loses the outside continuation.

Exercise 23.7

★☆☆ For arbitrary t:TX, k:XTA, and f:ATB, expand local(n,s,t;k)[f]. Give one syntactic equality which would fail if substitution accidentally changed an ordinary parameter and one which would fail if it descended into t.

Substitution equations and the monad

The construction is useful only if repeated explicit substitutions compose as ordinary substitution should. We prove this directly on the elementwise signature; no semantic interpretation is involved.

Theorem 23.11 — Explicit-substitution equations

Assume Σ and Γ have the polynomial presentation (23.4). Let t:TA, f:ATB, and g:BTC. Put ηA(a)=Var(a) and h(a):=f(a)[g]. Then Var(a)[f]=f(a),t[ηA]=t,t[f][g]=t[h]. All equalities are equality in the reindexing quotient of definition 23.4.

Proof of Theorem 23.11 — Explicit-substitution equations

Proof. Equation (23.22a) is the first defining clause of definition 23.9. For (23.22b), apply corollary 23.7 to the family PZ(u)u[ηZ]=u. For (23.22c), define QZ(u)B,C,f:ZTB,g:BTC.(u[f])[g]=u[(λz.f(z)[g])]. Both families respect reindexing because substitution is well defined by lemma 23.10. The variable case of P is Var(z)[ηZ]=ηZ(z)=Var(z) by the variable clause and the definition of ηZ. At an ordinary node, function extensionality and the induction hypotheses on every response branch give Opo(p,k)[ηA]=(23.17)Opo(p,λr.k(r)[ηA])=IHOpo(p,k). At a scoped node, the scoped computations are not traversed: Scopes(p;X;m;k)[ηA]=(23.17)Scopes(p;X;m;λx.k(x)[ηA])=IHScopes(p;X;m;k). This proves (23.22b).

For (23.22c), apply the induction just described to Q, and then instantiate its universal quantifiers with the displayed B,C,f,g. The variable case is Var(a)[f][g]=f(a)[g]=h(a)=Var(a)[h] by the variable clause. The ordinary case is Opo(p,k)[f][g]=(23.17)Opo(p,λr.k(r)[f][g])=IHOpo(p,λr.k(r)[h])=(23.17)Opo(p,k)[h]. The scoped case is the load-bearing one: Scopes(p;X;m;k)[f][g]=(23.17)Scopes(p;X;m;λx.k(x)[f][g])=IHScopes(p;X;m;λx.k(x)[h])=(23.17)Scopes(p;X;m;k)[h]. The body m is unchanged on all four lines. By lemma 23.10, the calculation descends from representatives to reindexing classes. ◻

Corollary 23.12 — Scoped-syntax monad

Under the polynomial hypothesis of theorem 23.11, define returna:=Var(a),t=f:=t[f]. Then T satisfies the three monad laws: returna=f=f(a),t=return=t,(t=f)=g=t=(λa.f(a)=g).

Proof of Corollary 23.12 — Scoped-syntax monad

Proof. Left and right unit are (23.22a) and (23.22b). Associativity is (23.22c). In each case, expand (23.23). ◻

The functorial renaming already present in the exact nested construction is not a second, unrelated traversal.

Lemma 23.13 — Renaming is return substitution

Under the same polynomial hypothesis, for r:AB and t:TA, Tr(t)=t[Varr].

Proof of Lemma 23.13 — Renaming is return substitution

Proof. For each set Z, let RZ(u) assert (23.25) for every set W and every map r:ZW. This family respects reindexing because both the functor action and substitution are well defined on the quotient. Apply corollary 23.7 to R and then instantiate it with the displayed r:AB. The variable case is Tr(Var(a))=Var(r(a))=Var(a)[Varr], by the functor and variable-substitution clauses. At an ordinary node, both sides preserve the parameter and apply the respective action pointwise to every response branch, so function extensionality applied to the induction hypotheses equates the resulting branch functions.

For a scoped representative [X,m,k], first compute the source functor action through its canonical nested representative. Packing gives the body qTk(m(q)). Applying Tr to the outer result applies T(Tr) to that body, hence T(Tr)(Tk(m(q)))=functoriality of TT((Tr)k)(m(q)). Unpacking and (23.12) therefore give the renamed node [X,m,(Tr)k]. Substitution by Varr gives instead [X,m,λx.k(x)[Varr]], and the induction hypothesis on each continuation branch gives k(x)[Varr]=Tr(k(x)). Thus the representatives agree pointwise. Well-definedness follows from lemma 23.10. ◻

The identity and composition laws for the source functor action now follow from theorem 23.11; the functor action used in lemma 23.5 and the monadic substitution are therefore compatible.

Example 23.14 — Two successive continuations

Let t:TX, k:XTA, f:ATB, and g:BTC. Then (once(t;k)=f)=g=(23.20)once(t;λx.(k(x)=f)=g)=associativityonce(t;λx.k(x)=(λa.f(a)=g)). Neither substitution duplicates nor rewrites t.

Exercise 23.8

★★☆ Using only (23.25) and theorem 23.11, prove Tid=id and T(gf)=TgTf. State exactly where left unit is used in the composition calculation.

Syntactic equations at three constructors

The monad laws determine substitution and composition at the three running constructors. The following calculations expose the relevant recursive position in each case.

Ordinary choice.

For f:ATB, or(t,u)=f=or(t=f,u=f). The operation is algebraic because both recursive children are response continuations.

Once.

For a smart once node, once(t)=f=once(t;f), not once(t=f). The scope body is not a continuation branch.

Local state.

For the source’s smart constructor, local(n,s,t):=local(n,s,t;Var), and therefore local(n,s,t)=f=local(n,s,t;f). The ordinary parameters n,s are stable, the local computation t stays inside the scope, and the continuation becomes f. An implementation which changes every field under bind has confused three different parts of the signature.

Equations (23.26)(23.28) belong to the abstract syntax monad; an operational interpretation requires a separate handler algebra.

Exact comparisons with neighboring abstractions

The words idiom, arrow, scoped operation, and higher-order operation describe different structures. Similar-looking examples are not a translation. We compare them only where a map can be written or a type obstruction can be exhibited.

From the scoped monad to an idiom

Every instance of the scoped syntax monad yields an applicative/idiom structure by the standard monadic translation pure(a):=returna,u<>v:=u=λf.v=λx.return(f(x)).

Proposition 23.15 — Monad-induced idiom

The operations in (23.29) satisfy the following well-typed applicative/idiom equations. In the identity law let v:TA; in homomorphism let f:AB and x:A; in interchange let u:T(AB) and y:A; and in composition let u:T(BC), v:T(AB), and w:TA. Then pure(idA)<>v=v,pure(f)<>pure(x)=pure(f(x)),u<>pure(y)=pure(λh.h(y))<>u,pure()<>u<>v<>w=u<>(v<>w). The last line is read left-associatively, with (f)(g)(x)=f(g(x)).

Proof of Proposition 23.15 — Monad-induced idiom

Proof. Expand pure and <> by (23.29). Identity and homomorphism reduce directly by the unit laws: pure(idA)<>v=definitionreturn(idA)=λh.v=λx.return(h(x))=left unitv=λx.return(x)=right unitv, and pure(f)<>pure(x)=definitionreturn(f)=λh.return(x)=λa.return(h(a))=left unit twicereturn(f(x)).

For interchange, put evy(h):=h(y). Both sides reduce to the same term: u<>pure(y)=(23.29)u=λh.return(y)=λa.return(h(a))=left unitu=λh.return(h(y)),pure(evy)<>u=(23.29)return(evy)=λe.u=λh.return(e(h))=left unitu=λh.return(h(y)).

For composition, factor both sides through the common normal form N:=u=λf.v=λg.w=λx.return(f(g(x))). Application associates to the left. Put P:=pure()<>u<>v. Expansion of (23.29), followed by left unit, gives P=u=λf.v=λg.return(fg). Expanding the final application and reassociating therefore yields P<>w=def., assoc.u=λf.v=λg.w=λx.return((fg)(x))=compositionN. On the right, first expand the inner application and then reassociate: u<>(v<>w)=definitionu=λf.(v<>w)=λy.return(f(y))=def., assoc.u=λf.v=λg.w=λx.return(f(g(x)))=definition of NN. ◻

Equation (23.29) is the exact one-way translation used here. We do not infer a converse from the applicative interface: pure and <> do not themselves define an operation of bind’s dependent type. This is a signature boundary, not a nondefinability theorem. Lindley’s comparison of idioms, arrows, and monads is used only as neighboring vocabulary, not as a theorem about our scoped signature [Lin14].

The Kleisli arrow

For the arrow comparison, use the exact Kleisli category of the monad and its cartesian product action; no abstract Arrow law package is imported. Take an arrow from A to B to be a function ATB, and define arr(f)(a):=return(f(a)),(gf)(a):=f(a)=g,first(f)(a,c):=f(a)=λb.return(b,c).

Proposition 23.16 — Kleisli category and product action

Let p:ATB, q:BTC, and r:CTD be Kleisli arrows; let f:AB and g:BC be ordinary maps; and let E be any set. The operations in (23.30) satisfy arr(idB)p=p,parr(idA)=p,(rq)p=r(qp),arr(gf)=arr(g)arr(f),firstE(arr(f))=arr(λ(a,e).(f(a),e)),firstE(qp)=firstE(q)firstE(p). The first four are the Kleisli-category laws; the last two are the product action equations used in this comparison. The subscript E only records the unchanged product component and is normally suppressed.

Proof of Proposition 23.16 — Kleisli category and product action

Proof. All equations are pointwise. For the left identity, (arr(idB)p)(a)=(23.30)p(a)=return=right unitp(a). For the right identity, (parr(idA))(a)=(23.30)return(a)=p=left unitp(a). Ordinary composition is preserved because (arr(g)arr(f))(a)=(23.30)return(f(a))=(returng)=left unitreturn(g(f(a)))=(23.30)arr(gf)(a). Associativity is ((rq)p)(a)=(23.30)p(a)=(λb.q(b)=r)=associativity(p(a)=q)=r=(23.30)(r(qp))(a). For the first product equation, firstE(arr(f))(a,e)=(23.30)return(f(a))=λb.return(b,e)=left unitreturn(f(a),e)=(23.30)arr(λ(a,e).(f(a),e))(a,e). For the second product equation, define Ne:=p(a)=λb.q(b)=λc.return(c,e). The left side reduces as follows: firstE(qp)(a,e)=(23.30)(qp)(a)=λc.return(c,e)=(23.30)(p(a)=q)=λc.return(c,e)=associativityNe. The right side reduces to the same term: (firstE(q)firstE(p))(a,e)=(23.30)firstE(p)(a,e)=firstE(q)=(23.30)(p(a)=λb.return(b,e))=firstE(q)=associativityp(a)=λb.return(b,e)=firstE(q)=left unitp(a)=λb.firstE(q)(b,e)=(23.30)Ne. Function extensionality completes each equation. ◻

This construction is one-way. An arbitrary arrow interface does not by itself select a canonical family of carriers TA, a return injection ATA, or the explicit scoped constructors (23.6). No converse reconstruction is claimed.

Scoped operations versus higher-order operations

The scoped signature (23.4) has one intermediate result set X at a scope node. Consequently every owned computation m(q):TX(qQs) returns the same X, and the outside continuation has domain exactly X: k:XTA. A higher-order operation has a different interface. Its fork positions may have result sets Bq, while the operation itself has an independently specified response set R. Its node data therefore have the shape ψq:HBq(qQ),κ:RHA. Here H names the later higher-order syntax only for this comparison; it is not identified with T.

To isolate the shape question, suppose a sortwise translation τZ:HZTZ is fixed for every set Z. This is an explicit assumption, not a translation theorem about the later calculus. Put X:=qQBq,ψ^q:=τBq(ψq),κ^(r):=τA(κ(r)). A first failed counterexample points only to the heterogeneous sets Bq. That objection is too weak: functorial renaming gives Tinq(ψ^q):TX(qQ). Thus tags repair heterogeneity. They do not repair the continuation: κ^ has domain R, while a scoped node over the coproduct carrier needs a continuation with domain X.

Definition 23.17 — Structural continuation translation

For sets X and R, a structural continuation translation from response set R to carrier X is a family ΦY:(RY)(XY) indexed by sets Y, natural in the continuation codomain: for every u:YZ and :RY, uΦY()=ΦZ(u). Naturality says that changing what happens after the continuation commutes with the conversion of its domain from R to X. It rules out a conversion which inspects or invents codomain-specific syntax.

Proposition 23.18 — Continuation-adapter criterion

Structural continuation translations from R to X are in bijection with functions ρ:XR. The two directions are ΦY()=ρ,ρ=ΦR(idR).

Proof of Proposition 23.18 — Continuation-adapter criterion

Proof. Given ρ:XR, precomposition defines (23.33). For u:YZ, uΦY()=definitionuρ=definitionΦZ(u), so (23.34) holds.

Conversely, let Φ satisfy the naturality equation and put ρ:=ΦR(idR). For any set Y and map :RY, apply naturality from codomain R to codomain Y, with post-map and continuation idR. Then ρ=definitionΦR(idR)=(23.34)ΦY(idR)=identity lawΦY(). Hence every structural translation is precomposition with the recovered adapter. Recovering an adapter from precomposition returns the original ρ, and reconstructing Φ from the recovered adapter returns the original family by the displayed calculation. ◻

The criterion gives a genuine counterexample. Take two fork positions with BL=N,BR=2,R=. At the level of the interface (23.31), take H=Id and τZ=Var:ZTZ. Choose ψL=0, ψR=false, and use the unique continuation Here is the empty response set, corresponding to the empty type 0 used for response domains in chapter 22. Thus A is a singleton function space. The higher-order node is therefore well formed, and all of its computation-valued fields have the assumed translation. The coproduct carrier X=N+2 makes the fork homogeneous, but no function X exists. More generally, any common carrier C reached by maps from both inhabited fork-result sets is inhabited, so it cannot admit an adapter C. By proposition 23.18, there is no structural conversion of this node to the scoped shape using only functorial fork translations and a natural continuation conversion.

There are restricted positive bridges. Suppose the sortwise translation τ above is fixed, a scoped symbol s has Qs=Q, a matching ordinary parameter p:Ps and an operation-specific adapter ρ:XR are supplied. Then the translated node is exactly Scopes(p;X;λq.Tinq(τBq(ψq));κ^ρ). Equation (23.32) translates the fork positions and (23.35) changes the continuation domain. Taking R=X and ρ=idX is the simplest case. The adapter is additional operation-specific signature data; it is absent from an arbitrary higher-order operation. This conditional node translation proves neither inclusion nor equivalence of the two calculi, and it transfers no elaboration theorem from the separate higher-order system.

Exercise 23.9

★☆☆ Let F:ATB have the scoped form F(a)=once(ta;ka),ta:TX,ka:XTB. Calculate first(F)(a,c) from (23.30) and then by (23.20). Identify the unchanged scoped computation and the new outside continuation.

Sources and mathematical scope

The syntax (23.5)(23.6), the explicit-continuation reading, the source bind clause (23.18), and the nondeterminism, exceptions, and local-state signatures come from Piróg, Schrijvers, Wu, and Jaskelioff [PSWJ18]. The algebraicity obstruction in (23.2)(23.3) is their motivating calculation. This chapter proves the elementwise reindexing, substitution equations, and monad laws locally rather than using a citation as a proof. The reindexing equation in definition 23.4 is the elementwise form of the explicit-substitution quotient in Section 3 of that source.

Plotkin and Power characterize the first-order commutation boundary recalled from chapter 22 [PP03]. Lindley gives the neighboring idiom/arrow terminology [Lin14]; the concrete translations (23.29)(23.30) are proved directly here.

Operational handlers require an evaluation relation and typing judgment not present in this syntax, so their safety theorems begin from additional data. Likewise, higher-order modular elaboration is a separate calculus rather than a metatheorem of T.

Suggested first pass.

Exercise 23.10, Exercise 23.12, Exercise 23.14 form the suggested first pass.

Exercise 23.10

★★☆ Give a polynomial scoped signature for a unary transaction constructor with no ordinary parameters. For M:TX, k:XTA, and f:ATB, write the explicit node transaction(M;k). Then calculate transaction(M;k)=f and compare it with the false-algebraicity candidate that pushes both the stored continuation and the new bind into the body and resets the outside continuation, transaction(M=(λx.k(x)=f);Var). Also instantiate the all-fields traversal (23.16) in the endomorphic homogeneous case X=A=B. Your answer must distinguish its general type failure from the semantic transaction failure of false algebraicity in (23.1).

Exercise 23.11

★★☆ Let s have two scoped positions and no ordinary parameters. Suppose h:XY, m1,m2:TX, and k:YTA. Starting only from (23.12), show [X,(m1,m2),kh]=[Y,(Th(m1),Th(m2)),k]. Now substitute f:ATB on both sides and prove that the resulting representatives are again related by the same h. Do not appeal to lemma 23.10 as a black box; reproduce its one-step calculation for this binary case.

Exercise 23.12

★★☆ Use ΓX=X×XE to write an elementwise catch node with protected computation M:TX, recovery family H:ETX, and outside continuation k:XTA. Let f:ATB and g:BTC. Calculate one and then two successive substitutions. Identify which of M,H,k changes at each step, and verify the associativity equation for this node without referring to an interpretation of exceptions.

Exercise 23.13

★★☆ For t:TA, start from the elementwise smart constructor (23.15), pack it with (23.13), and obtain its canonical member of T(TA). Then substitute f:ATB, canonicalize again, and derive the nested source clause (23.18) for this unary signature. Every use of functoriality or reindexing must be named.

Exercise 23.14

★★★ A later higher-order operation has two fork positions with result sets BL=N and BR=2, and an independent response set R. Its data are ψL:HN, ψR:H2, and κ:RHA. Assume a sortwise computation translation τZ:HZTZ. First use X=N+2 and the injections to homogenize the translated fork computations, and translate the continuation pointwise to κ^:RTA. Then prove directly that a family ΦY:(RY)(XY) natural in Y exists exactly when an adapter ρ:XR is supplied. Instantiate the result at R= to obtain a counterexample. Finally give the restricted positive bridge for R=X and ρ=idX, and identify the additional signature data used by that bridge.

Exercise 23.15

★★★ Practical project.scoped-substitution Implement a finite executable model of ordinary and scoped syntax using the bundled Kappa compiler. The representation must make ordinary parameters, scoped computations, and explicit continuations different fields. Maintain this invariant: substitution descends through every ordinary branch, preserves every ordinary parameter and scoped computation, and composes the new post-computation on the right of the stored continuation.

The observable result is the following exact six-line semantic report followed by its summary line:

PASS ordinary substitution descends through ordinary branches
PASS scoped substitution preserves ordinary parameters
PASS scoped substitution leaves the scoped computation untouched
PASS scoped substitution composes only the continuation
PASS tested substitution equations and monad laws hold
PASS false algebraicity and all-fields traversal are distinguished
All 6 scoped-operations corpus cases passed.

The decidable acceptance test must check all of the following:

  1. ordinary substitution descends through nested ordinary choice;

  2. local-scope ordinary parameters are unchanged;

  3. a once body is unchanged by outer substitution;

  4. continuation composition preserves left-to-right order;

  5. the tested left-unit, right-unit, and associativity instances hold; and

  6. deliberately constructed false-algebraicity and all-fields trees are pairwise distinct from the correct scoped-substitution tree and from each other.

Then test at least three separate semantic mutations: one which pushes substitution into the scoped body, one which corrupts an ordinary scoped parameter, and one which reverses continuation composition. Every mutant must still parse and type-check but fail the semantic oracle. Restore the accepted source and require the frozen six-case oracle again. Appendix E records the commands, digests, and the implementation-evidence boundary.

Search the book

Type to search the local edition.