Personal FAQ

Kent’s Original Complaint

This original document was later updated. Click on the above tabs to see the updates.

Problem Description

The thesis of this article is that Scheme is mis-designed in that it does not seriously allow for the implementation of unwind-protect.

The community of Scheme designers has rejected my repeated calls over the years to fix this problem. The The Revised⁵ Report on the Algorithmic Language Scheme, or more commonly R5RS, merely disclaims the issue rather than deal directly with the very real and important problems it points to.

In plain English, the above says that a "normal return" from call-with-input-file or call-with-output-file will close the file, but any non-local return (such as abort to toplevel) will simply leave the file open until and unless the gc can prove that no one is still going to access the file.

In plain English again, the above says that a "normal return" from with-input-from-file or with-output-to-file will close the file, but any non-local return (such as abort to toplevel) the behavior is implementation-dependent. Note that this behavior is not only unhelpful to those who want the file closed, but it is not even consistent wording with what call-with-output-file and call-with-input-file do.

The whole point of unwind-protect is to assure that files do get closed when a specific dynamic contour is exited, but the problem with Scheme is that the so-called "full continuation" created by call-with-current-continuation, or call/cc, does not allow for a clear delineation of dynamic contours.

A non-solution: dynamic-wind

Sometimes when I cite this problem, people tell me that dynamic-wind is the solution. It is not. dynamic-wind exists to solve a completely different problem. The purpose of dynamic-wind is to taken action on every entry and every exit to a dynamic contour for effects that can be undone and redone. For example, dynamic-wind can be used to implement what Common Lisp calls special variables (i.e., "dynamic" variables). In effect, every time you leave a particular multi-tasked context, you run the dynamic-wind unwind procedure to undo the special binding, and every time you resume the process, you redo the special binding. (Some have said that dynamic-wind is a poor tool for this because it only works for synchronized multi-tasking on a single-processor machine and not for asynchronous parallel processing on a multi-processor machine. That may be true--certainly I've seen it convincingly argued; however, it is beyond the scope of this article.)

The problem with dynamic-wind as a solution to my stated problem description is that one cannot write with-open-file using dynamic-wind. It simply does not suffice to close the file on every temporary process exit and then re-open it on every re-entry to the process. The file should stay open even when another process is running and should be closed when all operations in the controlling process are done. Only an operator like unwind-protect is up to this task.

Some abstract remarks on the paradigm of exit

In their most general form, full continuations allow one to exit but not to know if something will be resumed. This is a potentially serious problem for the garbage collector. Consider some analogous real-world paradigms:

The Movie Ticket Paradigm

When you go to the movies, you get a ticket. The ticket is torn by the person who admits you, but you can still exit and re-enter using the torn ticket. This is very much like a first-class continuation. When you leave, they don't know if you'll be back, so they continue showing the movie. (This analogy is not perfect because in first-class continuations, the process you leave is stopped waiting for you to resume it. Ignore that part of the analogy and focus on the locking of resources so that you can get back in at all.) Now, usually there are lots of people in the theater when you leave, so there is no real problem about continuing the movie while you're gone. And, fortunately, the movie eventually ends so your ticket does not remain good forever in the way a continuation would. But for a little while the situation is approximately the same--your ticket locks up the resources.

Suppose you ran out to your car to get a bar of candy that you'd left there and returned to find that they had said "we closed up because no one was there". You would be outraged. Your ticket automatically entitles you to re-entry whether you made arrangements with them or not. By default, your leaving entitles you to return. It is a multi-entry ticket similar to a continuation. In effect, whether you are there or not, the theatre is forced to hold the resource open for you.

The Theme Park Paradigm

When you go to a theme park such as Disneyland or Seaworld, you buy an all-day pass. However, in spite of this, if you leave you will have to pay anew to return. If you do want to return, you must have your hand-stamped saying "I was here and wish to reserve the right to return" even though you carry with you pass for an all-day experience.

The reasons for this relate to their fear that you will sell or give your ticket to someone else that they could have charged, but their actual business motivation is not relevant. What is relevant is that the continuation you get must be explicitly requested to be multi-use.

In the highly unlikely event that everyone were to leave the theme park at noon, each such person having bought a full-day experience but none having requested a hand-stamp reserving the right to return, the theme park is justified in closing. And while some people might later arrive diappointed that the park was not open when advertised, none would be entitled to any full or partial refund of their ticket. That is, the mere obtaining of the ticket is not what locks the resource; what locks the resource is a request to do so.

Summary of Abstract Issues

The two paradigms presented above are intended to illustrate the mechanism in terms that people might really have confronted so they can apply some everyday experiential knowledge to this. You can use that information as you proceed in thinking about this problem or not, it's up to you. Some of the specific situations I cited are obviously contrived, but it's important to understand that language semantics is a kind of contract, and the purpose of a contract is often not to protect the common case where everyone's going to do "the obvious thing", but rather to help you navigate the less common situations, where someone is doing something out of the ordinary and you want to be able to rigorously answer the question "was this person entitled"?

Proposed Solutions

Because I believe unwind-protect is quite an important operator and I would like to see it added to Scheme, I proposed two different styles of fixes (approximately mirroring the paradigms cited in the previous section) to fix the problem. My suggestions were rejected. But unwind-protect is also still not part of the language. I claim its omission is because it can't be added with any materially useful semantics without addressing the semantic dissonance between it and continuations. The designers as a group seem so enamored of the present continuation semantics and its simplicity that they don't want to injure that in order to allow some materially useful functionality.

Proposed fix #1: Change call-with-current-continuation

Under this suggestion, call-with-current-continuation would take an extra argument saying whether to give out a single-use continuation or a multi-use continuation. Multi-use continuations would still be problematic for unwind-protect, but people who had applications that were not going to be needing "full continuations" would not have to have them.

In this model, any exit from the call-with-current-continuation context, whether through the escape procedure or through normal exit, would void a single-use continuation. In that way, unwind-protect would know that it could safely run as the conceptual stack was unwound.

This model corresponds approximately to the movie theater paradigm. You identify at purchase time of your ticket if you want a ticket that lets you out multiple times. (Movie theaters don't, but could, sell you a ticket that did not let you out.)

So the equivalent of Common Lisp's block and return facility, where return can only be used once, would be done by:

  ;; This takes two arguments
  ;; arg1 (single-use-p): t if single use, nil otherwise
  ;; arg2 (value):        a function to receive the continuation
  (call-with-current-continuation t ;i.e., give me a single-use continuation
    (lambda (return) ..body..))

Another possible way to modify call-with-continuation is to make it yield two continuations, a "suspend" (non-final) continuation that can be reused and a "finish" (final) continuation that invalidates both itself and its corresponding "suspend" continuation.

Repeating the above example in this form:

  (call-with-current-continuations
    ;; Gives out two continuations, a suspend continuation 
    ;; and a finish continuation.  CL's RETURN uses the finish kind.
    (lambda (ignore return)
      ..body..))

Proposed fix #2: Change continuations themselves

This is more like the theme park paradigm above. Upon exit, one would have to identify their intent to return. As it stands now, continuations are always a function of n arguments which is the number of values being "returned". Instead, there could be one additional argument (an obligatory zeroth argument) that was true if this was the last use of this continuation and false otherwise. In the case that it was true, unwind-protect cleanup clauses would be free to run. In the case that it was false, they would not be free to run, but at least there would be a belief that holding open the resource was being done as a result of conscious thought about what was likely to be needed next rather than some slavish attention to duty that might be misplaced, since often multi-use continuations are not used multiple times.

So the equivalent of Common Lisp's block and return facility, where return can only be used once, would be done by:

  (call-with-current-continuation
    (lambda (trampoline) ; trampoline is a TWO argument continuation
                         ; arg1 (last-use-p): t if last use, nil otherwise
                         ; arg2 (value): an object to return
      (let ((return (lambda (x) (trampoline t x))))
        ..body..)))

Summary

Footnotes

Dorai Sitaram wrote a paper entitled "Unwind-protect in portable Scheme" addressing this issue in followup. It took me a while to get to this, so for a while I just cited it as "related work". I've now read it and have placed some comments here.