Three quality goals

[masak]

I try to avoid filing roadmap or meta issues, because they tend to grow uncontrollably, and their lack of clear observable makes them hard to close. Making an exception here, because I think that talking about these three things together reveals a bigger picture, and that they are fairly well-defined already (if not entirely straightforward to achieve).

Errors

Tracked since forever in #18. What I want as a user is not just the error symbol, but the location where the error happened, plus as much of "symptom, diagnosis, treatment plan" as the system can figure out.

I can see three very short-term things that need to be implemented:

• The interpreter needs to get (file, line, column) information from the code its interpreting, in order to flag where the error happened — "file" is probably something else for the built-in globals, and it's not very relevant in the REPL (Edit: Fill the globals with position information #443.)
• When printing the error, we want to be able to display the code surrounding the error — at least a few lines before and after; this is in a way an extension of having the (file, line, column) information
• We also want to be able to indicate/point to various parts that we have details about, pertinent to the error that happened. There's an example in Improve error message mechanism #18

(Edit: Also, stacktraces, I guess? This is a bit challenging in a tail-calilng interpreter, but it's doable. There's a nice blog post somewhere about a circular-list-of-circular-lists kind of data structure which can keep enough stacktrace information around.)

All of that is fairly straightforward and not that hard; need to Just Do It.

There's an added level of complexity later when the compiler and the bytecode interpreter gets involved. From what I can see, any time we do an optimizing transform, we need to take special care to preserve (file, line, column) information back to the original source. Certainly possible, but it feels like the kind of thing that needs to be carefully designed up-front, not added as a distracted afterthought.

Debugging

Tracked for a pretty long time in #52. Thinking about what I want here. It comes down to three things:

• Something that competes with "print debugging" in ergonomics and simplicity — end either wins or at least ends up tied with it. I want to be able to step through some code, do all the usual step into/step over, set breakpoints and watchpoints, etc. But also be able to inspect local scopes, including changing some values while the program is paused.
• The ability to ask "who set this value and when"? This approaches Pernosco-like debugging. I think by just instrumenting set-style assignments in the interpreter, keeping a list of events for each variable, this would be possible.
• The ability to "debug" an error that just happened. As a first step, I would be content having just a frozen state right where the interpreter abandoned due to the error, something in spirit very much like a core dump. Later when we're comfortable with this, we can think about also supporting some kind of editing/resumption; I'm not 100% sure how much is possible there, and it's not critical to have in a first iteration.

Modules

Tracked in #391. The current state is a bit untenable/ridiculous, with me copy+pasting definitions of prlf and the unit testing mechanism into various other files, with the attendant denormalization pitfalls. I also wish more and more often that I had easier access to the things defined in bel-masak-prelude.

I suggest a more radical plan than in #391, just to quickly get off the path of copy+pasting:

• Change the bin/bel argument handling to allow loading/executing several files in sequence. Then at least I can keep things in separate files. (Maybe symlink them instead of copy them.) Each file executes in the accumulated global environment from the previous files. (Edit: Allow bin/bel to read several files #426.)
• Make the interpreter intercept and handle an import special form. We keep the same ability to load several files one after another, but now we run each in its own global environment, and the import form allows us to use things defined in an earlier file to be linked/copied into another.
  • I'm still undecided if we should go with "import everything" or "selectively choose the things to import". I think maybe trying it out is the only way to get enough feedback. The module imports issue in Alma goes through a number of good options and rationales. It's probably a good idea to look at Racket, too — even if their solution looks extremely overpowered, it might have good insights in it.
  • Edit: No, dammit. No over-complicating things. Import everything, as in, all the added global names from the other module. It's not perfect, but it's the Simplest Thing. Also, it works for Python.
  • I'd expect things we said above about debugging and errors to keep working even with multiple compilation units linking things from each other. Just pointing out that that's table stakes.
• As a final step, abandon the file system as the primary way imports are resolved, and declare dependencies in a project-level configuration file, something like package.json. This doesn't need to be perfect, but I'd like for the following two things to hold from the start:
  1. The dependencies thus registered are uniquely identified so that they resolve to something immutable; if you depended directly on v0.2 of the unit-testing package, you won't suddenly get v0.3 because you re-installed your dependencies from scratch and it was suddenly available. Lessons from Go versioning are applied right from the start.
  2. There should still be a clear user story for holding a "local copy" of someone's published dependency. Maybe you needed to patch a thing in their code, and you don't want to wait around for them to publish a patch release. This is essentially a re-introduction of the "file system" kind of thinking, but it's no longer primary, and you need to clearly opt into it.

As a final note, I'd like for import to work in the REPL as well. Basically we should define it as a dynamic construct, with all the static benefits being added on top of that when possible.

I'm wondering if we can make the following work. When you import some file in the REPL, you also set up a file change listener to that source file. If the file is saved (presumably in a nearby editor), then the REPL environment automatically re-imports the contents, adding/changing and removing imported definitions as necessary. This would require keeping a bit of extra state to track where definitions came from, but that's no big deal. I feel this might address the "reload" problem mentioned in #391.