Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a Scheme REPL based on the Ribbit VM #39

Merged
merged 221 commits into from
Mar 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
221 commits
Select commit Hold shift + click to select a range
f62467f
Make the Vector wrapper more flexible
mossprescott Jan 16, 2024
7cf9749
“Big” computer, with large, flat memory
mossprescott Jan 16, 2024
1f3be94
Comments
mossprescott Jan 18, 2024
312c347
Actually parse hex literals in @-instructions
mossprescott Jan 21, 2024
4775886
First example for big.py
mossprescott Jan 21, 2024
3433e5c
More options for symbols and statics
mossprescott Jan 21, 2024
d2fd7fe
Cleanup
mossprescott Jan 21, 2024
0ab953d
Use new SCREEN builtin
mossprescott Jan 21, 2024
8a665af
Use an old trick to take 2 instructions out of each loop iteration
mossprescott Jan 21, 2024
319123d
Use new assembler features
mossprescott Jan 21, 2024
831657d
Parse “#-“ data when assembling
mossprescott Jan 21, 2024
98098b2
Comment, organize, and cleanup
mossprescott Jan 21, 2024
efdef91
More ROM size feedback
mossprescott Jan 21, 2024
15118a5
Handle keydown and TTY
mossprescott Jan 21, 2024
5310172
Fix spurious writes when reading from keyboard
mossprescott Jan 21, 2024
d1df3db
In-progress: practice moving chars between keyboard and screen
mossprescott Jan 21, 2024
27e3f9c
Correct some alignment errors
mossprescott Jan 21, 2024
2bc6ae4
Fix alignment for delimiters and 1
mossprescott Jan 21, 2024
3cd2e32
Commit the screenshot for reference
mossprescott Jan 21, 2024
a8e8f5e
Actually, assembler-allocated statics are a great idea after all
mossprescott Jan 22, 2024
b504b0f
Deal with shifted keys being held down
mossprescott Jan 22, 2024
27f283b
Update very outdated comment
mossprescott Jan 22, 2024
90435ab
Update to clarify why big.py doesn’t get a proper comparison
mossprescott Jan 22, 2024
00723ae
Write raw characters from tty to stdout
mossprescott Jan 22, 2024
a9fac8c
Fix ROM size TODO to make more sense
mossprescott Jan 22, 2024
944cc3f
Update for latest
mossprescott Jan 22, 2024
8f62cd3
A different hack for accessing SP
mossprescott Jan 22, 2024
2dc5a13
Improve some random comments
mossprescott Jan 22, 2024
2441a83
First steps towards executing (Ribbit) scheme
mossprescott Jan 27, 2024
0d739b7
Possibly legit decoding of instructions
mossprescott Jan 27, 2024
db32534
Sketch out instruction loop
mossprescott Jan 28, 2024
9e055cb
Better tracing
mossprescott Jan 30, 2024
b211a7d
Handle first instruction type: jump to global
mossprescott Jan 30, 2024
8c6ba26
More tracing options
mossprescott Jan 30, 2024
7a985ba
More tracing tweaks
mossprescott Feb 1, 2024
ed789cb
Fix TRACE_FINE so it works; cleanup
mossprescott Feb 1, 2024
c1c3b63
Implement const with integer operand (no check)
mossprescott Feb 1, 2024
e750654
Remove dead comments
mossprescott Feb 1, 2024
302b1d2
More tracing improvements
mossprescott Feb 2, 2024
f4d815a
Don’t use bogus jump for main
mossprescott Feb 2, 2024
11a2a90
Inevitable tracing additions
mossprescott Feb 2, 2024
71ce188
Fix up symbol table initialization, hopefully
mossprescott Feb 2, 2024
22b3c07
Implement primitive 0 (rib)
mossprescott Feb 2, 2024
b050300
Tracing again
mossprescott Feb 2, 2024
bc52475
Cleanup
mossprescott Feb 2, 2024
9a99941
Fix tracing;
mossprescott Feb 3, 2024
e50869e
Skip the “main” instruction in a cleaner way
mossprescott Feb 3, 2024
205d188
Factor out instruction decoding
mossprescott Feb 4, 2024
37200ee
Mostly meaningful description
mossprescott Feb 4, 2024
2470bd9
Fix rvm.run()
mossprescott Feb 4, 2024
b57ef0d
Implement set global
mossprescott Feb 4, 2024
f40c80d
Initialize the stack with an outer continuation
mossprescott Feb 4, 2024
419f4e5
Fix set (global)
mossprescott Feb 5, 2024
4b07a24
Make the stub “main” instr explicitly invalid for clarity
mossprescott Feb 5, 2024
d1564e1
Handle “halt” instructions in tracing
mossprescott Feb 5, 2024
672d507
Actually, honest-to-god complete execution of “42”
mossprescott Feb 5, 2024
ff61aa6
Unit test the for one Scheme program that will currently run
mossprescott Feb 5, 2024
1e8cfeb
Implement “+”, and test it
mossprescott Feb 5, 2024
86aa302
Implement “sub”, with a test
mossprescott Feb 5, 2024
dd209ad
Implement get (global)
mossprescott Feb 6, 2024
647266d
Halt on call to non-primitive (not implemented yet)
mossprescott Feb 6, 2024
8bc9c69
Tracing: actually decode ribs on the stack helpfully
mossprescott Feb 6, 2024
b9c974a
Factor out memory inspection
mossprescott Feb 6, 2024
51b7f87
Use inspector in the tests
mossprescott Feb 6, 2024
4a1300c
Useful tracing in tests
mossprescott Feb 7, 2024
8ab5cae
Show the value of a symbol when it appears in jump/call
mossprescott Feb 7, 2024
cc44a93
Implement jump to closure
mossprescott Feb 7, 2024
b87c408
Implement get (slot)
mossprescott Feb 7, 2024
3cd0f35
Write a test for (define) that’s actually Scheme
mossprescott Feb 7, 2024
0f8f940
Implement poke
mossprescott Feb 8, 2024
8e1c189
Summarize ROM space for the interpreter vs the program
mossprescott Feb 10, 2024
be44e34
Check for no trace fn
mossprescott Feb 10, 2024
ef9a446
Factor out common loop sequences
mossprescott Feb 10, 2024
ac2b4d4
Populate dummy primitive vector table entries
mossprescott Feb 10, 2024
4657139
Implement arg1
mossprescott Feb 10, 2024
56baac3
Implement call (closure)
mossprescott Feb 10, 2024
aa46f87
Comments; random cleanup
mossprescott Feb 10, 2024
31f423d
Use whole word per char on screen
mossprescott Feb 10, 2024
a47dbe2
Update for simplified screen buffer
mossprescott Feb 10, 2024
4f67763
Run a program from an .scm file
mossprescott Feb 11, 2024
177b51a
Fix row/colum label digits
mossprescott Feb 11, 2024
92529b0
Implement <
mossprescott Feb 11, 2024
35542ee
Tolerate unexpected pairs (as in continuations)
mossprescott Feb 11, 2024
5be1a38
Clean up redundant TODOs
mossprescott Feb 11, 2024
ae3f6cd
Default to —print and —trace
mossprescott Feb 11, 2024
d0967ae
Update tests for simplified screen buffer
mossprescott Feb 11, 2024
5e2cdaa
Test three trivial cases for <
mossprescott Feb 11, 2024
f329e02
Update TTY test for new address
mossprescott Feb 11, 2024
798f3f1
Start porting the character table example to Scheme
mossprescott Feb 11, 2024
809b83a
Implement “close”
mossprescott Feb 13, 2024
56226c2
Fix decoding: don’t drop “if” on the floor
mossprescott Feb 13, 2024
12f8745
Implement “if”
mossprescott Feb 13, 2024
65888db
Implement “*”
mossprescott Feb 13, 2024
f22871c
Test a first non-trivial function: factorial
mossprescott Feb 13, 2024
c4f6eec
Don’t try to run pytest over Ribbit’s Python code
mossprescott Feb 13, 2024
601c7e9
Fixup for decoding constants over 3
mossprescott Feb 13, 2024
e014695
Print the interpreter assembly before trying to parse it
mossprescott Feb 13, 2024
a3386ab
Random cleanup
mossprescott Feb 13, 2024
0cc502c
Remove “unimp” tag from the halt primitive
mossprescott Feb 13, 2024
be2dc18
Generate the full character map table
mossprescott Feb 13, 2024
aea9513
Add support for alternative memory layouts in codgen simulator
mossprescott Feb 13, 2024
539517f
Add logic to avoid sending bogus addresses to the ROM
mossprescott Feb 13, 2024
8d679a3
Move the range-check into the simulator
mossprescott Feb 14, 2024
4b06215
Revert the tests for range-checking the ROM in the simulator
mossprescott Feb 14, 2024
9e84882
Implement field0/1/2
mossprescott Feb 14, 2024
e47a018
Run tests on both simulators
mossprescott Feb 14, 2024
622a512
Dial back the update frequency a bit to keep up with the faster simul…
mossprescott Feb 14, 2024
c0d07a6
Run the actual Ribbit repl (which isn’t working yet)
mossprescott Feb 14, 2024
ea4b9eb
Implement field0/1/2-set!
mossprescott Feb 14, 2024
47bb7bd
Define a simple putchar in scheme
mossprescott Feb 14, 2024
9e29d8e
Implement eqv?
mossprescott Feb 14, 2024
c27df7e
Fix TTY wiring for alternative memory system
mossprescott Feb 14, 2024
b3b918c
Implement getchar (low-level, just reading the memory)
mossprescott Feb 14, 2024
39f8f90
Implement arg2
mossprescott Feb 14, 2024
241aa3d
Implement rib? primitive
mossprescott Feb 14, 2024
2555da8
Parse command-line args
mossprescott Feb 15, 2024
5bce57d
Suppress printing of generated code
mossprescott Feb 15, 2024
59ab486
Use predictable names for statics
mossprescott Feb 16, 2024
88a942d
Accept a pre-configured AssemblySource
mossprescott Feb 16, 2024
6e36366
Note on out-of bounds constant values
mossprescott Feb 16, 2024
a32ac5b
Update getchar comment
mossprescott Feb 16, 2024
456a075
Constants to enable useful logging for debug
mossprescott Feb 16, 2024
e394c3b
Fix name when storing to a static
mossprescott Feb 17, 2024
de1c072
Start on a second interpreter implemented in Jack
mossprescott Feb 17, 2024
bea72b1
Run tests with Jack interpreter
mossprescott Feb 17, 2024
ead7ec0
Jack interpreter: only const implemented
mossprescott Feb 17, 2024
d03fb32
Add the initial value for each symbol to the table in ROM
mossprescott Feb 17, 2024
6dbc87d
Initialize the symbol table
mossprescott Feb 17, 2024
11dd42b
First passing test for the Jack interpreter: test_add
mossprescott Feb 18, 2024
836bfa4
Implement primitive “id” for Jack
mossprescott Feb 18, 2024
856d37f
Implement get (symbol) in Jack
mossprescott Feb 18, 2024
c427c00
Unbreak trivial test for nw parameterization
mossprescott Feb 18, 2024
875df74
A slightly more interesting test for quotation
mossprescott Feb 18, 2024
44dfb6a
Implement call (closure) and get (slot) in Jack
mossprescott Feb 18, 2024
7608341
Implement jump (closure); factor out handling of slot/global target
mossprescott Feb 18, 2024
7a8e698
Implement < primitive
mossprescott Feb 18, 2024
2ef6772
Implement - primitive
mossprescott Feb 18, 2024
363ce2a
Implement “if” instruction
mossprescott Feb 18, 2024
ab8156b
Implement “close” primitive (in Jack)
mossprescott Feb 18, 2024
adae1f8
Implement “poke” primitive (and the rest of the dispatching)
mossprescott Feb 18, 2024
232aae5
Implement “*” primitive (for Jack)
mossprescott Feb 18, 2024
dbd5466
Fix comment
mossprescott Feb 18, 2024
60ff334
Learn how to spell “Interpreter”, maybe
mossprescott Feb 18, 2024
4e58d59
Implement field0/1/2(_set!) primitives
mossprescott Feb 18, 2024
456c8f0
Implement “eqv?” primitive
mossprescott Feb 18, 2024
fe11b21
Implement “rib?” primitive
mossprescott Feb 18, 2024
ee0d921
Implement “arg1” primitive
mossprescott Feb 18, 2024
4e9649f
Fix rendering ofcontinuations on the stack to be usefuil
mossprescott Feb 19, 2024
d70b8f0
Make instruction tracing in tests more useful
mossprescott Feb 19, 2024
e023395
Increase max_cycles for the Jack interpreter; and fail more usefully
mossprescott Feb 19, 2024
dd55159
Finally get around to optimizing leaf functions
mossprescott Feb 19, 2024
cb25a13
Collapse simple expressions into Store(static)
mossprescott Feb 19, 2024
1fdf66e
Include Static ref in Value; they don’t need to be flattened
mossprescott Feb 19, 2024
81b544e
Tighter code for Statics appearing in Binary expressions
mossprescott Feb 19, 2024
b37e87c
Reduce interval between checking for tty to keep up with faster reg.p…
mossprescott Feb 19, 2024
bb2ad68
Regularize handling of Reg and Static (i.e. use Eval)
mossprescott Feb 19, 2024
0784579
Comment on side-effects
mossprescott Feb 24, 2024
b24be0f
Mangle the interpreter a bit to avoid some argument/local access
mossprescott Feb 24, 2024
03b1b16
Manually inline alloc() into push()
mossprescott Feb 24, 2024
6ec5ca8
Split some locals so most end up in registers
mossprescott Feb 24, 2024
3520072
Split proc used only for dispatch from saved copy
mossprescott Feb 24, 2024
1917d0c
Commit some commented code related to inlining
mossprescott Feb 24, 2024
cbd38e3
Consistently simlify comparison expressions
mossprescott Feb 24, 2024
4fbc176
Remove a few instructions from the return sequence
mossprescott Feb 25, 2024
298f5de
Reduce memory traffic for “registers” by assigning some temps to D
mossprescott Feb 25, 2024
34748c4
Fix while liveness analysis
mossprescott Feb 26, 2024
0063274
Inline negated values in conditions
mossprescott Feb 26, 2024
b0b1ebf
Assign some more transient values to D
mossprescott Feb 26, 2024
53dfb36
Force each statement’s code generator to be explicit about handling Temp
mossprescott Feb 27, 2024
8ce23e5
Handle operands in D for basically all binary/comp ops
mossprescott Feb 27, 2024
2a137e0
Simplify negative constants under binary expressions for trivial savings
mossprescott Feb 27, 2024
f9de620
Disable very verbose logging
mossprescott Feb 27, 2024
7d10a83
Tighten code slightly in multiply() and push()
mossprescott Feb 27, 2024
2ccbe76
Save an instruction when doubling a register
mossprescott Feb 27, 2024
4237108
Add a “screenAddr” primitive, for 12% speedup
mossprescott Feb 27, 2024
6883e33
Manually inline calls to pop()
mossprescott Feb 27, 2024
4034db9
More manual inlining in Jack interpreter
mossprescott Feb 27, 2024
89232f4
Add a function pointer mechanism: Jack.symbol and Jack.invoke
mossprescott Feb 28, 2024
1cd0dff
Use function pointers to dispatch to primitive handlers
mossprescott Feb 28, 2024
8e43b4f
Switch to screenAddr primitive; cleanup
mossprescott Feb 28, 2024
ac7a4ef
Move initialization into Jack, using Jack.symbol(); improved —print
mossprescott Feb 28, 2024
f23aab8
Implement “getchar” primitive in Jack; echo.scm example
mossprescott Feb 28, 2024
7c23bd5
Massive simulation speedup (by throttling screen updates)
mossprescott Feb 29, 2024
5b35273
Implement “arg2” primitive
mossprescott Feb 29, 2024
d938b6d
Use io.scm; commit ribbit’s min library
mossprescott Feb 29, 2024
60b27c4
Count ribs interpreted
mossprescott Feb 29, 2024
f80eeaf
Stash the head of the symbol table in the “rib” rib
mossprescott Mar 7, 2024
75468ca
Limit depth of lists; put top of stack on the left
mossprescott Mar 10, 2024
c3ed61e
Catch exceptions from trace function
mossprescott Mar 10, 2024
596c57c
Debounce in getchar
mossprescott Mar 10, 2024
b32e214
Show a blinking cursor
mossprescott Mar 10, 2024
f4b1229
Use Jack interpreter by default
mossprescott Mar 10, 2024
cdc1cbf
Update head comment
mossprescott Mar 11, 2024
f131333
Exports seem to work: remove an old hack
mossprescott Mar 11, 2024
b5158ce
Update for many changes
mossprescott Mar 11, 2024
d1bae37
Manually disable the ##feature-arity-check flag
mossprescott Mar 13, 2024
51f012d
Fix typo: was logging wrong field of unrecognized “pair”
mossprescott Mar 13, 2024
befeb97
Line editing
mossprescott Mar 17, 2024
ef47d36
Update decsription of getchar and memory layout
mossprescott Mar 17, 2024
dc19911
Echo a line at a time, to show the new behavior
mossprescott Mar 17, 2024
c466b59
Implement quotient; fixing int->string
mossprescott Mar 17, 2024
67208c2
Elide unreachable jump after return
mossprescott Mar 17, 2024
8489219
Remove some spurious comments
mossprescott Mar 17, 2024
af45ee2
Neevermind with locals; factor out recursive divide with fewer redund…
mossprescott Mar 21, 2024
c61583b
Show mem consumption when not tracing
mossprescott Mar 21, 2024
29d7bbc
Add screen capture of the REPL
mossprescott Mar 25, 2024
9cd4b62
Add a test for “eval” from the library
mossprescott Mar 25, 2024
96db8c2
Run quotient tests only on the Jack interpreter
mossprescott Mar 25, 2024
3aad0f6
Don’t double-quote strings
mossprescott Mar 25, 2024
8f2d85f
Don’t quote unrecognized ribs; fix tests
mossprescott Mar 25, 2024
5e846fc
Remove dead code
mossprescott Mar 25, 2024
a9080d6
Add a link to scheme
mossprescott Mar 25, 2024
5a1358b
Keep original rom size when overwriting the contents
mossprescott Mar 25, 2024
90ef6be
Add —scale parameter
mossprescott Mar 26, 2024
31d3586
Halt when the heap is full
mossprescott Mar 26, 2024
a092020
Default to coarse logging (when is this used?)
mossprescott Mar 26, 2024
6cf374a
Update comments
mossprescott Mar 27, 2024
ac004c7
Disable logging of IR
mossprescott Mar 27, 2024
1a25ea9
More comment fixes
mossprescott Mar 27, 2024
4646f65
Increase max cycles
mossprescott Mar 27, 2024
8368b5e
Add an interactive example
mossprescott Mar 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/pythonapp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
pip install -r requirements.txt
- name: Test with pytest
run: |
pytest -v --doctest-modules
pytest -v --doctest-modules --ignore alt/scheme/ribbit
- name: Lint with flake8
run: |
pip install flake8
Expand Down
31 changes: 26 additions & 5 deletions alt/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,29 @@ See each module for instructions.

## Enhanced chips

Four alternative implementations use more or less chip hardware to make programs run faster, or to fit larger programs in ROM:
Four alternative implementations use more or less chip hardware to make programs run faster, or to
fit larger programs in ROM:

[alt/sp.py](sp.py) adds instructions for pushing/popping values to/from the stack, making programs more compact.
[alt/sp.py](sp.py) adds instructions for pushing/popping values to/from the stack, making programs
more compact and efficient.

[alt/threaded.py](threaded.py) adds lightweight CALL/RTN instructions, enabling a very compact "threaded interpreter" translation, which runs a little slower.
[alt/threaded.py](threaded.py) adds lightweight CALL/RTN instructions, enabling a very compact
"threaded interpreter" translation, which runs a little slower.

[alt/shift.py](shift.py) adds a "shiftr" instruction, and rewrites "push constant 16; call Math.divide" to use it instead; also a more efficient Math.multiply using shiftr.
[alt/shift.py](shift.py) adds a "shiftr" instruction, and rewrites
"push constant 16; call Math.divide" to use it instead; also a more efficient Math.multiply using shiftr.

[alt/eight.py](eight.py) is, finally, a _smaller_ CPU, by using an 8-bit ALU and 2 cycles per instruction.

[alt/big.py](big.py) has a single, flat memory space, with maximum RAM and the ability to read
data from ROM (and code from RAM.) This is much more flexible and realistic, but adds a cycle to
fetch each instruction from the shared memory system. Moving static data to ROM can dramtically
improve code size and performance, but because the computer uses character-mode graphics, these
metrics don't provide a direct comparison (even if you did port the VM and OS, which I haven't.)
This architecture is intended to support more sophisticated languages (e.g. BASIC, Scheme, or Forth),
and interactive programming.


## Enhanced compiler/translators

These implementations all use the standard CPU, and try to generate more efficient code for it:
Expand All @@ -30,6 +43,12 @@ local variables and expression evaluation, reserving the stack only for subrouti
[alt/reduce.py](reduce.py) adds an optimization phase after parsing and before the normal compiler runs, which
replaces certain function calls with lower-overhead "reduced" alternatives.


## Alternative languages

[alt/scheme](scheme/) provides a compiler and REPL for the Scheme language (circa R4RS), using the "big" architecture.


## Results

| Location | Nands | ROM size | Cycles per frame | Cycles for init |
Expand All @@ -39,10 +58,12 @@ replaces certain function calls with lower-overhead "reduced" alternatives.
| [alt/threaded.py](threaded.py) | 1,549 (+23%) | 8,100 (-68%) | 49,600 (+20%) | 173,750 (+34%) |
| [alt/shift.py](shift.py) | 1,311 (+4%) | 26,050 (+1%) | 19,800 (-52%) | _same_ |
| [alt/eight.py](eight.py) | 1,032 (-18%) | _same_ | +100% | +100% |
| [alt/big.py](big.py) | 1,448 (+14%) | ? | ? | ? |
| [alt/lazy.py](lazy.py) | _same_ | 23,650 (-8%) | 37,300 (-10%) | 111,000 (-14%) |
| [alt/reg.py](reg.py) | _same_ | 20,900 (-19%) | 19,150 (-54%) | 59,000 (-54%) |
| [alt/reg.py](reg.py) | _same_ | 18,200 (-29%) | 12,450 (-70%) | 55,250 (-57%) |
| [alt/reduce.py](reduce.py) | _same_ | 27,350 (+6.5%) | 20,300 (-51%) | _same_ |


**ROM Size** is the total number of instructions in ROM when Pong is compiled and translated
from the Jack source.

Expand Down
Loading
Loading