Foreign function interface and C API

[8tx7K38ej1aBTKWK]

It would be nice to have a way to call C functions directly in Villain. For example,

> (ccall "add" 40 2)
42

where add is defined in C as

vl_val add(vl_val a, vl_val b)
{
  int64_t x = vl_unwrap_int(a);
  int64_t y = vl_unwrap_int(b);

  return vl_wrap_int(x+y);
}

Or maybe we could use argc, argv similar to Janet's C API so we can have variable arity functions.

Then we can implement many of the special forms as stdlib functions, such as gensym, string->symbol, etc.

(define (gensym)
  (ccall "gensym"))

(define (string->symbol s)
  (ccall "str_to_symbol" s))

This makes it trivial to add new features such as printing strings, loading dynamic libraries (dlopen), or maybe GUIs, etc.

We also need to define some API in C to box and unbox values in Villain, which are simply int64_t,

typedef int64_t vl_val;

int64_t vl_unwrap_int(vl_val x);
vl_val  vl_wrap_int(int64_t x);
// ....

This is probably going to be a large feature, so I'll break it up into several parts:

• Implement ccall function, collect extern labels, and pass arguments according to ABI or with argc and argv
• C API for boxing, unboxing Villain values
• Convert existing functions, such as gensym, string->symbol to stdlib functions.

Possible improvements:

• Arity checking in C if using argc and argv
• Calling dynamic libraries (so we don't have to link everything at compile time)

[dvanhorn]

I think this would be great, thanks for proposing it.

[8tx7K38ej1aBTKWK]

I think the argc and argv approach would be better since this makes it easy to extend gensym to be a variable-arity function

(ccall "gensym")

(ccall "gensym" "if")

@abrassel If you need some help, I can also help implementing some parts later this weekend.

[abrassel]

I also like the argc and argv approach. Yes please! I'd love some help or at least someone to bounce ideas off of.

[minghui-liu]

How is thing feature going? I am also interested. Is there a branch created for this?

[8tx7K38ej1aBTKWK]

I'm currently designing the C API part to rewrite the print_result function in main. It will be available under a branch named capi later.

[minghui-liu]

OK let me know after you push the branch. I'd like to help. I am very comfortable programming in C.

[abrassel]

Ethan Hickman and I are also working on this.

[abrassel]

@8tx7K38ej1aBTKWK what's the current state of this? Would appreciate a brief overview of what you've done and what you would like to see implemented to wrap up this feature.

[8tx7K38ej1aBTKWK]

If we want to wrap up this feature, the quickest option is
to implement the ccall function. ffi-call is probably
only possible after we have a better UTF-32 string
representation.

I'll give a high-level description of ccall here. If
anyone is interested, the implementation should be
straightforward.

For example, if we have

(ccall "func" a b c)

The three arguments should be evaluated and pushed onto the
stack in reverse order (see prim2 and prim3).

+-------------+
| c (argv[2]) |
+-------------+         ^
| b (argv[1]) |         | increasing addr
+-------------+         |
| a (argv[0]) |         |
+-------------+ <- rsp

Similar to prim2 and prim3, we should also be pushing
#f onto the compile-time environment as a placeholders.

Then,

mov rdi, 3   ; argc
mov rsi, rsp ; argv
; pad stack
call func
; unpad stack

This should be enough to implement the ccall function.
externs.rkt should also be modified to include extern
labels for the first argument of ccall.