Add memfd shmem backend

[bernhl]

memfd backed shmprovider. Tested on android with ForkserverExecutor but should be usable on other Linux flavors. Minor caveat: Android 7 and older as well as old linux kernel (AFAICT < 3.17) are unsupported.

[bernhl]

The advantage of using memfd over default POSIX shared memory is automatic cleanup when killing fuzzer instances. This prevents the accumulation of leftover shm files. On android, memfd is the designated successor to ashmem and attempts have been made (but reverted) to remove ashmem. Still, supporting on the successor future-proofs libafl.

[domenukk]

Should we switch the default Android to be Memfd?

[bernhl]

I would refrain from switching to memfd on android, because the afl-compiler-rt.o.c is not yet compatible.

[bernhl]

Alternatively, memfd could be made default on android but forkserver examples use ashmem instead.

[domenukk]

Alternatively, memfd could be made default on android but forkserver examples use ashmem instead.

That's already the case; In many cases Forkserver uses UnixShMemProvider on MacOS etc

LibAFL/libafl_sugar/src/forkserver.rs

Line 113 in 89cff63

let shmem_provider = UnixShMemProvider::new().expect("Failed to init shared memory");

(not sure if this even works on Android? :D )

[domenukk]

We'll release 0.14 soon, anything left to do here?

[bernhl]

I converted this to a draft because there seems to be an issue with memfd based on nix (instead of syscall from libc). I'll investigate later today.

[domenukk]

Else we can also go with the syscall method for now if nyx is acting up :)

[bernhl]

Actually, using nix instead of syscall works if one understands the difference between as_raw_fd and into_raw_fd...

[domenukk]

Merging, but we may want to
a) have a testcase or example fuzzer for this
b) maybe make this the default at some point