Skip to content

Play Nice With Alien, Platypus and XS

Graham✈️✈️ edited this page Nov 27, 2019 · 9 revisions

Dynamic Libraries are great because they save space, and ease updates. They are terrible because they add a lot of complexity. It's no wonder that at least some modern languages, like Rust and Go encourage static linkage for this reason. For Perl and Alien there is more than one way to do it, so we have to deal with dynamic and static libraries. The TL;DR is that for system installs (one where the Alien uses the system library) it doesn't make much of a difference if it finds static or dynamic libraries. Either will most of the time just work if the operating system is correctly configured. For a share install (one where the package source is downloaded from the internet and built locally), you generally want to use static libraries for XS and need dynamic libraries for FFI. (If you are already onboard with me you can skip the next two paragraphs).

In more detail, dynamic libraries (Linux/BSD/Solaris: .so, macOS: .dylib, Windows: .dll) are implemented in subtly different ways on each platforms. The runtime linkage on different platforms work differently: each system uses its own set of environment variables and/or compiler flags. A specific challenge for Aliens is that if you ever re-install an Alien, and that pulls in a new or differently configured version of the Alienized package, it can break already installed XS modules that depend on it. Some packages just refuse to build static libraries, and for those there is an Alien::Build role Alien::Role::Dino that does its best to Alienize dynamic library-only packages. I recommend that if you do not need it that you don't use it. For FFI, perhaps obviously, dynamic libraries are required

Static libraries (UNIX: .a, Windows: .lib; although both are actually the same format) are more uniform. When you build an XS module and link against a static library, you get the XS and C code bundled with your distribution, plus the parts of the static library that are "reachable", that is, the parts that are actually used by your extension. The one nit here is that on platforms that require it, the static library must be compiled as relocatable code (if you've seen the gcc flag -fPIC that is what it is doing). Once the XS is built it doesn't matter if the Alien it depends on has been re-installed because the code from the library that is needed is already in the built XS. If you do need to upgrade the library used by the XS, you can re-install the Alien and the XS, but at least the existing XS install won't break.

So we want both dynamic libraries and static libraries, but we want XS to link against the static libraries and for FFI to use the dynamic libraries. The problem is, by default C compilers usually use the dynamic libraries, and each platform has a different interface for preferring static libraries. The recommended approach for Alien is to isolate the dynamic libraries so that they can be used by FFI, and ignored by the C compiler when building XS. We do this by putting the dynamic libraries in a dynamic directory instead of lib. If done correctly it just works. There are a couple of situations that you have to look out for though.

  1. The Alienized package has just libraries, no tools.

In this case the easiest thing to do is to move the dynamic libraries from the default install location lib to dynamic after the Alienized package is staged for install. There is, helpfully, an Alien::Build plugin that will do this for you called Gather::IsolateDynamic.

use alienfile;
plugin 'Gather::IsolateDynamic';
Clone this wiki locally