Is it possible to save/load CL_PROGRAM_BINARIES?

[trxcllnt]

Looking into improving container startup times, I noticed a common pattern is compiling kernels from source once per device, and loading the program binaries from disk on subsequent restarts.

Is it possible to get a handle on the underlying program binaries buffer in order to serialize them to disk, and if so, can the result be loaded via cl.createProgramWithBinaries()? Thanks Mike!

[mikeseven]

That's one part I've not touched ie containers. I definitely need to update. Regarding binary, it should work as usual. However, it's not portable. So you would need to generate binaries for each family of GPU. Some would support SPIR and be portable. What I would do, assuming your kernels don't change, is run them once and get the binary to disk or volume container. Then launch the app again using that binary.

…

--mike

________________________________ From: Paul Taylor <[email protected]> Sent: Monday, April 2, 2018 2:08:04 PM To: mikeseven/node-opencl Cc: Subscribed Subject: [mikeseven/node-opencl] Is it possible to save/load CL_PROGRAM_BINARIES? (#62) Looking into improving container startup times, I noticed a common pattern is compiling kernels from source once per device, and loading the program binaries from disk on subsequent restarts. Is it possible to get a handle on the underlying program binaries buffer in order to serialize them to disk, and if so, can the result be loaded via cl.createProgramWithBinaries()? Thanks Mike! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#62>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAxYLKpoTugjxZV8YYMrs3U1hajecbRhks5tkpMzgaJpZM4TEFzG>.

[trxcllnt]

Yep, I'm comfortable with that part. We can essentially make it part of the container's CMD startup script.

The part I'm stuck on is getting a reference to the raw char* pointer of the NoCLProgramBinary instances. The toString() value doesn't seem to include the PTX assembly, and there don't seem to be any other methods exposed to v8. Am I missing something?

p.s. I've looked into some other tools like clcc, but most I've found aren't maintained and/or segfault on various platforms, so a small node-opencl script seems like the most straightforward if it's possible.

[mikeseven]

Yes or could even bind mount a volume. Indeed, what you hit is the non standard part. It looks like you are using Nvidia GPU. You should try some of their examples to verify that you can save and load the binary and that it works. I wouldn’t use toString() because that’s really an ArrayBuffer you get back (like all operations with OpenCL). You should get a byte data view, save it and load it, and that will be the same as the native Nvidia example, likely PTX. From: Paul Taylor [mailto:[email protected]] Sent: Monday, April 2, 2018 3:03 PM To: mikeseven/node-opencl <[email protected]> Cc: Bourges-sevenier, Mikael <[email protected]>; Comment <[email protected]> Subject: Re: [mikeseven/node-opencl] Is it possible to save/load CL_PROGRAM_BINARIES? (#62) Yep, I'm comfortable with that part. We can essentially make it part of the container's CMD script. The part I'm stuck on is getting a reference to the raw char* pointer of the NoCLProgramBinary instances. The toString() value doesn't seem to include the PTX assembly, and there don't seem to be any other methods exposed to v8. Am I missing something? p.s. I've looked into some other tools like clcc, but most I've found aren't maintained and/or segfault on various platforms, so a small node-opencl script seems like the most straightforward if it's possible. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#62 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAxYLJiMPZYk242aIPkEEo5cDKi4jzm-ks5tkp_9gaJpZM4TEFzG>.

[trxcllnt]

For reference, here's a snippet of what I'm trying to do (using our high-ish level wrapper lib around node-opencl):

const square = fs.readFileSync(path.resolve(__dirname, `square.cl`)).toString();

for (const device of CLPlatform.devices('all')) {
    const context = device.newContext();
    const program = context.newProgram(square).build();
    const binaries = program.BINARIES;
    const binarySizes = program.BINARY_SIZES;
    const binaryBuffers = binaries.map((x) => bufferFromBinary(x));
    try { fs.mkdirSync(path.resolve(__dirname, device.TYPE_NAME)); } catch(e) {}
    fs.writeFileSync(path.resolve(__dirname, device.TYPE_NAME, 'square.bin'), binaryBuffers[0]);
    fs.writeFileSync(path.resolve(__dirname, device.TYPE_NAME, 'square.sizes'), binarySizes[0]);
    program.release();
    context.release();
}

function bufferFromBinary(programBinary) {
  /* todo: implement for real */
  return new Buffer(programBinary);
  /* tested, doesn't work:
    const bin = programBinary.toString();
    const str = bin.substring(bin.indexOf('@') + 1, bin.lastIndexOf(']'));
    const buf = new Buffer(str);
    return buf;
  */
}

Yes or could even bind mount a volume.

Yep, exactly. Startup script just checks if the mounted volume has compiled binaries, and if so skips compilation.

It looks like you are using Nvidia GPU. You should try some of their examples to verify that you can save and load the binary and that it works.

Ah I'm just trying to do this on my MacbookPro at the moment, but I'll test on an Nvidia GPU when I get home tonight. I was using PTX as a stand-in for whatever the Apple/Intel/AMD drivers I have right now use.

I wouldn’t use toString() because that’s really an ArrayBuffer you get back (like all operations with OpenCL)

To clarify, are you saying the NoCLProgramBinary instances are ArrayBuffers? Or the raw pointer is? If the latter, is there a way to get a pointer to the ArrayBuffer from JS without calling toString()? I've also tried getRaw(), enumerating keys, slicing the bytes from the toString() result, creating a new Uint8Array from each NoCLProgramBinary instance, etc.

Thanks!

[mikeseven]

That looks raidonable :-) Gotta check what the issue is. Yes in general any data on node-opencl is ArrayBuffer or similar in latest nodejs so data is aligned properly on cpu and gpu memory without incurring translation. I do use other types of buffers for convenience but they are slower because they require translation.

…

--mike

________________________________ From: Paul Taylor <[email protected]> Sent: Monday, April 2, 2018 3:23:11 PM To: mikeseven/node-opencl Cc: Bourges-sevenier, Mikael; Comment Subject: Re: [mikeseven/node-opencl] Is it possible to save/load CL_PROGRAM_BINARIES? (#62) For reference, here's a snippet of what I'm trying to do (using our high-ish level wrapper lib around node-opencl): const square = fs.readFileSync(path.resolve(__dirname, `square.cl`)).toString(); for (const device of CLPlatform.devices('all')) { const context = device.newContext(); const program = context.newProgram(square).build(); const binaries = program.BINARIES; const binarySizes = program.BINARY_SIZES; const binaryBuffers = binaries.map((x) => bufferFromBinary(x)); try { fs.mkdirSync(path.resolve(__dirname, device.TYPE_NAME)); } catch(e) {} fs.writeFileSync(path.resolve(__dirname, device.TYPE_NAME, 'square.bin'), binaryBuffers[0]); fs.writeFileSync(path.resolve(__dirname, device.TYPE_NAME, 'square.sizes'), binarySizes[0]); program.release(); context.release(); } function bufferFromBinary(programBinary) { /* todo: implement for real */ return new Buffer(programBinary); /* tested, doesn't work: const bin = programBinary.toString(); const str = bin.substring(bin.indexOf('@') + 1, bin.lastIndexOf(']')); const buf = new Buffer(str); return buf; */ } Yes or could even bind mount a volume. Yep, exactly. Startup script just checks if the mounted volume has compiled binaries, and if so skips compilation. It looks like you are using Nvidia GPU. You should try some of their examples to verify that you can save and load the binary and that it works. Ah I'm just trying to do this on my MacbookPro at the moment, but I'll test on an Nvidia GPU when I get home tonight. I was using PTX as a stand-in for whatever the Apple/Intel/AMD drivers I have right now use. I wouldn’t use toString() because that’s really an ArrayBuffer you get back (like all operations with OpenCL) To clarify, are you saying the NoCLProgramBinary instances are ArrayBuffers? Or the raw pointer is? If the latter, is there a way to get a pointer to the ArrayBuffer from JS without calling toString()? I've also tried getRaw(), enumerating keys, slicing the bytes from the toString() result, creating a new Uint8Array from each NoCLProgramBinary instance, etc. Thanks! — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#62 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAxYLLzl6FMzafZE04-qrfBuRar-Vvtaks5tkqTPgaJpZM4TEFzG>.