-
Notifications
You must be signed in to change notification settings - Fork 6
VIV. Engine Management II (AvailableEngines)
When JDLL is first installed it is "empty", to be able to run models, the corresponding Deep Learning (DL) frameworks (engines) are need to be downloaded and setup (explained here).
Each model might need a different DL framework and version, thus for several models the user might end up with many new engines installed. JDLL offers tools to keep track of what is available and what has been installed to minimize conflicts and improve user experience.
These tools are separted in two classes, one helps understanding which engines have already been installed: io.bioimage.modelrunner.versionmanagement.InstalledEngines and the other one provides information about the engines that can be installed: io.bioimage.modelrunner.versionmanagement.AvailableEngines.
In this page the class io.bioimage.modelrunner.versionmanagement.AvailableEngines and its methods are going to be detailed to help users understand how they can retrieve the information about the engines that can be installed and used from JDLL.
The class comprises various methods for obtaining information about engines compatible with JDLL, which, in turn, aids in improving engine management.
All the information about the engines supported by JDLL can be found in the engines JSON file. This file enumerates all the engines available in JDLL together with the name of their DL framework, the version number in Python, the version number in Java, the operating system that supports it, the minimum Java version required to use it, whether it supports GPU or not, whether it supports CPU or not, whether it supports Rosetta or not and all the JAR files that are needed to run the engine.
The information of each of the engines can be instantiated as a DeepLearningVersion object to easily access each of its fields. This is better described in the JDLL Wiki page Engine Management I (DeepLearningVersion).
The AvailableEngines object can be instantiated by reading the engines JSON file. The follwwing code snippet creates an AvailableEngines instance:
BufferedReader br = new BufferedReader(new InputStreamReader(
AvailableEngines.class.getClassLoader().getResourceAsStream("availableDLVersions.json")));
Gson g = new Gson();
AvailableEngines availableEngines = g.fromJson(br, AvailableEngines.class);
All the versions present in the JSON file can be retrieved by:
availableEngines.getVersions();
The two snippets of code above together is exactly what the method AvailableEngines.getAll() does. Note that the code above is relatively low level compared to the simple method call AvailableEngines.getAll(). It is easier to use the static methods that are going to be detailed below. However, it is always useful to understand how the low level object works in case the user wants to implement a more complex logic.
Method that returns a list of all the DL engines that are supported by JDLL. A particular engine can only exist for a single OS, thus if a DL framework version is supported by Windows, Linux and MacOS, the engine will be repeated 3 times, each of them having a different OS field.
The output is a List of instances of DeepLearningVersion from which the wanted information about the engine can be obtained.
To return only the engines compatible with the OS where the software is runnnig use AvailableEngines.getForCurrentOS().
Below there is an example using the method. First all the engines are retrieved, then they are filtered for a specific version, framework and GPU compatibility. At the end, it can be seen how three engines for three different os are retrieved, while the actual DL engine is the same (Tensorflow 1.12.0 CPU).
List<DeepLearningVersion> allVersions = AvailableEngines.getAll();
List<DeepLearningVersion> versions = allVersions.stream.filter(v -> v.getFramework().equals("tensorflow")
& v.getVersion().equals("1.12.0")
& v.getGPU() == false)
.collect(Collectors.toList());
for (DeepLearningVersion vv : versions)
System.out.println(vv.getOS());
Output:
windows-x86_64
linux-x86_64
macosx-x86_64
Method that returns a list of all the DL engines that are supported by JDLL on the OS where the software is running. This method also filter the engines that are not compatible with the Java version of the JVM. In addition, in the case the computer is using Rosetta the engines that do not support it are filtered too.
The output is a List of instances of DeepLearningVersion from which the wanted information about the engine can be obtained.
Below there is an example using the method. First all the engines are retrieved, then they are filtered for a specific version, framework and GPU compatibility. At the end, it can be seen that only one engine is returned and that the engine is for the current OS.
List<DeepLearningVersion> allVersions = AvailableEngines.getForCurrentOS();
List<DeepLearningVersion> versions = allVersions.stream.filter(v -> v.getFramework().equals("tensorflow")
& v.getVersion().equals("1.12.0")
& v.getGPU() == false)
.collect(Collectors.toList());
for (DeepLearningVersion vv : versions)
System.out.println(vv.getOS());
Output:
windows-x86_64
Method that returns a list of all the DL engines that are supported by JDLL on the OS where the software is running. This method also filter the engines that are not compatible with the Java version of the JVM. In addition, in the case the computer is using Rosetta the engines that do not support it are filtered too.
The output is a List of instances of DeepLearningVersion from which the wanted information about the engine can be obtained.
- framework: name of the DL framework of interest. The list of possible frameworks can be found here.
Below there is an example using the method.
String framework = "tensorflow";
List<DeepLearningVersion> allVersions = AvailableEngines.filterByFrameworkForOS(framework);
System.out.println(allVersions .get(0).getFramework());
System.out.println(allVersions.size());
Output (retrieve all the tensorflow engines for Windows):
tensorflow
14
boolean AvailableEngines.isEngineSupportedInOS(String framework, String version, Boolean cpu, Boolean gpu)
Method that finds whether the engine defined by the input arguments is supported in the OS where the software is running. The method also requires that the engine is compatible with the Java version of the system and with the Rosetta, if the computer is using it.
The method AvailableEngines.getEnginesForOsByParams(String framework, String version, Boolean cpu, Boolean gpu) is very similar to this one , but instead of returning whether the engine of interest exists or not, it returns a list with the engines that satisfy the conditions.
- framework: name of the DL framework of interest. The list of possible frameworks can be found here. If
framework=null, the argument is ignored in the filtering process. - version: the version of the DL framework. If
version=null, the argument is ignored in the filtering process. - cpu: whether the engine needs to support CPU or no. If
cpu=null, the argument is ignored in the filtering process. - gpu: whether the engine needs to support GPU or no. If
gpu=null, the argument is ignored in the filtering process.
In the example below, all the arguments are set to null. It is equivalent to ask whether there is any engine supported in the system OS, with the JVM Java version and the Rosetta status. If the JVM Java version was 7 or lower, the result would have been false, because all the engines require at least Java 8.
Note how Boolean and Integer are used instead of boolean and int, this happens because the first ones allow setting the values to null.
String framework = null;
String version = null;
Boolean cpu = null;
Boolean gpu = null;
boolean exist = AvailableEngines.isEngineSupportedInOS(framework, version, cpu, gpu);
System.out.println(exist);
Output:
true
Another example to show the method can be used to see if a specific engine exists.
String framework = "tensorflow";
String version = "1.13.0";
Boolean cpu = true;
Boolean gpu = true;
boolean exist = AvailableEngines.isEngineSupportedInOS(framework, version, cpu, gpu);
System.out.println(exist);
Output:
true
Or to see if there is GPU support for a certain DL framework. In this example we do not care about CPU support or the version.
String framework = "tensorflow";
String version = null;
Boolean cpu = null;
Boolean gpu = true;
boolean exist = AvailableEngines.isEngineSupportedInOS(framework, version, cpu, gpu);
System.out.println(exist);
Output:
true
AvailableEngines.getEnginesForOsByParams(String framework, String version, Boolean cpu, Boolean gpu)
Method that finds the JDLL supported engines defined by the input arguments in the OS where the software is running. The method also requires that the engine is compatible with the Java version of the system and with the Rosetta, if the computer is using it.
The output is a List of instances of DeepLearningVersion from which the wanted information about the engine can be obtained.
- framework: name of the DL framework of interest. The list of possible frameworks can be found here. If
framework=null, the argument is ignored in the filtering process. - version: the version of the DL framework. If
version=null, the argument is ignored in the filtering process. - cpu: whether the engine needs to support CPU or no. If
cpu=null, the argument is ignored in the filtering process. - gpu: whether the engine needs to support GPU or no. If
gpu=null, the argument is ignored in the filtering process.
In the example below, all the arguments are set to null. It is equivalent to the method AvailableEngines.getForCurrentOS(). If the JVM Java version was 7 or lower, the result would have been false, because all the engines require at least Java 8.
Note how Boolean and Integer are used instead of boolean and int, this happens because the first ones allow setting the values to null.
String framework = null;
String version = null;
Boolean cpu = null;
Boolean gpu = null;
List<DeepLearningVersion> list = AvailableEngines.isEngineSupportedInOS(framework, version, cpu, gpu);
System.out.println(list.size());
Output:
33
Another example to show the method can be used to look for a specific engine.
String framework = "tensorflow";
String version = "1.13.0";
Boolean cpu = true;
Boolean gpu = true;
List<DeepLearningVersion> engineList = AvailableEngines.isEngineSupportedInOS(framework, version, cpu, gpu);
System.out.println(engineList.size());
Output:
1
Or to retrieve all the engines that support GPU for a certain framework. In this example we do not care about CPU support or the version.
String framework = "tensorflow";
String version = null;
Boolean cpu = null;
Boolean gpu = true;
List<DeepLearningVersion> gpuList = AvailableEngines.isEngineSupportedInOS(framework, version, cpu, gpu);
System.out.println(gpuList.size());
Output:
16