Python backend and acquisition engine

[henrypinkard]

Pycro-Manager gives access to the Micro-Manaer Core through the Java wrapper (MMCoreJ) via a ZMQ-based Java-Python translation layer. This comes with the advantage that it can make use of Java libraries like the various GUI components and NDTiffStorage library. However, the ZMQ bridge has a speed limit of ~100-200 MB/s, meaning that data-intensive applications cannot stream data directly to Python without saving to disk first. Saved image callbacks alleviate the shortcoming of this in many, but not all applications. Thus cannot do things like running live mode on a high speed camera without saving to disk. Applications like viewing a real time 3D PSF in napari would work a lot better if this were rectified.

The solution is that there needs to be python or c++ wrapped in python components for doing all of the things that Pycro-Manager's Java libraries do now: Data storage (NDTiffStorage), Image viewer (NDViewer), and Acquisition engine (AcqEngJ).

A drop in replacement for NDViewer is already complete now that the napari viewer is part of the API. c++ layer file saving that is available to both Java and Python would solve the data storage problem. This leaves the need for a pure python acquisition engine.

The best way to do this is probably to make an AcqEngPy that is identical in its API and functionality to AcqEngJ. This would allow PM users to seemlessly switch between running in concert with the MM GUI to running pure python applications with no change in code. Now with the help of @ieivanov's many awesome tests, there's a clear way to determine if it is working as expected, thus ensuring it could become usable and stable quickly.

Another advantage of a python acquisition engine, especially one that is not essential for many of the functions of pycro-manager is that it can be used to more quickly prototype new and better acquisition abstractions

The question is when to start trying to do this. Using pymmcore, this could be started right now (but maybe be used only with streaming to RAM, or required that users figure out how to save data themselves). Or it could be started post-c++ file saving to avoid this restriction. Or it could be started once a new and better lower level access to device layer comes online (writeup of this coming soon)

[tlambert03]

just for the sake of completeness for anyone who comes across this issue, this was indeed the goal of pymmcore-plus: python or native-lang components for all the things that the java libraries do. pymmcore-plus has an acquisition engine, image-viewer (view napari-micromanager) and widgets to control pretty much all of the core functionalities (via pymmcore-widgets).

I know when this has come up in the past you were less keen on the pure-python/no-java approach, but seeing that you're now considering adopting a similar approach to what pymmcore-plus has been doing, I'd be curious to hear any honest opinions you have as to why pymmcore-plus isn't a good starting point for you

[henrypinkard]

Good to hear from you @tlambert03. It would be great to discuss your ideas for this in more depth. If you have bandwidth, would you like to connect over email and find a time to chat on zoom?

[henrypinkard]

Closed by #685