consistent python packages installation

[joamatab]

What's the recommended way to register python packages inside klayout's python?

We are using a script
that requires you to manually specify your lib/python location

Where could we store some information like this for each PDK? Maybe in the tech.lyt?

Ideally, one day we can build klayout GUI from conda in Linux, MacOs and Linux and then use the conda packages for this

[klayoutmatthias]

I don't think tech.lyt is the right place as it is intended to dynamically switch technologies and even have different technologies loaded at the same time.

I'm not sure I understand what you want to achieve: do you want to run KLayout in the context of a certain PDK and load PDK-specific Python modules? In that case, you can simply set the KLAYOUT_PYTHONPATH environment variable to point to the location of your PDK-specific libraries. This variable acts like PYTHONPATH, but was renamed because users encountered conflicts with third-party tools using PYTHONPATH.

If you want to run KLayout in an isolated context, you can also set the KLAYOUT_HOME environment variable which lets KLayout use this directory for configuration files and script location. Once you set KLAYOUT_HOME and KLAYOUT_PYTHONPATH, you are in full control over the files KLayout reads.

Additionally, here is the explanation how packages and technologies interact with Python:

1. If a package contains a "python" subdirectory, this is added to sys.path. So package specific libraries can be supplied. Proper naming is required so there is no conflict with other potential libraries.
2. If a package contains a "pymacro" subdirectory, auto-exec macros from there will be executed on start and menu items will be created for macros bound to menus.
3. If a technology folder contains a "pymacro" subdirectory, auto-exec macros from there will be executed on start and menu items will be created for macros bound to menus. These menu entries will only be visible when the respective technology is active (i.e. used in the current view).

But packages and technologies do not establish an "application context" but rather modular extensions which can be added, removed or (in case of technologies) activated for specific views.

Matthias

[thomasdorch]

Hi Matthias,

When using the KLAYOUT_PYTHONPATH environment variable, and open KLayout it seems to be appending "/python" to every path I include:

KLAYOUT_PYTHONPATH=/home/thomas/mambaforge/envs/gdsfactory/lib/python310.zip:/home/thomas/mambaforge/envs/gdsfactory/lib/python3.10:/home/thomas/mambaforge/envs/gdsfactory/lib/python3.10/lib-dynload:/home/thomas/mambaforge/envs/gdsfactory/lib/python3.10/site-packages klayout

In KLayout's macro development window, sys.path then returns:

'/home/thomas/mambaforge/envs/gdsfactory/lib/python310.zip/python', '/home/thomas/mambaforge/envs/gdsfactory/lib/python3.10/python', '/home/thomas/mambaforge/envs/gdsfactory/lib/python3.10/lib-dynload/python', '/home/thomas/.local/lib/python3.10/site-packages/python', '/home/thomas/mambaforge/envs/gdsfactory/lib/python3.10/site-packages/python'

Hence why the macro script @joamatab referenced does essentially the same thing, but without appending "/python" to the paths. Is there a way to just pull in the plain paths?

[klayoutmatthias]

I don't know why Py_SetPath does, but there is nothing that makes KLayout append "python" to the path (pya.cc, line 301++):

  const char *python_path = getenv ("KLAYOUT_PYTHONPATH");
  if (python_path) {

    std::wstring path = tl::to_wstring (tl::to_string_from_local (python_path));
    Py_SetPath (path.c_str ());

  }

I also can't confirm that "python" is appended (mine is system python3 on Ubuntu-22).

Matthias

[thomasdorch]

Ah so it looks like it adds the paths both with and without the "python" bit appended actually (on Debian Testing).

[klayoutmatthias]

No ... it doesn't. No "python" added.

But: KLayout adds additional paths. Namely:

• $KLAYOUT_HOME/python
• subfolders of $KLAYOUT_HOME/pymacros to allow "import" of packages located there
• "python" subfolders of packages ($KLAYOUT_HOME/salt/xyz/python)
• The same from all folders listed in $KLAYOUT_PATH.

I assume this is what happens to you.

If you want to avoid that, I suggest to use

KLAYOUT_HOME=/dev/null
KLAYOUT_PATH=

Kind regards,

Matthias