faiss fails to import on python==3.12 because of deprecated numpy.distutils

[GaetanLepage]

Summary

To know whether the system supports SVE, faiss uses deprecated numpy.distutils.cpuinfo.
This has been removed and crashes on Python 3.12 (on aarch64-linux systems) with:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 1, in <lambda>
  File "/nix/store/6iq3nhgdyp8a5wzwf097zf2mn4zyqxr6-python3-3.12.5/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/nix/store/x3g3x2s64qwsrrrg6cpfc84zycc4hlbh-python3.12-faiss-1.9.0/lib/python3.12/site-packages/faiss/__init__.py", line 16, in <module>
    from .loader import *
  File "/nix/store/x3g3x2s64qwsrrrg6cpfc84zycc4hlbh-python3.12-faiss-1.9.0/lib/python3.12/site-packages/faiss/loader.py", line 88, in <module>
    instruction_sets = supported_instruction_sets()
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/x3g3x2s64qwsrrrg6cpfc84zycc4hlbh-python3.12-faiss-1.9.0/lib/python3.12/site-packages/faiss/loader.py", line 53, in supported_instruction_sets
    if is_sve_supported():
       ^^^^^^^^^^^^^^^^^^
  File "/nix/store/x3g3x2s64qwsrrrg6cpfc84zycc4hlbh-python3.12-faiss-1.9.0/lib/python3.12/site-packages/faiss/loader.py", line 43, in is_sve_supported
    import numpy.distutils.cpuinfo
ModuleNotFoundError: No module named 'numpy.distutils'

Platform

OS: NixOS

Faiss version: 1.9.0

Installed from: Nix

Faiss compilation options:

Running on:

• CPU
• GPU

Interface:

• C++
• Python

Reproduction instructions

• import faiss

[mengdilin]

acked, Nix is not the official supported way of installing faiss. If you need to be unblocked quickly, can you try installing faiss from conda-forge? I will check if https://anaconda.org/anaconda/py-cpuinfo can be used to replace the deprecated module

[GaetanLepage]

acked, Nix is not the official supported way of installing faiss. If you need to be unblocked quickly, can you try installing faiss from conda-forge? I will check if https://anaconda.org/anaconda/py-cpuinfo can be used to replace the deprecated module

I am not a faiss user myself. I do maintenance on the python ecosystem within nixpkgs.
For now, we have patched it by inserting a return False statement before the problematic import statement.

There is no particular urgency don't worry.

[cachitas]

I am also facing this exact same issue using Debian.
Specifically, installing faiss in a Docker container based on python:3.12-slim-bookworm.

[mengdilin]

@cachitas is this on x86 or aarch64?

[cachitas]

aarch64

[asadoughi]

This doesn't appear to be aarch64 specific: based on the numpy documentation it applies to all numpy uses in Python 3.12 and newer.

[andresdelgadoc98]

i have this issue:

 Traceback (most recent call last):
  File "/home/vboxuser/projects/chatbot-ia/index.py", line 3, in <module>
    app = create_app()
          ^^^^^^^^^^^^
  File "/home/vboxuser/projects/chatbot-ia/src/__init__.py", line 9, in create_app
    from .routes import rag,db
  File "/home/vboxuser/projects/chatbot-ia/src/routes/db.py", line 3, in <module>
    import faiss
  File "/home/vboxuser/projects/chatbot-ia/env/lib/python3.12/site-packages/faiss/__init__.py", line 16, in <module>
    from .loader import *
  File "/home/vboxuser/projects/chatbot-ia/env/lib/python3.12/site-packages/faiss/loader.py", line 88, in <module>
    instruction_sets = supported_instruction_sets()
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vboxuser/projects/chatbot-ia/env/lib/python3.12/site-packages/faiss/loader.py", line 53, in supported_instruction_sets
    if is_sve_supported():
       ^^^^^^^^^^^^^^^^^^
  File "/home/vboxuser/projects/chatbot-ia/env/lib/python3.12/site-packages/faiss/loader.py", line 43, in is_sve_supported
    import numpy.distutils.cpuinfo
ModuleNotFoundError: No module named 'numpy.distutils' 

with arm64

[guyueh1]

I have the same issue when trying to use faiss on an arm linux platform and numpy==1.26.0. I think the problematic import import numpy.distutils.cpuinfo only happend with this platform (aarch64, linux). Could the team fix this? Thanks!