…-hnsw

Summary:
Pull Request resolved: facebookresearch#3422

Found vec_io failing when running some benchmarking.
There is no such field named big_endian in sys. So, reverting it to original field byteorder

Reviewed By: algoriddle

Differential Revision: D56718607

fbshipit-source-id: 553f1d2d6bc967581142a92282e534f3f164e8f9

Summary:
Pull Request resolved: facebookresearch#3423

Adding small fixes to run experiments from fbcode.
1. Added buck target
2. Full import path of faiss bench_fw modules
3. new dataset path to run tests locally as we can't use  an existing directory ./data in fbcode.

Reviewed By: algoriddle, junjieqi

Differential Revision: D57235092

fbshipit-source-id: f78a23199e619b640a19ca37f8b52ff0abdd8298

Summary:
Pull Request resolved: facebookresearch#3487

Sometimes it is not useful to serialize the storage index along with a HNSW index. This diff adds a flag that supports skipping the storage of the index.

Searchign and adding to the index is not possible until a storage index is added back in.

Reviewed By: junjieqi

Differential Revision: D57911060

fbshipit-source-id: 5a4ceee4a8f53f6f746df59af3942b813a99c14f

…kresearch#3494)

Summary:
Pull Request resolved: facebookresearch#3494

This tasks focus on the refine index construction tutorial with different index refinement on fp16/sq8 quantization. The python version was added a while ago.

Reviewed By: junjieqi

Differential Revision: D58161983

fbshipit-source-id: 1c598fe612b5dee3952c5f7398e6802e117f141d

Summary:
Adding build folder to gitignore, so that they don't show up in the commit tree while building from source

Pull Request resolved: facebookresearch#3492

Reviewed By: junjieqi

Differential Revision: D58171359

Pulled By: asadoughi

fbshipit-source-id: b0efed348769328a3bdbcc13098dcb84cadb6c4f

Summary:
Pull Request resolved: facebookresearch#3502

include probably added by vscode

Reviewed By: mengdilin

Differential Revision: D58411537

fbshipit-source-id: 3035f690d26decc937fb492c54ffa2f974ee2db8

Summary:
Add support for `use_raft` in the torch version of `knn_gpu`. The numpy version already has this support, see https://github.com/facebookresearch/faiss/blob/main/faiss/python/gpu_wrappers.py#L59

Pull Request resolved: facebookresearch#3509

Reviewed By: mlomeli1, junjieqi

Differential Revision: D58489851

Pulled By: algoriddle

fbshipit-source-id: cfad722fefd4809b135b765d0d43587cfd782d0e

…rch#3512)

Summary:
Pull Request resolved: facebookresearch#3512

Pull Request resolved: facebookresearch#3510

GitHub hosted runners some with the build-essentials package pre-installed, self-hosted runners on AWS do not have this package. This made it all steps other than the `all targets` one fall back to the system executables which unintentially worked on GitHub hosted runners but not on the self-hosted ones. This diff fixes it by pulling the line that adds conda bin to path early in the cmake build action.

Reviewed By: asadoughi

Differential Revision: D58513853

fbshipit-source-id: 23e95459e0031c96bd142515db07d1b700d713cf

Summary:
Pull Request resolved: facebookresearch#3515

Fix typo `test_io_no_storage`

Reviewed By: kuarora, asadoughi

Differential Revision: D58540190

fbshipit-source-id: b8b9cacd7ea6005c0edb94014de74188450318c1

…cebookresearch#3516)

Summary:
Pull Request resolved: facebookresearch#3516

This diff seeks to simplify the steps that install conda packages and environment configuration into a single step at the start of the cmake build action.

Reviewed By: mnorris11

Differential Revision: D58560454

fbshipit-source-id: ee2c6b36865809f31eb335cfb3c2fffdccaa318d

Summary:
Pull Request resolved: facebookresearch#3517

MSVC doesn't support unsigned int for OMP

Reviewed By: kuarora, junjieqi, ramilbakhshyiev

Differential Revision: D58591594

fbshipit-source-id: ac7d6b37a82f9543be3e0fe418f0f6b439751475

Summary:
Pull Request resolved: facebookresearch#3519

Fixing the conda conflicts because of `_openmp_mutex` build versions. This change pins that version for RAFT conda package builds.

Reviewed By: algoriddle

Differential Revision: D58646659

fbshipit-source-id: 4c1eaa9f08bd354da016b9399a36698007a497d8

…h#3522)

Summary:
Pull Request resolved: facebookresearch#3522

Quick fix to unblock nightly

Reviewed By: mlomeli1

Differential Revision: D58694193

fbshipit-source-id: ea323991cc2e2b958fc11ab614dcd6e09d4c072c

Summary:
Pull Request resolved: facebookresearch#3524

Searches with the metric abs(dot(query, database))
This makes it possible to search vectors that are closest to a hyperplane

* adds support for alternative metrics in faiss.knn in python

* checks that it works with HNSW

* simplifies the extra distances interface by removing the template on

Reviewed By: asadoughi

Differential Revision: D58695971

fbshipit-source-id: 2a0ff49c7f7ac2c005d85f141cc5de148081c9c4

Summary:
Pull Request resolved: facebookresearch#3531

**In this diff**
1. I have add bench_fw to bento faiss kernel target
2. First part of notebook is changed to analyze sift1M results

Reviewed By: algoriddle

Differential Revision: D58823037

fbshipit-source-id: a67d4638af4368f0575bd289ce7aff8cf1fcd38b

…ookresearch#3527)

Summary:
Pull Request resolved: facebookresearch#3527

**Context**
Design Doc: [Faiss Benchmarking](https://docs.google.com/document/d/1c7zziITa4RD6jZsbG9_yOgyRjWdyueldSPH6QdZzL98/edit)

**In this diff**
1. Be able to reference codec and index from blobstore (bucket & path) outside the experiment
2. To support #1, naming is moved to descriptors.
3. Build index can be written as well.
4. You can run benchmark with train and then refer it in index built and then refer index built in knn search. Index serialization is optional. Although not yet exposed through index descriptor.
5. Benchmark can support index with different datasets sizes
6. Working with varying dataset now support multiple ground truth. There may be small fixes before we could use this.
7. Added targets for bench_fw_range, ivf, codecs and optimize.

**Analysis of ivf result**: D58823037

Reviewed By: algoriddle

Differential Revision: D57236543

fbshipit-source-id: ad03b28bae937a35f8c20f12e0a5b0a27c34ff3b

Summary:
### Description
Add new signed 8 bit scalar quantizer, `QT_8bit_direct_signed` to ingest signed 8 bit vectors ([-128 to 127]).

### Issues Resolved
facebookresearch#3488

Pull Request resolved: facebookresearch#3501

Reviewed By: mengdilin

Differential Revision: D58639363

Pulled By: mdouze

fbshipit-source-id: cf7f244fdbb7a34051d2b20c6f8086cd5628b4e0

… cuvs-migrate

… cuvs-migrate

… cuvs-migrate