Commits on Nov 20, 2024

1. Faiss GPU: bfloat16 brute-force kNN support (facebookresearch#4018) …

   Summary:
   
   
   This diff adds support for bfloat16 vector/query data types with the GPU brute-force k-nearest neighbor function (`bfKnn`).
   
   The change is largely just plumbing the new data type through the template hierarchy (so distances can be computed in bfloat16).
   
   Of note, by design, all final distance results are produced in float32 regardless of input data type (float32, float16, bfloat16). This is because the true nearest neighbors in many data sets can often differ by only ~1000 float32 ULPs in terms of distance which will result in possible false equivalency. This seems to be one area where lossy compression/quantization thoughout does not work as well (and is also why `CUBLAS_MATH_DISALLOW_REDUCED_PRECISION_REDUCTION` is set in `StandardGpuResources.cpp`. However, given that there is native bf16 x bf16 = fp32 tensor core support on Ampere+ architectures, the matrix multiplication itself should use them.
   
   As bfloat16 support is quite lacking on AMD/ROCm (see [here](https://rocm.docs.amd.com/projects/HIPIFY/en/latest/tables/CUDA_Device_API_supported_by_HIP.html), very few bf16 functions implemented), bf16 functionality is completely disabled / not compiled for AMD ROCm.
   
   Reviewed By: mdouze
   
   Differential Revision: D65459723

   

   Jeff Johnson authored and facebook-github-bot committed Nov 20, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for 95728a3
   • Browse repository at this point

   Copy the full SHA

   95728a3 View commit details

   Browse the repository at this point in the history