Skip to content

Incompatibility with python >= 3.9 -- repeating only one item #563

@Evarin

Description

@Evarin

Hello,

I am not able to use nmslib for py3.9 or 3.10, for a code that works very well in a conda+py3.8 install.

Steps to reproduce:

conda create -n testnmslib39 python=3.9 # adapt for 3.8 or 3.10
conda activate testnmslib39
pip install nmslib numpy ipykernel

And then the code:

import nmslib
import numpy as np

oracle = nmslib.init(method='hnsw', space='cosinesimil', data_type=nmslib.DataType.DENSE_VECTOR)
source = np.random.random((1000, 100)) * 2.0 - 1.0
oracle.addDataPointBatch(source)
oracle.createIndex(print_progress=True)

query = np.random.random((1, 100)) * 2.0 - 1.0
print(oracle.knnQuery(query))

With python 3.8, I get:

(array([919, 942, 534,  43, 368, 528,  22, 114, 302, 879], dtype=int32),
 array([0.64898074, 0.70981157, 0.7411968 , 0.75587684, 0.7559525 ,
        0.75614583, 0.76265025, 0.7647664 , 0.76834595, 0.76985985],
       dtype=float32))

With python 3.9 or 3.10, I get:

(array([171, 171, 171, 171, 171, 171, 171, 171, 171, 171], dtype=int32),
 array([0.6919109, 0.6919109, 0.6919109, 0.6919109, 0.6919109, 0.6919109,
        0.6919109, 0.6919109, 0.6919109, 0.6919109], dtype=float32))

Note that:

  • It installed nmslib-2.1.1-cp38-cp38-manylinux2010_x86_64.whl for 3.8, nmslib-2.1.1-cp39-cp39-manylinux2010_x86_64.whl for 3.9, and built from nmslib-2.1.1.tar.gz for 3.10
  • With python3.8 and 3.9, I get a warning "Your CPU supports instructions that this binary was not compiled to use: SSE3 SSE4.1 SSE4.2 AVX AVX2", but not with 3.10 (seems coherent with the facts that it had to recompile because no binary is available).
  • Recompiling on 3.10 with CFLAGS=-msse2 to get the same warning did not solve the issue. On the contrary, recompiling on 3.8 with -march=native did not yield any issue.
  • I have the same issue without conda and python3.10 virtualenv
  • The results are the same with knnQueryBatch or with knnQuery, and with any parameters inside createIndex and on QueryTimeParams (efS, M, post...)
  • It seems really similar to sw-graph Returns the Same Results Over and Over again #561, but as I did more thorough examination and easiest minimal installation, I thought I'd give it more visibility creating a new issue

My computer&distro infos :

Kernel: 6.8.0-52-generic x86_64 bits: 64 compiler: N/A Desktop: Cinnamon 6.0.4 tk: GTK 3.24.33
    wm: muffin vt: 7 dm: LightDM 1.30.0 Distro: Linux Mint 21.3 Virginia base: Ubuntu 22.04 jammy
CPU:
  Info: 6-core model: Intel Core i7-10750H bits: 64 type: MT MCP smt: enabled arch: Comet Lake
    rev: 2 cache: L1: 384 KiB L2: 1.5 MiB L3: 12 MiB
  Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions