logoalt Hacker News

hliyanyesterday at 2:55 AM1 replyview on HN

Wait, your title says this "runs locally"?


Replies

ArnavAgrawal03yesterday at 3:18 PM

Yes! If you're running the local version and it's taking long, that an indication that your GPU isn't being used properly. This can be traced back to the `colpali_embedding_model.py` file, where you can set the device and attention you want PyTorch to use.