Repository navigation
bug: Quantized Models not Loading #105
Description
Activity
It seems to be an issue related to the download mechanism. If you check ~/.local/share/whispercpp, you will see the files being downloaded are much smaller than they should be. For now a solution is to download the models manually from https://huggingface.co/ggerganov/whisper.cpp/tree/main and place in this folder.
By the way, I am on Mac so on Windows and Ubuntu the folder path might be different. Try to figure out where the model files are getting downloaded to.
No, it is definitely not an issue with the download mechanism.
The model downloaded manually fails the same way.Fixed here
AIWintermuteAI@512426f
Need to build and install locally of course.@aarnphm are you still interested in maintaining the package? I see there is a cron job updating submodules, but it has no effect on build, since target commit is specified in rules/deps.bzl.
If you do not plan on maintaining the package, perhaps I will be able to at least set up a CI as GH Action to periodically test with new commits of whisper.cpp.
Describe the bug
When I pass the path of a quantized model into Whisper, it attempts to load the model however fails with the message
whisper_model_load: ERROR not all tensors loaded from model file - expected 245, got 3. I don't believe this is an issue with the bindings or whisper.cpp, but rather an issue with the version of whisper.cpp built by Bazel.A few lines before the error is printed,
whisper_model_load: f16 = 8is printed, howeverf16should beftypeaccording to this commit on whisper.cpp. I confirmed this by loading the quantized version into whisper.cpp, also confirming the quantized model is valid. I've built and installed the whisper.cpp bindings directly from source, so I am unsure why Bazel isn't building the right commit.To reproduce
from_pretrainedExpected behavior
The model loads and runs without error.
Environment
Python: 3.10.9
OS: WSL Ubuntu 22.04
Bazel: 6.0.0 (installed from the Bazel script in
./tools/)Bindings commit:
ca5863c04c9e0eb3359ef0330dde06747cef406cwhisper.cpp submodule commit:
14bee39b290f81381a259c53a169ffac8fcce987