<@U01PL2YSG8L> I think this is a somewhat importan...
# community-help
s
@Kishore Nallan I think this is a somewhat important issue: https://github.com/typesense/typesense/issues/2652 Please refer to my last comment there.
k
The huggingface downloads have become flaky it seems. We will add some corrective mechanisms.
πŸ‘ 1
s
A status would really be the best. Imagine you're building an app around typesense (say, a document management system for example, dockerized) The user will install docker and it'll pull all images - inclusive ts And then he'll upload docs… and either get errors or hidden failures if the models aren't available yet And it’s impossible for us dev to say β€žmodel is pullingβ€œ or even β€žit's missingβ€œ because we'd have to scan the file on disk to know Now, a tiny Status on the Health endpoint saying models[β€žβ€œ] would basically make live a lot easier
Or even just models = true|false At least we've something to work with
@Kishore Nallan - what is your recommendation in the interim? I literally got a couple dozen users complaining my app to be broken, when in reality it is the model not pulling (and then, sometimes it does, out of the blue) I am a bit at a loss how I can resolve this other than telling the user "exec into docker, pull the model" There really should be a progress, status, something we can use (and based upon, force-retry, and set different, more sensible timeouts. 4 seconds is strangely short)
Strike that, it is impossible to even manually fix that
Copy code
/path/to/host/folder/mapped/to/dockerfor/typesense-data/models tree
.
└── ts
    β”œβ”€β”€ clip-vit-b-p32
    β”‚   β”œβ”€β”€ clip_image_processor.onnx
    β”‚   β”œβ”€β”€ clip_tokenizer.onnx
    β”‚   β”œβ”€β”€ config.json
    β”‚   β”œβ”€β”€ model.onnx
    β”‚   └── vocab.txt
    └── e5-small-v2
        β”œβ”€β”€ config.json
        β”œβ”€β”€ model.onnx
        └── vocab.txt

4 directories, 8 files
yet TS will still try to download the models, fail, and say it cant use them. Extremely frustrating, really. Not even as "fake custom models" one can load them.
k
I will increase the default timeout to 30 seconds. The problem is that on startup, we have to load the model before we can begin indexing. If the huggingface API is just having trouble or getting throttled either we have to totally keep retrying or proceed to load the collection without the models, which is what we do today. However, even if we provide a status API, only way to restore the models are by restarting Typesense at the moment.
πŸ™Œ 1
Fix available in
30.2.rc6
-- there is a 30 second timeout now.
πŸš€ 1
s
@Kishore Nallan I just installed
Copy code
typesense:30.2.rc6
I still get these:
Copy code
typesense-1  | E20260317 20:03:41.202178   593 http_client.cpp:228] CURL timeout. Time taken: 4.00107, method: GET, url: https://huggingface.co/typesense/models/resolve/main/e5-small-v2/config.json
typesense-1  | E20260317 20:03:41.203266   593 embedder_manager.cpp:26] Failed to initialize local model ts/e5-small-v2, error: Model not found
worker-1     | 20:03:41 Successfully completed app.worker.tasks.process_document_task('7d404439-2daa-4607-a204-6e5812d699cc') job in 0:01:12.506816s on worker 47c5f990ff22451db18fe9849405dafe
worker-1     | 20:03:41 dcm: Job OK (fe14ead1-b209-4487-ab8b-af29fe2e3b10)
worker-1     | 20:03:41 Result is kept for 500 seconds
Yes, I am on a VPN, yes, my ping is horrible (some 300ms), but this should simply in no way affect the download. It has to wait until it is done. I download large stuff all the time (literally, a whole *arr stack is pulling 4k movies and stuff all day long lol) I also still see a timeout at 4s, instead of 30 My docker compose says:
image: typesense/typesense:30.2.rc6
And I saw it being pulled as I did docker compose up Not sure what I am doing wrong here.
The very least I think would be either of or all of: β€’ custom timeout setting β€’ endpoint to poll for readiness, no timeout, just poll until the model is here! β€’ instruction how to add it manually instead However I really think the endpoint is the only real solid solution. This way, a program integrating typesense can poll it, await and if ready, startup (or throw some sort of warning to the user at least)
k
I suspect that the model configuration fetch API is also timing out. I will also patch this. That should complete the entire life cycle of fetches for models. I will share a build with that shortly.
πŸ™ 1
Please check
30.2.rc9
πŸ‘€ 1
s
Its weird. I have replaced the image tag, docker down and up (which rebuilt since my image isnt tagged locally) and pulled ts properly However I saw no reference to something from hugginfface at all in the logs - only somet things like these:
Copy code
typesense-1  | I20260319 13:38:51.684044   455 batched_indexer.cpp:432] Running GC for aborted requests, req map size: 1, reference_q.size: 0
typesense-1  | I20260319 13:39:00.701645   454 raft_server.cpp:759] Term: 12, last_index: 61, committed: 61, known_applied: 61, applying: 0, pending_writes: 0, queued_writes: 1, inflight_searches: 0, local_sequence: 153
typesense-1  | I20260319 13:39:00.701742   603 raft_server.h:60] Peer refresh succeeded!
typesense-1  | I20260319 13:39:10.702626   454 raft_server.cpp:759] Term: 12, last_index: 61, committed: 61, known_applied: 61, applying: 0, pending_writes: 0, queued_writes: 1, inflight_searches: 0, local_sequence: 153
typesense-1  | I20260319 13:39:10.702680   519 raft_server.h:60] Peer refresh succeeded!
typesense-1  | I20260319 13:39:20.703769   454 raft_server.cpp:759] Term: 12, last_index: 61, committed: 61, known_applied: 61, applying: 0, pending_writes: 0, queued_writes: 1, inflight_searches: 0, local_sequence: 153
typesense-1  | I20260319 13:39:20.704027   523 raft_server.h:60] Peer refresh succeeded!
typesense-1  | I20260319 13:39:30.704882   454 raft_server.cpp:759] Term: 12, last_index: 61, committed: 61, known_applied: 61, applying: 0, pending_writes: 0, queued_writes: 1, inflight_searches: 0, local_sequence: 153
typesense-1  | I20260319 13:39:30.705044   600 raft_server.h:60] Peer refresh succeeded!
typesense-1  | I20260319 13:39:40.705996   454 raft_server.cpp:759] Term: 12, last_index: 61, committed: 61, known_applied: 61, applying: 0, pending_writes: 0, queued_writes: 1, inflight_searches: 0, local_sequence: 153
typesense-1  | I20260319 13:39:40.706101   603 raft_server.h:60] Peer refresh succeeded!
typesense-1  | I20260319 13:39:50.365322   555 text_embedder.cpp:20] ONNX shared libs: off
typesense-1  | I20260319 13:39:50.365353   555 text_embedder.cpp:33] Loading model from disk: /data/models/ts_clip-vit-b-p32/model.onnx
typesense-1  | I20260319 13:39:50.707489   454 raft_server.cpp:759] Term: 12, last_index: 61, committed: 61, known_applied: 61, applying: 0, pending_writes: 0, queued_writes: 1, inflight_searches: 0, local_sequence: 153
typesense-1  | I20260319 13:39:50.707552   519 raft_server.h:60] Peer refresh succeeded!
typesense-1  | I20260319 13:39:51.276821   555 image_processor.cpp:8] Loading image processor from /data/models/ts_clip-vit-b-p32/clip_image_processor.onnx
typesense-1  | I20260319 13:39:51.280105   555 embedder_manager.cpp:24] Finished initializing local model: ts/clip-vit-b-p32
typesense-1  | I20260319 13:39:51.280119   555 field.cpp:889] Model init done.
typesense-1  | I20260319 13:39:52.690093   455 batched_indexer.cpp:432] Running GC for aborted requests, req map size: 0, reference_q.size: 0
As if the model already where on local disk But that could only be possible if the model was pulled in the background overnight, since yesterday it clearly was not there despite repeated down/up (always failed at the 4s timeout) I am now going to whipe out local host folder, I guess, and start fresh to see if it works
I can confirm models seem to work now, just that it is extremely confusing in the logs it says:
Copy code
typesense-1  | I20260319 13:46:29.595633   583 text_embedder.cpp:33] Loading model from disk: /data/models/ts_e5-small-v2/model.onnx
typesense-1  | I20260319 13:46:29.786890   583 embedder_manager.cpp:24] Finished initializing local model: ts/e5-small-v2
typesense-1  | I20260319 13:46:29.786909   583 field.cpp:889] Model init done.
typesense-1  | I20260319 13:46:29.792979   570 embedder_manager.cpp:21] Validating and initializing local model: ts/clip-vit-b-p32
typesense-1  | I20260319 13:46:33.584232   453 raft_server.cpp:759] Term: 2, last_index: 3, committed: 3, known_applied: 3, applying: 0, pending_writes: 0, queued_writes: 1, inflight_searches: 0, local_sequence: 6
typesense-1  | I20260319 13:46:33.584306   497 raft_server.h:60] Peer refresh succeeded!
typesense-1  | I20260319 13:46:43.585709   453 raft_server.cpp:759] Term: 2, last_index: 3, committed: 3, known_applied: 3, applying: 0, pending_writes: 0, queued_writes: 1, inflight_searches: 0, local_sequence: 6
typesense-1  | I20260319 13:46:43.585795   507 raft_server.h:60] Peer refresh succeeded!
typesense-1  | I20260319 13:46:44.571918   454 batched_indexer.cpp:432] Running GC for aborted requests, req map size: 1, reference_q.size: 0
This despite the fact that the model is not there when container starts up (after total wipe) So it appears you only log the model pulling process on failure, I guess. Bottom line, it now has pulled the models twice reliably which should mean it is solved.
k
Excellent, good to hear that it's working now.