Gauthier Robe
12/04/2025, 7:34 PMopenai_url property; it doesn't seem to be available for NLS?
I was hoping to use something like this:
{
"id": "NL-llama.cpp-ministral-3-14B",
"model_name": "openai/Ministral-3-14B-Instruct-2512-Q4_K_M.gguf",
"api_key": "NOT_NEEDED",
"openai_url": "http://xxxxxxxx:5000",
"max_bytes": 16000,
"temperature": 0,
"system_prompt": "Be precise and accurate in parsing queries"
}
Any other options? Thank you!Alan Martini
12/04/2025, 7:50 PMGauthier Robe
12/04/2025, 8:12 PMapi_url and no other changes: Request failed with HTTP code 400 | Server said: OpenAI API error: File Not Found
• using api_url and `vllm namspace`: Request failed with HTTP code 400 | Server said: vLLM API error: File Not Found
Interestingly, if I don't include api_url or openai_url then the error is Incorrect API key provided so it seems that including a URL does "something"Gauthier Robe
12/04/2025, 8:22 PM{
"id": "NL-llama.cpp-ministral-3-14B",
"model_name": "openai/Ministral-3-14B-Instruct-2512-Q4_K_M.gguf",
"api_key": "NOT_NEEDED",
"api_url": "http://xxxxx:5000/v1/chat/completions",
"max_bytes": 16000,
"temperature": 0,
"system_prompt": "Be precise and accurate in parsing queries"
}
/v1/chat/completions was missing.
Now the model is created successfully. I'll check if it works