Tool Calling with OpenAI provider not working #9758
Description
I am currently using Onyx version 3.0.5 with two providers: Ollama for local model execution and OpenAI for accessing models on the IONOS AI Model Hub.
While tool calling woks as expected with the Ollama provider, I have encountered an issue when using the OpenAI provider with the Meta-Llama-3.1-405B-Instruct-FP8 model. Specifically, although the LLM response includes the tool calling request, Onyx fails to execute the tool calls.
To troubleshoot this issue, I have verified the functionality of tool calling with the Meta-Llama-3.1-405B-Instruct-FP8 model using the Opencode CLI client, and confirmed that it works as intended. This leads me to assume that the problem may be related to the integration of Onyx with the OpenAI provider, particularly with regards to tool calling.