Azure AI Search response issue

[EilertSkram]

Discussed in #1446

^{Originally posted by EilertSkram September 3, 2024}
Hi,

Im getting an error when I add Azure Search to my extraBody, it works prior to this. Error message:

"error": {
        "type": "Object",
        "message": "Invalid chat message detected: message content must be string",
        "stack":
            
        "requestid": "reqId",
        "code": 400
      },
      "code": 400
    }

My env. :

MODELS=`[
{
  "id": id,
  "name": name,
  "displayName": displayname,
  "parameters": {
    "temperature": 0.5,
    "max_new_tokens": 4096
  },
  "endpoints": [
    {
      "type": "openai",
      "baseURL": "azure baseURL",
      "defaultHeaders": {
        "api-key": "key"
      },
      "defaultQuery": {
        "api-version": "2024-02-15-preview"
      },
      "extraBody": {
        "data_sources": [
          {
            "type": "azure_search",
            "parameters": {
              "endpoint": "endpoint",
              "index_name": "index",
              "role_information": "You are an assistant that provides answers based on the provided data sources whenever possible. If the data is not found or relevant, use your general knowledge to answer.",
              "in_scope": false,
              "authentication": {
                "type": "api_key",
                "key": "key"
              }
            }
          }
        ],
      }
    }
  ]
}
]`

I'm curious to why it fails, as the model's response in text format both when it finds files and when it doesn't. At first it seemed to be due to the title generation, but after disabling that, it seems to be an issue with it generation or index.

Ive verified the string output in a standalone js-file:

const baseURL = `${endpoint}/openai/deployments/${deployment_name}`;
const defaultHeaders = {
  "api-key": key,
};
const defaultQuery = {
  "api-version": version
};

const openai = new OpenAI({
  apiKey: key,
  baseURL: baseURL,
  defaultHeaders: defaultHeaders,
  defaultQuery: defaultQuery,
});

async function main() {
  const stream = await openai.chat.completions.create(
    {
      messages: [
        { role: "system", content: "" },
        { role: 'user', content: 'what is 1+1?' },
        {role: 'assistant', content: "2"},
        { role: 'user', content: 'What was my last question?' }

      ],
      model: deployment_name,
      data_sources: [
        {
          type: 'azure_search',
          parameters: {
            endpoint: search_endpoint,
            index_name: index,
            role_information: "You are an assistant that provides answers based on the provided data sources whenever possible. If the data is not found or relevant, use your general knowledge to answer. ",
            in_scope: false,
            authentication: {
              type: 'api_key',
              key: search_key,
            }
          }
        }
      ],
      stream: true, 
    }
  );

  // Generation
  for await (const chunk of stream) {
    process.stdout.write(chunk.choices[0].delta?.content || "");
  }
}

main();

Which yields the output: Your last question was: "what is 1+1?"%

After looking at index.tsand generate.ts I'm still unsure to why it fails. Does anyone have insights on this?