Ags web surfer

[husseinmozannar]

Why are these changes needed?

Adding WebSurfer into AutoGen Studio.

• enable adding WebSurfer agent to any team
• enable viewing websurfer text messages in the ags ui
• enable viewing websurfer multimodal messages in ui.

Websurfer agent can be added using the AGS team spec as follows:

{
  "version": "1.0.0",
  "component_type": "team",
  "name": "websurfer_team",
  "participants": [
    {
      "component_type": "agent",
      "name": "websurfer_agent",
      "agent_type": "MultimodalWebSurfer",
      "model_client": {
        "component_type": "model",
        "model": "gpt-4o-2024-08-06",
        "model_type": "OpenAIChatCompletionClient"
      },
      "headless": true,
      "logs_dir": "logs",
      "downloads_folder": "logs",
      "to_save_screenshots": false,
      "use_ocr": false,
      "animate_actions": false
    }
  ],
  "team_type": "RoundRobinGroupChat",
  "termination_condition": {
    "component_type": "termination",
    "termination_type": "MaxMessageTermination",
    "max_messages": 3
  }
}

Related issue number

Closes #4290

[husseinmozannar]

Some current issues:

• width of websurfer messages because of page metadata breaks the view, it becomes full width and hides the agent graph viz

Not related to this, but things to fix later:

• multi-line input
• sessions should be on the LHS side bar, there should be a top toggle for the conditions, to make it look more in lign with current chat UIs out there.
• definition of model client is kinda not great, you want to define it somewhere separately or have a default so that you easily add it to the agent.

[victordibia]

Looking good.

• Fixed width issue .. also enabled text truncation and ability to view images fullscreen.

Other issues

• will open an issue to explore UX for sessions on the LHS
• multiline input is already supported .. is press shift + enter in the textbox, you get multiline
• there is an issue to enable drag and drop definition of components including models .. Implement a drag and drop team builder interface for AGS #4202
• there is an issue to improve how runs are cancelled Use External termination condition to stop execution in AGS #4376 in AGS

Overall, we should define some protocol for showing tracking files generated by agents. Currently we stuff the b64 string of an image in the message itself. It would be useufl to have a protocol eg., for storing files to some location and passing in the url and the ui can view those files (ideally in a user owned directory mounted by the web api) (cc @ekzhu ).