Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to use Function wrappers in the new api ? #2

Open
100xbiz opened this issue Jul 7, 2024 · 0 comments
Open

How to use Function wrappers in the new api ? #2

100xbiz opened this issue Jul 7, 2024 · 0 comments

Comments

@100xbiz
Copy link

100xbiz commented Jul 7, 2024

Problem:
There is no clear example or documentation (currently outdated) on how to add function wrappers (are function
wrappers deprecated ?) that can dynamically accept these variables and use them in function callbacks during chat
prompt execution.

Request:
Please provide an example or extend the documentation to show how to configure function wrappers that can accept
dynamic variables during a chat session.

Example Scenario:
Here’s a simplified version of what I am trying to achieve, focusing on modifying booking dates:

Here there is a small modification; the customer before entering the chat session will provide the booking id, in a
form (no AI involved) and the static data like the IP address of the customer is passed by the calling application
and the :

Final req = Static data + AI input params

@Service
public class CustomerSupportAssistant {

    private final ChatClient chatClient;
    private final BookingService bookingService;

    @Autowired
    public CustomerSupportAssistant(
            ChatClient.Builder modelBuilder, 
            VectorStore vectorStore, 
            ChatMemory chatMemory, BookingService bookingService
    ) {
        this.bookingService = bookingService;

        this.chatClient = modelBuilder
                .defaultSystem("""
                        You are a customer chat support agent of an airline named "Funnair".
                        Respond in a friendly, helpful, and joyful manner.
                        You are interacting with customers through an online chat system.
                        Use the provided functions to change bookings.
                        Today is {current_date}.
                    """)
                .defaultAdvisors(
                        new PromptChatMemoryAdvisor(chatMemory),
                        new QuestionAnswerAdvisor(vectorStore, SearchRequest.defaults()))
                // removed the default functions to add dynamic input functions
                .build();
    }

    public Flux<String> chat(
            String chatId, 
            String userMessageContent, 
            Booking booking, // is now passed by the calling application / code
            String ipAddress // cannot be expected to be passed by customer in chat so is now passed by the calling application / code
    ) {
        var dynamicChatOptions = OpenAiChatOptions.builder()
                .withModel("gpt-3.5-turbo")
                .withMaxTokens(100)
                .withFunctionCallbacks(
                        List.of(
                                FunctionCallbackWrapper.builder(
                                        (ChangeBookingDatesRequest request) -> {
                                            bookingService.changeBooking(
                                                    booking.id(),
                                                    request.from(), // for validation if invalid dates are provided
                                                    request.to(), 
                                                    request.ipAddress() // security check or audit log
                                            );
                                            return "Booking dates changed successfully";
                                        })
                                        .withName("changeBooking")
                                        .withDescription("Change booking dates with additional parameters")
                                        .build()))
                .build();

        return this.chatClient.prompt()
                .withOptions(dynamicChatOptions)
                .system(s -> s.param("current_date", LocalDate.now().toString()))
                .user(userMessageContent)
                .advisors(a -> a
                        .param(CHAT_MEMORY_CONVERSATION_ID_KEY, chatId)
                        .param(CHAT_MEMORY_RETRIEVE_SIZE_KEY, 100))
                .stream().content();
    }
}

ERROR

When I try to implement the above approach, I'm getting the following error:

The code works and there is no error till I add the dynamicChatOptions to the chatClient.prompt() method.

my setup :

spring:
  ai:
    openai:
      api-key: ${OPEN_AI_KEY}
      chat:
        options:
          model: gpt-3.5-turbo-0125
{
  "error": {
    "message": "You didn't provide an API key. You need to provide your API key in an Authorization header using Bearer auth (i.e. Authorization: Bearer YOUR_KEY), or as the password field (with blank username) if you're accessing the API from your browser and are prompted for a username and password. You can obtain an API key from https://platform.openai.com/account/api-keys.",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant