Asynchronous Chat

[Fab8573]

Hello,

First, I would like to express my sincere thanks for sharing your source code.

Currently, I'm using the HemulGM wrapper for OpenAI, and your code will be invaluable as I integrate Mistral into my tests. With the HemulGM code, implementing asynchronous chat functionality is straightforward, as events are triggered when the result is ready. Specifically, these events include:

TOpenAIChat.OnChat: Triggered when the response is complete.

TOpenAIChat.OnBeginWork: Marks the start of the process.
TOpenAIChat.OnEndWork: Indicates the end of the process.
 => These events are particularly useful for displaying a wait cursor or similar indicators during processing

Can we imagine implementing something similar on your side?

Best regards,

[MaxiDonkey]

Hello,

First and foremost, thank you for your interest in this project. Your request is both insightful and greatly appreciated. I’ll make sure to add it to my list of pending tasks.

Currently, I am waiting for Mistral.AI to release the API extension for agent creation, which is a critical piece that I am eagerly anticipating.

In the meantime, if you’re looking to quickly establish an asynchronous mode, you could implement a "temporary" solution as follows:

Create methods:

procedure MyClass.DoOnWorkStart(Sender: TObject);
…
procedure MyClass.DoOnWorkEnd(Sender: TObject);
…
procedure MyClass.DoOnChat(Sender: TObject);
…

Using the first Chat (non-Stream) example, we can quickly create an asynchronous task as follows:

// system.Threading;

var Task: ITask := TTask.Create(
  procedure()
  begin
    TThread.Synchronize(TThread.Current,
      procedure
      begin
        var Chat := MistralAI.Chat.Create(
          procedure (Params: TChatParams)
          begin
            Params.Model('mistral-tiny');
            Params.Messages([TChatMessagePayload.User(Memo2.Text)]);
            Params.MaxTokens(1024);
          end);
        try
          for var Choice in Chat.Choices do
            Memo1.Lines.Add(Choice.Message.Content);
          DoOnChat(Chat);
        finally
          Chat.Free;
        end;
        DoOnWorkEnd(Self);
      end);
  end);
DoOnWorkStart(Self);
Task.Start;

I am aware that this is only a draft solution. I will dedicate more time to it as soon as I have finished the Wrapper for Delphi, which will allow the use of Anthropic APIs for its models.

Best regards,

[MaxiDonkey]

Hello,

A new version has been released to handle asynchronous chat (streamed and non-streamed).

Best regards