Skip to content

Add a provider-neutral representation for streaming function-call updates #7753

Description

@danroth27

Background and motivation

AI providers can stream a function call's name and arguments incrementally. Microsoft.Extensions.AI currently exposes the completed call as FunctionCallContent, while incremental argument fragments are generally available only through provider-specific ChatResponseUpdate.RawRepresentation.

This requires middleware and UI integrations to understand each provider's native SDK. The same problem also occurs for protocol-backed IChatClient implementations: for example, an AG-UI client receives TOOL_CALL_START and TOOL_CALL_ARGS events but has no provider-neutral MEAI content type through which to expose them.

A common abstraction would allow provider clients and protocol adapters to represent the same lifecycle:

provider or protocol update
-> FunctionCallUpdateContent
-> eventual completed FunctionCallContent

This would support predictive UI, MCP Apps, progress displays, logging, and other consumers that need to react while arguments are being generated.

Related scenario: ag-ui-protocol/ag-ui#2245.

API Proposal

namespace Microsoft.Extensions.AI;

public sealed class FunctionCallUpdateContent : AIContent
{
    public FunctionCallUpdateContent(string callId);

    public string CallId { get; }

    public string? Name { get; set; }

    public string? ArgumentsDelta { get; set; }
}

FunctionCallUpdateContent represents an incremental update only. ArgumentsDelta is an arbitrary text fragment and is not required to contain valid JSON.

Providers and protocol adapters would continue emitting the existing FunctionCallContent when the call is complete.

API Usage

await foreach (ChatResponseUpdate update in client.GetStreamingResponseAsync(messages))
{
    foreach (var callUpdate in update.Contents.OfType<FunctionCallUpdateContent>())
    {
        UpdateToolProgress(
            callUpdate.CallId,
            callUpdate.Name,
            callUpdate.ArgumentsDelta);
    }

    foreach (var call in update.Contents.OfType<FunctionCallContent>())
    {
        // The call is complete and its arguments are parsed.
    }
}

A provider integration could emit:

FunctionCallUpdateContent(call_1, name: "write_document")
FunctionCallUpdateContent(call_1, argumentsDelta: "{\"document\":\"Once")
FunctionCallUpdateContent(call_1, argumentsDelta: " upon a time\"}")
FunctionCallContent(call_1, "write_document", completeArguments)

FunctionInvokingChatClient should continue invoking only completed FunctionCallContent instances.

Alternative Designs

Continue using RawRepresentation. This requires provider-specific casts and prevents middleware from working across providers and protocol-backed clients.

Define an AG-UI-specific content type. The concept is not specific to AG-UI; native provider clients expose the same incremental lifecycle.

Expose only an accumulated partial JSON string. Individual fragments more accurately represent provider behavior and allow consumers to choose their own accumulation or parsing strategy.

Risks

  • Consumers may incorrectly treat individual argument fragments as valid JSON.
  • Providers differ in when they supply call IDs and function names.
  • Parallel calls require providers to maintain stable call correlation.
  • Aggregation must not cause a function to execute before the completed FunctionCallContent is available.

The new content should be additive and should not change existing coalescing or function-invocation behavior.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    api-suggestionEarly API idea and discussion, it is NOT ready for implementationuntriaged

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions