Skip to content

gRPC client with AddStandardResilienceHandler reports Unavailable instead of Cancelled when the caller cancels during a transient failure #7741

Description

@gitmln

Description

A gRPC client configured with AddGrpcClient(...).AddStandardResilienceHandler() and cancelled by the caller while an attempt is in flight fails with StatusCode.Unavailable if that attempt completes with a transient HTTP failure. It should be StatusCode.Cancelled.

Any failure the retry strategy would have retried triggers it (with the default HttpRetryStrategyOptions that's 5xx, 408, 429 and HttpRequestException). The reported status varies with the failure, but it's never Cancelled.

The cause is in the Polly version pinned in eng/packages/General.props (8.4.2). Its RetryResilienceStrategy returns the last, failed outcome instead of throwing when the token is cancelled:

if (context.CancellationToken.IsCancellationRequested || isLastAttempt || !handle)
{
    return outcome;
}

Polly changed this in 8.5.2 (App-vNext/Polly#2456): cancellation now surfaces as OperationCanceledException.

Reproduction Steps

The simplest console app: https://github.com/gitmln/DotnetExtensionsRepro

dotnet run                          # Polly 8.4.2, as pinned by Microsoft.Extensions.Http.Resilience 9.10.0
dotnet run -p:PollyVersion=8.7.0

The relevant part:

services
    .AddGrpcClient<GreeterClient>(options => options.Address = new Uri("https://dummy"))
    .ConfigurePrimaryHttpMessageHandler(() => new CancelThenFailHandler(cts))
    .AddStandardResilienceHandler();

// CancelThenFailHandler: cancel the caller's token, then complete the attempt with a 503
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
    cts.Cancel();
    return Task.FromResult(new HttpResponseMessage(HttpStatusCode.ServiceUnavailable) { Version = HttpVersion.Version20 });
}

await client.SayHelloAsync("dummy", cts.Token);   // RpcException — which StatusCode?

The call fails with Cancelled only on Polly 8.7.0.

The same scenario as a test in this repo, using the generated Greeter.GreeterClient and the existing GrpcResilienceTests fixture: gitmln/dotnet-extensions@ab484b9. On main (0fe52df) both the sync and async variants fail; with the Polly packages at 8.7.0 (gitmln/dotnet-extensions@754ada5) they pass.

Expected behavior

RpcException with StatusCode.Cancelled.

Actual behavior

RpcException with StatusCode.Unavailable.

Regression?

No. Present for as long as a Polly older than 8.5.2 has been referenced.

Known Workarounds

None that I can see.

Configuration

  • main at 0fe52df, Polly 8.4.2 via eng/packages/General.props
  • net8.0 on Microsoft.NETCore.App / AspNetCore.App 8.0.30; SDK 10.0.400 (global.json rollForward relaxed locally), Windows 11 10.0.26200, win-x64

Other information

I've seen #7719. This isn't an argument about which Polly to ship long term — only that the pinned one has a user-visible bug in a documented scenario. Branch with the test and the bump, ready for a PR if that's useful: https://github.com/gitmln/dotnet-extensions/tree/polly-retry-cancellation-issue

I used AI/Claude Code for the investigation and to draft the test

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugThis issue describes a behavior which is not expected - a bug.untriaged

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions