Skip to content

Fix STS AssumeRole 501 error on S3-compatible storage by making inline policy optional #5425

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zhang-arvin wants to merge 1 commit into
base: main
Choose a base branch
from
+19 −8
Open

Fix STS AssumeRole 501 error on S3-compatible storage by making inline policy optional #5425

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions ...e/src/main/java/org/apache/polaris/core/storage/aws/AwsCredentialsStorageIntegration.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -222,16 +222,19 @@ static StorageAccessConfig compute(AwsStorageCredentialCacheKey key) {
.externalId(awsStorageConfig.getExternalId())
.roleArn(awsStorageConfig.getRoleARN())
.roleSessionName(roleSessionName)
.policy(
policyString(
awsStorageConfig,
key.allowedReadLocations(),
key.allowedListLocations(),
key.allowedWriteLocations(),
region)
.toJson())
.durationSeconds(storageCredentialDurationSeconds);

if (!Boolean.TRUE.equals(awsStorageConfig.getNoInlinePolicy())) {

@vigneshio vigneshio Sep 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once CatalogEntity actually copies the field, getNoInlinePolicy() will return the configured value; until then it’s always null, so the inline policy is still included. After the wiring is in place, please add a test that verifies AssumeRoleRequest omits the policy field when the no-inline-policy flag is true.

@zhang-arvin zhang-arvin Sep 7, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @vigneshio — agreed, the flag isn't effective until it's wired end-to-end. I'll add noInlinePolicy to the AwsStorageConfigInfo OpenAPI schema next to kmsUnavailable, map it bidirectionally in CatalogEntity.getAwsStorageConfigInfo() / getAwsStorageConfigurationInfo(), and add the AssumeRoleRequest test verifying Policy is omitted when the flag is true.

request.policy(
policyString(
awsStorageConfig,
key.allowedReadLocations(),
key.allowedListLocations(),
key.allowedWriteLocations(),
region)
.toJson());
}

List<Tag> sessionTags = key.sessionTags();
if (!sessionTags.isEmpty()) {
request.tags(sessionTags);
Expand Down
8 changes: 8 additions & 0 deletions ...s-core/src/main/java/org/apache/polaris/core/storage/aws/AwsStorageConfigurationInfo.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,14 @@ public URI getInternalEndpointUri() {
*/
public abstract @Nullable Boolean getKmsUnavailable();

/**
* Flag indicating whether to omit the inline session policy from STS AssumeRole requests. Set to
* {@code true} for S3-compatible STS implementations (e.g. VAST Data) that do not support inline

@flyingImer flyingImer Sep 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we confirm that Policy is the unsupported field on the reported VAST build before making its omission the compatibility switch? The successful CLI request in #5099 also uses a different role ARN format from the catalog config and omits ExternalId. VAST’s current guide for the 5.4.3+ flow lists Policy as supported. That does not prove the reported hotfix behaves the same, but it leaves the root cause unresolved. Do we have a request capture or controlled reproduction that holds the other fields constant and shows the 501 disappears when only Policy is removed?

@zhang-arvin zhang-arvin Sep 7, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@flyingImer we don't have a request capture from the reporter's VAST build beyond what's in #5099, and we can't reproduce against their deployed version ourselves, so we can't fully confirm Policy is the unsupported field from that capture alone. Given that, I'd rather keep the switch narrowly scoped to the inline policy (the only element shown to differ in the failing vs. succeeding capture) and treat it as a config-level compatibility option that defaults to current behavior. If maintainers prefer, I can also gate it so it only takes effect when explicitly enabled on S3-compatible storage. Happy to iterate.

* session policies. When enabled, the temporary credentials inherit the full permissions of the
* assumed role; ensure the IAM role has appropriate permissions for the required operations.
*/
public abstract @Nullable Boolean getNoInlinePolicy();

@vigneshio vigneshio Sep 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This property is not yet integrated into the OpenAPI schema or the entity mappings. Please add it adjacent tokmsUnavailablein the AwsStorageConfigInfo schema, and ensure it is mapped bidirectionally between CatalogEntity.getAwsStorageConfigInfo and getAwsStorageConfigurationInfo.

@zhang-arvin zhang-arvin Sep 7, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @vigneshio — agreed, the flag isn't effective until it's wired end-to-end. I'll add noInlinePolicy to the AwsStorageConfigInfo OpenAPI schema next to kmsUnavailable, map it bidirectionally in CatalogEntity.getAwsStorageConfigInfo() / getAwsStorageConfigurationInfo(), and add the AssumeRoleRequest test verifying Policy is omitted when the flag is true.


/** Endpoint URI for STS API calls */
@Nullable
public abstract String getStsEndpoint();
Expand Down