Skip to content

fix: shade protobuf-java, which Fess no longer ships - #3

Merged
marevol merged 1 commit into
mainfrom
fix/shade-protobuf-java
Sep 15, 2026
Merged

marevol merged 1 commit into
mainfrom
fix/shade-protobuf-java

Conversation

@marevol

@marevol marevol commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Found while verifying 15.9.0. On current Fess main the plugin cannot build a Cloud Storage client:

java.lang.NoClassDefFoundError: com/google/protobuf/MessageOrBuilder
	at com.google.cloud.storage.DefaultStorageRetryStrategy.<clinit>(DefaultStorageRetryStrategy.java:39)
	at com.google.cloud.storage.StorageRetryStrategy.getDefaultStorageRetryStrategy(StorageRetryStrategy.java:90)
	at com.google.cloud.storage.HttpStorageOptions$HttpStorageDefaults.getStorageRetryStrategy(HttpStorageOptions.java:345)
	at com.google.cloud.storage.HttpStorageOptions.<init>(HttpStorageOptions.java:75)
	...
	at com.google.cloud.storage.HttpStorageOptions$Builder.build(HttpStorageOptions.java:295)
Caused by: java.lang.ClassNotFoundException: com.google.protobuf.MessageOrBuilder

StorageOptions.newBuilder()...build() is where both users of the SDK start: GcsStorageClient (storage.type=gcs) and fess-crawler's GcsClient#init, which this plugin registers for gcs: URLs. With the plugin installed:

  • System > Storage shows a storage access error naming GcsStorageClient and the NoClassDefFoundError. An upload is answered with success, but no file appears.
  • A gcs:// file crawl indexes nothing; the failure URL records the error.
  • GET /api/admin/storage/list answers with status 0 and an empty list, so the failure does not show there at all.
  • From the second attempt in the same JVM on, the error only says Could not initialize class com.google.cloud.storage.DefaultStorageRetryStrategy; the name of the missing class is gone.

Cause

The shade's artifactSet excludes com.google.protobuf:protobuf-java, and the comment above the list says each entry is an artifact the war still ships. protobuf-java reached the war as a dependency of org.opensearch:opensearch: codelibs/fess#3438 excluded it there and codelibs/fess#3439 removed the opensearch jar altogether. Neither the war nor this jar has com.google.protobuf.* any more.

Change

  • Remove the protobuf-java exclude, so the SDK's protobuf-java 4.33.6 is shaded.
  • Correct the comment, which claimed the war ships every excluded artifact.

The whole exclude list was re-checked against mvn dependency:list -DincludeScope=runtime on current Fess main. Apart from protobuf-java, two more entries are no longer in the war: woodstox-core and stax2-api, which left with codelibs/fess#3430 (they came in through java-saml's xmlsec). They stay excluded, and the comment now says why:

  • Nothing in this jar uses them except jackson-dataformat-xml, and the SDK uses that only from XmlObjectParser and MultipartUploadHttpRequestManager, the machinery of the XML API's MultipartUploadClient.
  • A walk over the class references from GcsStorageClient, GcsClientCreator, fess-crawler's GcsClient and its gcs: URL handler never reaches jackson-dataformat-xml, and the storage round trip below passes without them.
  • woodstox-core registers itself in META-INF/services for javax.xml.stream.XMLInputFactory, XMLOutputFactory and XMLEventFactory. Shaded, it would become the StAX implementation of every Fess process that loads this jar.

The remaining version-skewed entries are still in the war, at the war's versions: gson 2.11.0 (SDK: 2.13.2), grpc-api and grpc-context 1.70.0 (1.82.2), error_prone_annotations 2.36.0 (2.48.0), j2objc-annotations 3.0.0 (3.1).

Why shade it rather than put it back in the war

Removing the OpenSearch server jar and what it brought in was deliberate, and Fess itself does not use protobuf-java. The Cloud Storage SDK in this plugin does: its HTTP transport still loads protobuf types, starting with the default retry strategy. Only an installation that uses GCS needs it.

Compatibility

The jar grows by 1.80 MiB (15,651,606 → 17,542,777 bytes): 756 classes under com/google/protobuf/ and the 12 google/protobuf/*.proto files. Nothing is removed, nothing is duplicated, there is no module descriptor at the root, and the shade's overlap warnings are unchanged. (META-INF/versions/9/module-info.class, from jackson-dataformat-xml and jackson-datatype-jsr310, was already in the jar and is untouched; the manifest has no Multi-Release attribute.)

Verification

  • mvn clean package: 1 test, 0 failures.
  • Linkage check on a class path made of Fess main's runtime jars (dependency:copy-dependencies -DincludeScope=runtime at codelibs/fess@5bd477e, 191 jars), Fess's own classes and the plugin jar:
    • before: StorageOptions.newBuilder()...getService() fails with NoClassDefFoundError: com/google/protobuf/MessageOrBuilder; new GcsStorageClient(...) and fess-crawler's GcsClient#init() then fail with Could not initialize class com.google.cloud.storage.DefaultStorageRetryStrategy;
    • after: all three succeed, and against a local fake-gcs-server GcsStorageClient runs ensureBucketExists, isAvailable, uploadObject, listObjects, downloadObject (identical bytes, non-ASCII object name), deleteObject and close without error.
  • Class-reference scan of the jar against the same class path: the com.google.protobuf group is gone. The groups that remain were there before and are untouched: the gRPC transport, the OpenTelemetry SDK, App Engine URL Fetch, GraalVM and the slf4j 2 SPI, which belong to transports and integrations the pom already excludes, and stax2 through jackson-dataformat-xml.
  • In the 15.9.0 verification, adding protobuf-java to WEB-INF/lib alone brought back the Storage page, the upload/list/download/delete round trip and the gcs:// crawl; gRPC, OpenTelemetry and Woodstox were not needed.

Building a Cloud Storage client fails on current Fess main with
NoClassDefFoundError: com/google/protobuf/MessageOrBuilder, raised from
DefaultStorageRetryStrategy.<clinit>. Both GcsStorageClient and
fess-crawler's GcsClient build through it, so the storage page, uploads
and gcs: crawling all break once the plugin is installed.

protobuf-java reached the war through org.opensearch:opensearch, which
codelibs/fess#3438 and codelibs/fess#3439 took out. Stop excluding it so
the SDK's protobuf-java 4.33.6 is shaded.

The comment claimed every excluded artifact is still in the war. That is
no longer true of woodstox-core and stax2-api either; they left with
codelibs/fess#3430. They stay excluded because only jackson-dataformat-xml
uses them and neither GcsStorageClient nor the gcs: crawler client reaches
it, and the comment now says so.
@marevol marevol added this to the 15.9.0 milestone Sep 15, 2026
@marevol marevol self-assigned this Sep 15, 2026
@marevol
marevol merged commit fc0cd2f into main Sep 15, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant