fix: shade reactive-streams, which Fess no longer ships - #1
Merged
Merged
Conversation
Building an S3 client fails on current Fess main with NoClassDefFoundError: org/reactivestreams/Publisher, raised from DefaultS3BaseClientBuilder.finalizeServiceConfiguration. Both S3StorageClient and fess-crawler's S3Client build through it, so the storage page, the storage API and s3: crawling all break once the plugin is installed. The shade excluded org.reactivestreams:* on the assumption that the war carries it with OpenSearch. It did until codelibs/fess#3439 dropped the opensearch jar. Stop excluding it so the SDK's reactive-streams 1.0.4 is shaded, and correct the comment. The remaining excludes (httpclient5, httpcore5, httpcore5-h2, slf4j-api) are still shipped by the war.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while verifying 15.9.0. On current Fess main the plugin cannot build an S3 client at all:
Both users of the SDK go through that builder:
S3StorageClient(storage.type=s3) and fess-crawler'sS3Client#init, which this plugin registers fors3:URLs. With the plugin installed:GET /api/admin/storage/listanswers with status 2.s3://file crawl indexes nothing while the job reports success; the failure URL carries the stack above.Cause
The shade's
artifactSetexcludesorg.reactivestreams:*, on the assumption written next to it that "reactive-streams arrives there with OpenSearch". It did: reactive-streams 1.0.4 was a direct dependency oforg.opensearch:opensearchand so sat inWEB-INF/lib. codelibs/fess#3439 dropped the opensearch jar, and reactive-streams went with it. Neither the war nor this jar hasorg.reactivestreams.*any more.Change
org.reactivestreams:*exclude, so the SDK's reactive-streams 1.0.4 (it comes in throughhttp-auth-spi) is shaded.The other excludes were re-checked against
mvn dependency:list -DincludeScope=runtimeon current Fess main and are all still shipped by the war: httpclient5 5.6.1, httpcore5 and httpcore5-h2 5.4, slf4j-api 1.7.36.Why shade it rather than put it back in the war
Removing the OpenSearch server jar and what it brought in was deliberate, and Fess itself does not use reactive-streams; the AWS SDK in this plugin does. reactive-streams 1.0.4 is an 11.6 KB jar of four interfaces and the
FlowAdaptersbridge, with no dependencies. fess-ds-microsoft365 needs it for the same reason and shades the same version; the two copies are byte-identical (same CRC-32 for all 14 entries), so on a class path that has both plugins, whichever copy is found first is the same class.Compatibility
The jar grows by 10,992 bytes (8,728,399 → 8,739,391). The only new entries are the 14 under
org/reactivestreams/; nothing is removed, nothing is duplicated, there is no module descriptor at the root, and the shade's overlap warnings are unchanged.Verification
mvn clean package: 4 tests, 0 failures.dependency:copy-dependencies -DincludeScope=runtimeat codelibs/fess@5bd477e, 191 jars), Fess's own classes and the plugin jar:S3Client.builder()...build(),new S3StorageClient(...)and fess-crawler'sS3Client#init()each fail withNoClassDefFoundError: org/reactivestreams/Publisher;S3StorageClientrunsensureBucketExists,isAvailable,uploadObject,listObjects,downloadObject(identical bytes, non-ASCII key),deleteObjectandclosewithout error.org.reactivestreamsand the optional AWS CRT client (software.amazon.awssdk.crt, which neither this plugin nor Fess has ever shipped); after, only the CRT references remain.reactive-streams-1.0.4.jartoWEB-INF/libalone brought back the Storage page, upload, list, download and delete, and thes3://crawl.