refactor: compile against the forked OpenSearch package and drop the opensearch jar - #3439
Merged
Merged
Conversation
…client Fess shipped org.opensearch:opensearch (17.9 MB) purely for its request and response types and its query builders; it never runs a node. Those classes are now forked and pruned into org.codelibs.fesen.opensearch.* inside fesen-httpclient, so the dependency and its exclusion block go away and every org.opensearch import moves to the forked package. Apache Lucene stays as the real org.apache.lucene and now arrives through fesen-httpclient, which keeps all thirteen modules Fess resolved before. Fess's own org.codelibs.fess.opensearch.* (DBFlute/ESFlute output) is unrelated and untouched. Three things are not a plain rename: - QueryShardContext is deliberately absent from the fork, so QueryBuilder no longer declares toQuery and AbstractQueryBuilder no longer declares doToQuery. The overrides in KnnQueryBuilder, HybridQueryBuilder, StoredLtrQueryBuilder and DefaultQueryBuilder only threw UnsupportedOperationException or delegated, so they are removed along with the now-unused imports. These builders exist to be serialised to JSON, which is unaffected. KnnQueryBuilderTest's test_doToQuery_isUnsupported covered a method that no longer exists and goes with it. - The seeded Index Exporter job embeds a class name in its script body, so scheduled_job.bulk moves to org.codelibs.fesen.opensearch.index.query. QueryBuilders too. The startup reload is create-only, so an existing installation keeps its stale copy and needs a separate migration. - IndexingHelper.deleteDocumentByQuery and LanguageHelper.createScript are part of the plugin-facing API and now take and return forked types; the plugins that call them migrate separately. Removing the dependency drops opensearch-3.8.0.jar and its twelve siblings plus reactive-streams, 17.84 MiB off the resolved runtime classpath (229 jars to 215).
Removing org.opensearch:opensearch took SnakeYAML off the resolved classpath at 2.6 and left jackson-dataformat-yaml to supply 2.5 instead. ThemeManifest parses theme.yaml with SnakeYAML directly, so the version it gets should not be decided by an unrelated Jackson module: declare it, at the release it has been running. The log4j2 entry that silenced org.opensearch.bootstrap no longer names anything. The forked tree has no bootstrap package - the node runtime was cut - and the original jar is gone, so the logger cannot match. Remove it rather than leave a rule pointing at a package that does not exist. 7469 tests pass.
This was referenced Sep 13, 2026
marevol
added a commit
that referenced
this pull request
Sep 14, 2026
#3439 left two things behind. `SearchEngineClient.pitSegments` overrides a `Client` method that the fork does not have. The package rename was applied to every `org.opensearch.*` reference without checking that each type survived: `PitSegmentsRequest` did not, so the import resolves to nothing and the method is an `@Override` of nothing. Removed, with the two imports it needed. `AbstractDocumentSearcherTest` arrived from #3440 after the migration branch was cut, so its six `org.opensearch.*` imports were never rewritten. They are now. Nothing else in the tree still names `org.opensearch`.
This was referenced Sep 15, 2026
fix(deps): restore zstd-jni so .zst and .tar.zst files are extracted again
codelibs/fess-crawler#211
Merged
marevol
added a commit
to codelibs/fess-ds-microsoft365
that referenced
this pull request
Sep 15, 2026
Every data store fails in Microsoft365Client's constructor on current Fess main with NoClassDefFoundError: org/reactivestreams/Publisher, raised while azure-identity's CredentialBuilderBase creates its ClientLogger. reactive-streams is on the shade's include list, but it resolved as provided under the provided org.opensearch:opensearch dependency, so the shade never took it. The war used to supply it next to the opensearch jar until codelibs/fess#3439 removed that jar. Declare reactive-streams 1.0.4, the version reactor-core declares, as a direct dependency so it resolves at compile scope and is shaded. Excluding it from the opensearch dependency does not work (reactor-core's copy sits under the same node), and dropping that dependency changes the resolved versions of reactor-core, error_prone_annotations and jna.
marevol
added a commit
to codelibs/fess-storage-gcs
that referenced
this pull request
Sep 15, 2026
Building a Cloud Storage client fails on current Fess main with NoClassDefFoundError: com/google/protobuf/MessageOrBuilder, raised from DefaultStorageRetryStrategy.<clinit>. Both GcsStorageClient and fess-crawler's GcsClient build through it, so the storage page, uploads and gcs: crawling all break once the plugin is installed. protobuf-java reached the war through org.opensearch:opensearch, which codelibs/fess#3438 and codelibs/fess#3439 took out. Stop excluding it so the SDK's protobuf-java 4.33.6 is shaded. The comment claimed every excluded artifact is still in the war. That is no longer true of woodstox-core and stax2-api either; they left with codelibs/fess#3430. They stay excluded because only jackson-dataformat-xml uses them and neither GcsStorageClient nor the gcs: crawler client reaches it, and the comment now says so.
marevol
added a commit
to codelibs/fess-storage-s3
that referenced
this pull request
Sep 15, 2026
Building an S3 client fails on current Fess main with NoClassDefFoundError: org/reactivestreams/Publisher, raised from DefaultS3BaseClientBuilder.finalizeServiceConfiguration. Both S3StorageClient and fess-crawler's S3Client build through it, so the storage page, the storage API and s3: crawling all break once the plugin is installed. The shade excluded org.reactivestreams:* on the assumption that the war carries it with OpenSearch. It did until codelibs/fess#3439 dropped the opensearch jar. Stop excluding it so the SDK's reactive-streams 1.0.4 is shaded, and correct the comment. The remaining excludes (httpclient5, httpcore5, httpcore5-h2, slf4j-api) are still shipped by the war.
marevol
added a commit
to codelibs/fess-docs
that referenced
this pull request
Sep 15, 2026
…fig and plugin upgrades (#536) An installation that upgrades from 15.8 by following install/upgrade.rst still ends up with several things broken that the guide did not mention. All seven languages of the development tree (15.9) are updated. - Groovy: every job 15.8 seeded is stored as groovy and the startup seeding is create-only, so Default Crawler and the other bundled jobs fail without fess-script-groovy, mostly visible only in fess.log. The section now lists every kind of stored setting that 15.9 evaluates as Groovy and what each does without the plugin, tells upgraders to look for the startup warning, and gives both ways out: install the plugin, or switch each kind to JavaScript with the field that selects its engine (Thumbnail Purger's 1000L literal needs an edit). - Index Exporter: the stored script names org.opensearch.index.query.QueryBuilders, removed by codelibs/fess#3439; give the replacement script and fix the examples in config/admin-index-export.rst. - Carried-over configuration: ZIP Step 3 copies fess_config.properties and fess.in.sh, and RPM keeps a modified fess_config.properties. List the keys whose 15.8.0 value now misbehaves (job.default.script, job.template.script, crawler.file.protocols, search_engine.http.url, the jcifs options, removed keys) and what a 15.8 fess.in.sh lacks. Say that DEB overwrites the file without prompting or a backup. - jcifs 3.x property names (codelibs/fess#3433): SMB connect and session timeouts now take effect, 35 s to 60 s. - The four keys removed by codelibs/fess#3435. - Plugins: bin/fess-setup upgrade plugins and check; what a Playwright crawl configuration does without the plugin. - Replace the ZIP size figure, which did not match the released fess-15.8.0.zip, with the Playwright driver bundle's share of it. The startup warning, the Playwright crawler warning and path mappings being left unapplied describe codelibs/fess#3451, #3445 and #3452.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fesen-httpclientnow carries the OpenSearch DTOs and query builders Fess needs underorg.codelibs.fesen.opensearch.*, pruned from 8,578 classes to 3,751. This drops theorg.opensearch:opensearchdependency and moves Fess onto them.WEB-INF/libgoes from 229 jars / 146,843,781 bytes to 215 jars / 135,178,555 bytes — 11.67 MiB off every distribution. The 17.09 MiB server jar and its twelve small siblings leave, along withreactive-streams;fesen-httpclientgrows from 0.44 MiB to 7.09 MiB. Noorg/opensearch/class remains anywhere in the distribution — all 219 jars were swept.All 13 Lucene modules stay at 10.5.0. The old exclusion block had been suppressing eight of them; the fork declares them directly, so they come back.
The change
264 files, 2,049 import lines. 142 of the 210 main files are checked-in ESFlute output, so 81% of the occurrences are generated code.
src/main/webapphas none. Fess's ownorg.codelibs.fess.opensearch.*package is untouched — the rewrite was anchored so it could not match it.Three things were not a rename:
QueryShardContextis gone from the fork — it was the bridge whose removal collapsedindex/mapper/**,index/fielddata/**andsearch/internal/**. Four classes overrode a method taking it:KnnQueryBuilder,HybridQueryBuilder,StoredLtrQueryBuilderandDefaultQueryBuilder. All four bodies threwUnsupportedOperationExceptionor delegated to one that did — these builders exist to be serialised to JSON, which still works — so the overrides are deleted.KnnQueryBuilderTest.test_doToQuery_isUnsupportedcalled the removed method and could not compile; it is the only test removed.org.opensearch.index.query.QueryBuildersin its script string. The seed is updated. See the upgrade note below.IndexingHelper.deleteDocumentByQuery(SearchEngineClient, QueryBuilder), its four protected siblings, andLanguageHelper.createScript(Map, String)returningScript. Five plugin repositories follow them and ship together.Two side effects of dropping the dependency, both fixed here:
ThemeManifestparsestheme.yamlwith it directly but never declared it — it was arriving through the opensearch jar, and without itjackson-dataformat-yamldecides the version. Now declared at the release it has been running.org.opensearch.bootstrapno longer names anything: the forked tree has no bootstrap package and the original jar is gone. Removed.error_prone_annotationsmoves 2.45.0 → 2.36.0. Annotations only, unused at runtime, left alone.Upgrade note for the persisted scheduled job
fess_config.scheduled_job/_id=index_exporteris reloaded at every webapp boot, but withOpType.CREATE, so an existing document conflicts and is skipped. The corrected seed therefore does not reach an upgraded installation, and Fess has no migration framework to hang a fix on.The exposure is small and loud: the job ships
available:falsewith an empty cron expression, so it never fires on its own. An administrator who had enabled it will see it fail with aScriptEngineExceptionin the job log rather than fail silently or corrupt anything. The fix is a one-line edit in Admin > Scheduler, replacing the package prefix. This is release-note material; rewriting the document at boot would silently overwrite a script an administrator may have customised.Verification
mvn clean packageand the full suite — 7,469 tests, 0 failures._nodes/statsparsed with all 30 sections populated. ZeroNoClassDefFoundError,ClassNotFoundException,NoSuchMethodErrororServiceConfigurationErroracross all 13 log files, and zero ERROR-level lines.The facet result is the one that matters most:
search/aggregationswas cut from 1,042 classes to 334, and aggregation response parsers are only exercised at runtime.Ordering
Requires codelibs/fesen-httpclient#43, codelibs/fess-suggest#98 and codelibs/fess-crawler#208 to be released first. The five plugin repositories that follow the core signatures above must be released with this, not after it.