From 225ca83a2f06b63fd3751877481182d16e22141b Mon Sep 17 00:00:00 2001 From: Shinsuke Sugaya Date: Thu, 10 Sep 2026 13:50:04 +0900 Subject: [PATCH] feat(fess): install the plugins 15.9 unbundled, with bin/fess-setup Fess 15.9 moved the Groovy script engine, the S3 and GCS storage backends and the four SSO authenticators out of the war. Each is a plugin now, so an image built from a 15.9 distribution alone quietly loses behaviour the 15.8 images had: a data store, job or crawler script that leaves script_type unset still resolves to groovy, and s3:, gcs: and every sso.type stop working. The three snapshot images install all seven at build time with `bin/fess-setup install plugin`. It resolves the version that fits the build -- for a -SNAPSHOT, the latest timestamped build in the snapshot repository -- and checks each jar against the SHA-1 the repository publishes. An unresolvable name exits 1, so a missing plugin fails the build instead of producing an image that is short a feature with nothing in the log to say so. The seven jars add 36,392,732 bytes. Verified by building all three variants with docker build, standing the post-fess#3429 fess-setup.jar in for the one today's published snapshot still carries. Each image ends with seven jars in app/WEB-INF/plugin under their timestamped Maven names, owned fess:fess, and `fess-setup list installed` sees all seven. `install plugin fess-sso-nosuchthing` exits 1, which is the claim the comment makes. bin/fess-setup resolves java through JAVA_HOME in all three bases, which matters for amazoncorretto:21-al2023: it ships no `which`, so the launcher's fallback branch would not have worked there. run.sh rejected the new names. FESS_PLUGINS accepts a fixed set of artifact prefixes, and fess-sso- and fess-storage- were not among them, so FESS_PLUGINS=fess-sso-saml:15.9.0 was reported as unrecognized and skipped. Both are added, and with them fess-thumbnail-, which was already missing before 15.9 -- fess-thumbnail-playwright is published and was being rejected. The prefix list, the error message and the README table all have to agree, so all three are updated together. --- README.md | 20 +++++++++++++++++++- fess/snapshot-al2023/Dockerfile | 20 +++++++++++++++++++- fess/snapshot-al2023/run.sh | 9 ++++++++- fess/snapshot-noble/Dockerfile | 20 +++++++++++++++++++- fess/snapshot-noble/run.sh | 9 ++++++++- fess/snapshot/Dockerfile | 18 +++++++++++++++++- fess/snapshot/run.sh | 9 ++++++++- 7 files changed, 98 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index ece62f6..d8b4bcd 100644 --- a/README.md +++ b/README.md @@ -279,6 +279,24 @@ behaviours differ: - **The legacy `elasticsearch.*` configuration keys are gone.** The `ES_*` environment variables these images accept are unaffected; the entrypoint translates them to `SEARCH_ENGINE_*`. +15.9 also moved several features out of the war and into plugins. **The images install all seven +at build time** with `bin/fess-setup install plugin`, so they behave as the 15.8 images did and +none of these belongs in `FESS_PLUGINS`: + +| What stops working without it | Plugin | +|-------------------------------|--------| +| `script_type=groovy` -- and any data store, job or crawler script that leaves `script_type` unset, which still resolves to groovy | `fess-script-groovy` | +| `s3:` crawling and the S3 storage backend | `fess-storage-s3` | +| `gcs:` crawling and the GCS storage backend | `fess-storage-gcs` | +| `sso.type=saml` | `fess-sso-saml` | +| `sso.type=spnego` | `fess-sso-spnego` | +| `sso.type=entraid`, and the legacy `aad` | `fess-sso-entraid` | +| `sso.type=oic` -- the one name that differs from its plugin's | `fess-sso-oidc` | + +To drop one from an image, run `bin/fess-setup remove plugin ` in a derived image. Keep a +plugin on the same line as the Fess it runs in: a 15.9 SSO plugin in a 15.8 Fess registers a +second copy of an authenticator that war already declares, and `/sso/` then fails. + ## Troubleshooting ### Common Issues @@ -351,7 +369,7 @@ environment: - FESS_PLUGINS=fess-ds-wikipedia:15.8.0 fess-ds-git:15.8.0 ``` -Entries are `plugin-name:version` pairs separated by spaces, and the name has to start with one of `fess-ds-`, `fess-ingest-`, `fess-llm-`, `fess-script-`, `fess-theme-` or `fess-webapp-`. A name that is not recognized, or a version that cannot be downloaded, is skipped and does not stop the container from starting, so check the boot log after adding a plugin. +Entries are `plugin-name:version` pairs separated by spaces, and the name has to start with one of `fess-ds-`, `fess-ingest-`, `fess-llm-`, `fess-script-`, `fess-sso-`, `fess-storage-`, `fess-theme-`, `fess-thumbnail-` or `fess-webapp-`. A name that is not recognized, or a version that cannot be downloaded, is skipped and does not stop the container from starting, so check the boot log after adding a plugin. Semantic search no longer needs a plugin. It became part of Fess in 15.8, and `fess-webapp-semantic-search` is not published for 15.8 or later. diff --git a/fess/snapshot-al2023/Dockerfile b/fess/snapshot-al2023/Dockerfile index e0c9880..a7a350a 100644 --- a/fess/snapshot-al2023/Dockerfile +++ b/fess/snapshot-al2023/Dockerfile @@ -49,7 +49,25 @@ RUN set -x && \ # 8. Set Fess environment variables echo "export FESS_APP_TYPE=$FESS_APP_TYPE" >> /usr/share/fess/bin/fess.in.sh && \ echo "export FESS_OVERRIDE_CONF_PATH=/opt/fess" >> /usr/share/fess/bin/fess.in.sh && \ - # 9. Clean up dnf cache to reduce image size + # 9. Install the plugins that carry what the distribution used to bundle. 15.9 moved the + # Groovy script engine, the S3 and GCS storage backends and the four SSO authenticators + # out of the war, so leaving them out would drop behaviour the 15.8 image had: a data + # store, job or crawler script with no script_type still resolves to groovy, and s3:, + # gcs: and every sso.type would stop working. bin/fess-setup resolves the version that + # fits this build, taking the latest timestamped build from the snapshot repository for + # a -SNAPSHOT, and checks every jar against the SHA-1 the repository publishes, so a + # plugin that cannot be resolved fails the build instead of leaving the image short. + # The package installs the tree as fess:fess, so the jars are chowned to match. + /usr/share/fess/bin/fess-setup install plugin \ + fess-script-groovy \ + fess-storage-gcs \ + fess-storage-s3 \ + fess-sso-saml \ + fess-sso-spnego \ + fess-sso-entraid \ + fess-sso-oidc && \ + chown -R fess:fess /usr/share/fess/app/WEB-INF/plugin && \ + # 10. Clean up dnf cache to reduce image size dnf clean all # Set working directory diff --git a/fess/snapshot-al2023/run.sh b/fess/snapshot-al2023/run.sh index e281ebb..c6d8a98 100755 --- a/fess/snapshot-al2023/run.sh +++ b/fess/snapshot-al2023/run.sh @@ -80,12 +80,19 @@ download_plugin() { plugin_id=$1 plugin_name=$(echo ${plugin_id} | sed -e "s/:.*//") plugin_version=$(echo ${plugin_id} | sed -e "s/.*://") + # The prefixes Fess itself loads, which is PluginHelper.ArtifactType. fess-sso- and + # fess-storage- arrived in 15.9, when the SSO authenticators and the S3 and GCS storage + # backends left the war; fess-thumbnail- was missing before that. Keep the message below + # in step with this list. if [[ ${plugin_name} == fess-ds-* ]] \ || [[ ${plugin_name} == fess-ingest-* ]] \ || [[ ${plugin_name} == fess-script-* ]] \ || [[ ${plugin_name} == fess-llm-* ]] \ || [[ ${plugin_name} == fess-theme-* ]] \ || [[ ${plugin_name} == fess-webapp-* ]] \ + || [[ ${plugin_name} == fess-sso-* ]] \ + || [[ ${plugin_name} == fess-storage-* ]] \ + || [[ ${plugin_name} == fess-thumbnail-* ]] \ ; then plugin_file="${plugin_name}-${plugin_version}.jar" if [[ ${plugin_version} == *-SNAPSHOT ]] ; then @@ -121,7 +128,7 @@ download_plugin() { mv "${temp_dir}/${plugin_file}" "${plugin_dir}" chown fess:fess "${plugin_dir}/${plugin_file}" else - print_log ERROR "Unrecognized plugin ${plugin_id} in FESS_PLUGINS. Expected :, where starts with fess-ds-, fess-ingest-, fess-llm-, fess-script-, fess-theme- or fess-webapp-. Skipping it." + print_log ERROR "Unrecognized plugin ${plugin_id} in FESS_PLUGINS. Expected :, where starts with fess-ds-, fess-ingest-, fess-llm-, fess-script-, fess-sso-, fess-storage-, fess-theme-, fess-thumbnail- or fess-webapp-. Skipping it." fi } diff --git a/fess/snapshot-noble/Dockerfile b/fess/snapshot-noble/Dockerfile index 9bd35e2..d73c477 100644 --- a/fess/snapshot-noble/Dockerfile +++ b/fess/snapshot-noble/Dockerfile @@ -52,7 +52,25 @@ RUN set -x && \ # 8. Set Fess environment variables echo "export FESS_APP_TYPE=$FESS_APP_TYPE" >> /usr/share/fess/bin/fess.in.sh && \ echo "export FESS_OVERRIDE_CONF_PATH=/opt/fess" >> /usr/share/fess/bin/fess.in.sh && \ - # 9. Clean up apt cache to reduce image size + # 9. Install the plugins that carry what the distribution used to bundle. 15.9 moved the + # Groovy script engine, the S3 and GCS storage backends and the four SSO authenticators + # out of the war, so leaving them out would drop behaviour the 15.8 image had: a data + # store, job or crawler script with no script_type still resolves to groovy, and s3:, + # gcs: and every sso.type would stop working. bin/fess-setup resolves the version that + # fits this build, taking the latest timestamped build from the snapshot repository for + # a -SNAPSHOT, and checks every jar against the SHA-1 the repository publishes, so a + # plugin that cannot be resolved fails the build instead of leaving the image short. + # The package installs the tree as fess:fess, so the jars are chowned to match. + /usr/share/fess/bin/fess-setup install plugin \ + fess-script-groovy \ + fess-storage-gcs \ + fess-storage-s3 \ + fess-sso-saml \ + fess-sso-spnego \ + fess-sso-entraid \ + fess-sso-oidc && \ + chown -R fess:fess /usr/share/fess/app/WEB-INF/plugin && \ + # 10. Clean up apt cache to reduce image size apt-get clean && \ rm -rf /var/lib/apt/lists/* diff --git a/fess/snapshot-noble/run.sh b/fess/snapshot-noble/run.sh index f69505f..cca3748 100755 --- a/fess/snapshot-noble/run.sh +++ b/fess/snapshot-noble/run.sh @@ -80,12 +80,19 @@ download_plugin() { plugin_id=$1 plugin_name=$(echo ${plugin_id} | sed -e "s/:.*//") plugin_version=$(echo ${plugin_id} | sed -e "s/.*://") + # The prefixes Fess itself loads, which is PluginHelper.ArtifactType. fess-sso- and + # fess-storage- arrived in 15.9, when the SSO authenticators and the S3 and GCS storage + # backends left the war; fess-thumbnail- was missing before that. Keep the message below + # in step with this list. if [[ ${plugin_name} == fess-ds-* ]] \ || [[ ${plugin_name} == fess-ingest-* ]] \ || [[ ${plugin_name} == fess-script-* ]] \ || [[ ${plugin_name} == fess-llm-* ]] \ || [[ ${plugin_name} == fess-theme-* ]] \ || [[ ${plugin_name} == fess-webapp-* ]] \ + || [[ ${plugin_name} == fess-sso-* ]] \ + || [[ ${plugin_name} == fess-storage-* ]] \ + || [[ ${plugin_name} == fess-thumbnail-* ]] \ ; then plugin_file="${plugin_name}-${plugin_version}.jar" if [[ ${plugin_version} == *-SNAPSHOT ]] ; then @@ -121,7 +128,7 @@ download_plugin() { mv "${temp_dir}/${plugin_file}" "${plugin_dir}" chown fess:fess "${plugin_dir}/${plugin_file}" else - print_log ERROR "Unrecognized plugin ${plugin_id} in FESS_PLUGINS. Expected :, where starts with fess-ds-, fess-ingest-, fess-llm-, fess-script-, fess-theme- or fess-webapp-. Skipping it." + print_log ERROR "Unrecognized plugin ${plugin_id} in FESS_PLUGINS. Expected :, where starts with fess-ds-, fess-ingest-, fess-llm-, fess-script-, fess-sso-, fess-storage-, fess-theme-, fess-thumbnail- or fess-webapp-. Skipping it." fi } diff --git a/fess/snapshot/Dockerfile b/fess/snapshot/Dockerfile index 8129b43..f94ed94 100644 --- a/fess/snapshot/Dockerfile +++ b/fess/snapshot/Dockerfile @@ -71,7 +71,23 @@ RUN set -x && \ /usr/share/fess/app/WEB-INF/classes/fess_env_*.properties \ /usr/share/fess/app/WEB-INF/classes/tika.xml \ /etc/fess && \ - # 11. Set proper ownership for all Fess-related directories and files + # 11. Install the plugins that carry what the distribution used to bundle. 15.9 moved the + # Groovy script engine, the S3 and GCS storage backends and the four SSO authenticators + # out of the war, so leaving them out would drop behaviour the 15.8 image had: a data + # store, job or crawler script with no script_type still resolves to groovy, and s3:, + # gcs: and every sso.type would stop working. bin/fess-setup resolves the version that + # fits this build, taking the latest timestamped build from the snapshot repository for + # a -SNAPSHOT, and checks every jar against the SHA-1 the repository publishes, so a + # plugin that cannot be resolved fails the build instead of leaving the image short. + /usr/share/fess/bin/fess-setup install plugin \ + fess-script-groovy \ + fess-storage-gcs \ + fess-storage-s3 \ + fess-sso-saml \ + fess-sso-spnego \ + fess-sso-entraid \ + fess-sso-oidc && \ + # 12. Set proper ownership for all Fess-related directories and files chown -R fess:fess /usr/share/fess /etc/default/fess /etc/fess /opt/fess /var/tmp/fess /var/log/fess /var/lib/fess # Set working directory diff --git a/fess/snapshot/run.sh b/fess/snapshot/run.sh index f2bbf71..493e354 100755 --- a/fess/snapshot/run.sh +++ b/fess/snapshot/run.sh @@ -71,12 +71,19 @@ download_plugin() { plugin_id=$1 plugin_name=$(echo ${plugin_id} | sed -e "s/:.*//") plugin_version=$(echo ${plugin_id} | sed -e "s/.*://") + # The prefixes Fess itself loads, which is PluginHelper.ArtifactType. fess-sso- and + # fess-storage- arrived in 15.9, when the SSO authenticators and the S3 and GCS storage + # backends left the war; fess-thumbnail- was missing before that. Keep the message below + # in step with this list. if [[ ${plugin_name} == fess-ds-* ]] \ || [[ ${plugin_name} == fess-ingest-* ]] \ || [[ ${plugin_name} == fess-script-* ]] \ || [[ ${plugin_name} == fess-llm-* ]] \ || [[ ${plugin_name} == fess-theme-* ]] \ || [[ ${plugin_name} == fess-webapp-* ]] \ + || [[ ${plugin_name} == fess-sso-* ]] \ + || [[ ${plugin_name} == fess-storage-* ]] \ + || [[ ${plugin_name} == fess-thumbnail-* ]] \ ; then plugin_file="${plugin_name}-${plugin_version}.jar" if [[ ${plugin_version} == *-SNAPSHOT ]] ; then @@ -112,7 +119,7 @@ download_plugin() { mv "${temp_dir}/${plugin_file}" "${plugin_dir}" chown fess:fess "${plugin_dir}/${plugin_file}" else - print_log ERROR "Unrecognized plugin ${plugin_id} in FESS_PLUGINS. Expected :, where starts with fess-ds-, fess-ingest-, fess-llm-, fess-script-, fess-theme- or fess-webapp-. Skipping it." + print_log ERROR "Unrecognized plugin ${plugin_id} in FESS_PLUGINS. Expected :, where starts with fess-ds-, fess-ingest-, fess-llm-, fess-script-, fess-sso-, fess-storage-, fess-theme-, fess-thumbnail- or fess-webapp-. Skipping it." fi }