docs(15.9): upgrade guide covers stored Groovy jobs, carried-over config and plugin upgrades

[marevol]

The 15.9 upgrade guide describes each change on its own terms, but an installation that upgrades from 15.8 by following it step by step still ends up with several things broken that the guide never mentions. This fills those gaps in install/upgrade.rst and corrects the query examples in config/admin-index-export.rst, in all seven languages (ja, en, de, es, fr, ko, zh-cn). Only the development tree (15.9) is touched.

Found while verifying 15.9.0 by upgrading a 15.8.0 installation with the guide as written.

The bundled jobs stay on Groovy, and the guide did not say where to look

codelibs/fess#3425 stopped bundling Groovy. The script engine section said only that "a job that was already saved as groovy" needs the plugin, which reads as if it concerned custom jobs.

In fact every job that 15.8.0 seeded is stored with scriptType=groovy. That covers all 14 entries in scheduled_job.bulk, Default Crawler, Suggest Indexer and Config Reloader included. SearchEngineClient loads the seed with OpType.CREATE, so an upgraded installation keeps all of them.

Without fess-script-groovy, each job fails whenever it runs, and Default Crawler never starts a crawl. Nine of the fourteen have Logging turned off, so the only trace is a Failed to execute job warning in fess.log.

The section is restructured around every kind of stored setting that 15.9 evaluates as Groovy when no engine is recorded, and what each does without the plugin. Each kind has its own fallback in core:

Setting	Fallback to Groovy in
scheduled jobs	ScheduledJob#getScriptType
web and file configs with field.script.*	CrawlingConfig#getScriptType
data configs with a script	AbstractDataStore#getScriptType
boost rules	DocBoostMatcher
groovy: path mappings	PathMappingHelper

The section then:

• tells upgraders to look on the first start for the fess.log warning that begins Settings use the script engine groovy, which is not registered. It counts each kind of setting and names job.default.script when a carried-over file still sets groovy.

• gives both ways out:

  • bin/fess-setup install plugin fess-script-groovy and a restart, after which the warning is no longer logged; or
  • switching each kind to JavaScript through the field that selects its engine.

• gives the switch per kind, using the screen labels from fess_label_*.properties in each language:

  Setting	Where to select JavaScript
  scheduled jobs	Execution Method
  document boost rules	Script Type
  web and file configs	config.script.type=javascript in Config Parameters
  data configs	script_type=javascript in Parameters
  path mappings	the javascript: prefix of Replacement
  new jobs	job.default.script

• points out that two stored job scripts are not valid JavaScript as they are: Thumbnail Purger's 1000L literal and Index Exporter (next section).

• describes a groovy: path mapping without its engine as not applied, with URLs left unchanged.

"Jobs created after the upgrade get javascript" is now qualified, because that holds only when job.default.script was not carried over (see the configuration group below).

Index Exporter names a package that no longer exists

codelibs/fess#3439 moved the query builders to org.codelibs.fesen.opensearch.* and updated the seeded script. As that PR's own upgrade note says, the seed is create-only, so an upgraded installation keeps org.opensearch.index.query.QueryBuilders. The job then fails with or without the Groovy plugin.

A new section gives the one-line replacement script. The three examples in config/admin-index-export.rst used the old package as well; they are corrected in every language.

Configuration files carried over from 15.8

ZIP Step 3 copies the old fess_config.properties and bin/fess.in.sh whole. An RPM upgrade keeps a modified /etc/fess/fess_config.properties (%config(noreplace)).

A new section lists the keys whose 15.8.0 value now does the wrong thing, from a key-by-key diff of the file between the fess-15.8.0 tag and main:

key	15.8.0	15.9	effect when kept
job.default.script	groovy	javascript	new jobs default to Groovy, which contradicted the guide
job.template.script	Groovy form	JavaScript form	a job created from a crawl configuration gets a Groovy script
crawler.file.protocols	...,storage,s3,gcs	file,smb,smb1,ftp	s3: and gcs: paths are accepted without their plugin
search_engine.http.url	http://localhost:9201	http://localhost:9200	used whenever SEARCH_ENGINE_HTTP_URL is unset (ResourceUtil#getFesenHttpUrl), which it is in a 15.8 fess.in.sh
jvm.crawler.options, jvm.thumbnail.options	old jcifs names	jcifs.client.*	SMB timeouts stay at the jcifs defaults
crawler.default.script and the four keys below	present	removed	no effect

The same section covers the rest of what a copied file carries over:

• A 15.8 fess.in.sh leaves SEARCH_ENGINE_HTTP_URL unset.
• It also lacks the Node.js discovery that the Playwright section relies on.
• Upgraders should diff against the 15.9 files instead of copying them.

The surrounding steps now point there:

• Step 3 has a warning that links to the new section.
• The RPM note says a modified file keeps its old values.
• The DEB warning says the file is overwritten without a prompt or a backup copy, so it has to be backed up first.

Properties

• jcifs (fix: use the jcifs 3.x property names in the crawler JVM options fess#3433).
  • 15.8 passed jcifs.smb.client.* and jcifs.smb1.smb.client.*, which jcifs 3 ignores.
  • 15.9 passes jcifs.client.*, so the SMB connect and session timeouts take effect for the first time and go from 35 s (the jcifs default) to 60 s.
  • A carried-over file keeps the ignored names.
• Removed keys (chore(config): drop four properties whose value has no effect fess#3435). Four keys had no effect and are removed. The section notes that rag.chat.message.max.length still works as a system property.

Plugins

• Updating Plugin Versions now points to bin/fess-setup upgrade plugins and bin/fess-setup check (feat(setup): add check, upgrade plugins and list installed fess#3417). It also notes that upgrade plugins does not install the plugins split out in 15.9. Step 3's copy of the plugin directory gets a note pointing there.
• The Playwright section now says what happens to a configuration that names playwright without the plugin:
  • it is still crawled, but with the plain HTTP client, so text that only JavaScript produces is not indexed;
  • the job ends successfully and no failure URL is recorded;
  • the crawler log shows one warning per crawling configuration, naming the plugin and the two commands.

Distribution size

"The ZIP drops from 438.5 MiB to 204.7 MiB" does not match the released artifacts: fess-15.8.0.zip is 479,344,489 bytes (457.1 MiB). The sentence now states something that can be checked against that zip: the Playwright driver bundle that carried the Node.js executables took up 204.3 MiB of it.

The guide gives no figure for 15.9. A zip built from current main is 102.8 MiB, but that number still depends on dependency releases: codelibs/fess#3444 only takes ICU4J out of the distribution once fess-suggest is released.

The chapter intro no longer counts "eight changes", since the chapter now has more sections.

Ordering

Parts of the text describe log output and behaviour from pull requests that are still open. Everything else describes behaviour already on codelibs/fess main.

• Requires fix(script): warn at startup about settings that use a missing script engine fess#3451 for the startup warning that the script engine section tells upgraders to look for.
• Requires fix(pathmap): leave the URL unchanged when a script mapping has no engine fess#3452 for a groovy: path mapping being left unapplied, with URLs unchanged, when its engine is missing.
• Requires fix(crawler): warn once when client.crawlerClients names a client that is not registered fess#3445 for the crawler-log warning described in the Playwright section.

Verification

• python tools/check_headings.py */15.9: no mismatched section titles.
• docutils parse of all 14 changed files before and after, with doctitle_xform=False as Sphinx uses:
  • exactly one top-level section per file;
  • no new system messages;
  • no inline markup left unparsed, which is the failure to watch for in ja, ko and zh-cn, where literals sit next to CJK text;
  • identical section, table, list and literal-block counts across the seven languages.
• Each of the four new labels is defined once and referenced in every language.
• Screen labels checked against fess_label_{en,ja,de,es,fr,ko,zh_CN}.properties, using the keys the admin JSPs use:
  • scheduledjob_scriptType
  • boost_document_rule_script_type
  • config_parameter
  • handler_parameter
  • handler_script
  • replacement
  • scheduledjob_jobLogging
• Checked against codelibs/fess fess-15.8.0 and main:
  • the seeded scheduled_job.bulk: script types, Logging flags, and the Thumbnail Purger and Index Exporter scripts;
  • the fess_config.properties key diff;
  • the fess.in.sh diff;
  • the script-type fallbacks named above;
  • the engine URL fallback in SearchEngineClient#open;
  • the FessSetup usage text;
  • the release asset size of fess-15.8.0.zip and the size of its driver-bundle-1.60.0.jar entry.