Skip to content

ci: let a build see a sibling repository's snapshot the day it is published - #80

Merged
marevol merged 1 commit into
mainfrom
ci/check-sibling-snapshots
Sep 10, 2026
Merged

marevol merged 1 commit into
mainfrom
ci/check-sibling-snapshots

Conversation

@marevol

@marevol marevol commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Two lines of configuration. This is what currently blocks codelibs/fess#3428, and it will block the six plugin repositories the moment core publishes a snapshot they need.

What happens today

These two repositories are how one repository in this family sees another's change, and Maven's default updatePolicy for a snapshot is daily. A build whose ~/.m2 already resolved a -SNAPSHOT earlier the same day never asks again — so a CI job that caches the local repository cannot see a sibling repository's snapshot published minutes ago.

Measured, not assumed

codelibs/fess-crawler#204 merged 02:48:54
its snapshot deployed 02:49:31
codelibs/fess#3428 re-run restored its cache 02:59:29
that run failed 03:03:24

The failure:

DiXmlParseFailureException: Failed to parse the dependency XML.
[Dependency XML] crawler/client.xml included by app.xml
Caused by: java.lang.NoClassDefFoundError: software/amazon/awssdk/auth/credentials/AwsCredentials

crawler/client.xml in the previous fess-crawler-lasta still declared s3Client, whose class needs the AWS SDK that #3428 removes, and LastaDi resolves a class attribute when it parses the XML. The published jar was already correct: downloading fess-crawler-lasta-15.9.0-20260910.024931-36.jar shows zero s3Client components and zero s3:.* registrations.

The job never looked. Searching the 2951-line log:

  • Cache restored from key: Linux-maven-1184b2e..., 262 MB
  • Downloading from codelibs.org.snapshot0 occurrences
  • Downloading/Downloaded of any fess-crawler artifact — 0 occurrences

The workflow caches ~/.m2/repository with restore-keys: Linux-maven-, so a differing pom.xml hash still restores an older cache, and mvn is invoked without -U. Nothing in that combination can expire before the next day.

Reproduced and verified locally, on the same shape of cache

My ~/.m2 held the same stale jar — unzip -p .../fess-crawler-lasta-15.9.0-SNAPSHOT.jar crawler/client.xml counted 1 s3Client component.

After mvn install of this branch and a single mvn dependency:resolve in the Fess branch:

  • the log shows Downloading from codelibs.org.snapshot: .../fess-crawler-lasta/15.9.0-SNAPSHOT/maven-metadata.xml, then Downloaded
  • build 20260910.024931-36 appears in ~/.m2
  • the resolved jar now counts 0 s3Client components, 0 s3:.* registrations, and names the plugin that took over

Then, using nothing but published artifacts plus the new plugin jar, a LastaDi container on a plain JVM built clientFactory and mapped s3://bucket/key to org.codelibs.fess.crawler.client.s3.S3Client, with file: unaffected — 9 checks, all passing. Without the plugin, which is what CI's "Run Fess" step does, the container still initialises. So #3428 was correct all along.

Why here rather than -U in one workflow

Every consumer inherits this pom. The plugin repositories' CI checks out codelibs/fess-parent, installs it from source, and then resolves org.codelibs.fess:fess from these repositories — so fess-sso-saml, fess-sso-spnego, fess-sso-entraid, fess-sso-oidc, fess-storage-s3 and fess-storage-gcs are all in the same window whenever core publishes. Fixing it in one workflow would leave six.

Cost

One metadata request per snapshot dependency per build — for Fess that is five artifacts. -o skips it entirely, and a release build has no snapshot dependencies to check. The comment in the pom records the incident so the setting is not later removed as noise.

…lished

These two repositories are how one repository in this family sees another's
change, and Maven's default updatePolicy for a snapshot is daily. A build whose
local repository already resolved a -SNAPSHOT earlier the same day never asks
again, so a CI job that caches ~/.m2 cannot see a sibling's snapshot published
minutes ago. updatePolicy always removes that window.

Measured, not assumed. codelibs/fess-crawler#204 merged at 02:48:54 and its
snapshot deployed at 02:49:31. codelibs/fess#3428, which depends on it, was
re-run at 02:59:29 -- ten minutes after the fix was published -- and failed with

  DiXmlParseFailureException: crawler/client.xml included by app.xml
  Caused by: NoClassDefFoundError: software/amazon/awssdk/auth/credentials/AwsCredentials

because it built against the previous fess-crawler-lasta. The job log contains
no request to maven.codelibs.org at all: not one "Downloading from
codelibs.org.snapshot" line, while "Cache restored from key: Linux-maven-..."
sits at the top. The published jar was correct the whole time.

Reproduced and fixed locally on the same shape of cache. Before, the resolved
fess-crawler-lasta-15.9.0-SNAPSHOT.jar declared one s3Client component; after
installing this pom and resolving once, it declares none and names the plugin
that took over, and the timestamped build the fix landed in appears in ~/.m2.
The Fess branch's DI container then built clientFactory and mapped
s3://bucket/key, using nothing but published artifacts.

Every consumer inherits this, which is the point: the plugin repositories'
CI installs this pom from source and then resolves org.codelibs.fess:fess from
these repositories, so they are in the same window whenever core publishes.

The cost is one metadata request per snapshot dependency per build. Offline
builds skip it, and a release build has no snapshot dependencies to check.
@marevol
marevol merged commit 9978695 into main Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant