ci: let a build see a sibling repository's snapshot the day it is published - #80
Merged
Merged
Conversation
…lished These two repositories are how one repository in this family sees another's change, and Maven's default updatePolicy for a snapshot is daily. A build whose local repository already resolved a -SNAPSHOT earlier the same day never asks again, so a CI job that caches ~/.m2 cannot see a sibling's snapshot published minutes ago. updatePolicy always removes that window. Measured, not assumed. codelibs/fess-crawler#204 merged at 02:48:54 and its snapshot deployed at 02:49:31. codelibs/fess#3428, which depends on it, was re-run at 02:59:29 -- ten minutes after the fix was published -- and failed with DiXmlParseFailureException: crawler/client.xml included by app.xml Caused by: NoClassDefFoundError: software/amazon/awssdk/auth/credentials/AwsCredentials because it built against the previous fess-crawler-lasta. The job log contains no request to maven.codelibs.org at all: not one "Downloading from codelibs.org.snapshot" line, while "Cache restored from key: Linux-maven-..." sits at the top. The published jar was correct the whole time. Reproduced and fixed locally on the same shape of cache. Before, the resolved fess-crawler-lasta-15.9.0-SNAPSHOT.jar declared one s3Client component; after installing this pom and resolving once, it declares none and names the plugin that took over, and the timestamped build the fix landed in appears in ~/.m2. The Fess branch's DI container then built clientFactory and mapped s3://bucket/key, using nothing but published artifacts. Every consumer inherits this, which is the point: the plugin repositories' CI installs this pom from source and then resolves org.codelibs.fess:fess from these repositories, so they are in the same window whenever core publishes. The cost is one metadata request per snapshot dependency per build. Offline builds skip it, and a release build has no snapshot dependencies to check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two lines of configuration. This is what currently blocks codelibs/fess#3428, and it will block the six plugin repositories the moment core publishes a snapshot they need.
What happens today
These two repositories are how one repository in this family sees another's change, and Maven's default
updatePolicyfor a snapshot is daily. A build whose~/.m2already resolved a-SNAPSHOTearlier the same day never asks again — so a CI job that caches the local repository cannot see a sibling repository's snapshot published minutes ago.Measured, not assumed
The failure:
crawler/client.xmlin the previousfess-crawler-lastastill declareds3Client, whose class needs the AWS SDK that #3428 removes, and LastaDi resolves aclassattribute when it parses the XML. The published jar was already correct: downloadingfess-crawler-lasta-15.9.0-20260910.024931-36.jarshows zeros3Clientcomponents and zeros3:.*registrations.The job never looked. Searching the 2951-line log:
Cache restored from key: Linux-maven-1184b2e..., 262 MBDownloading from codelibs.org.snapshot— 0 occurrencesDownloading/Downloadedof anyfess-crawlerartifact — 0 occurrencesThe workflow caches
~/.m2/repositorywithrestore-keys: Linux-maven-, so a differingpom.xmlhash still restores an older cache, andmvnis invoked without-U. Nothing in that combination can expire before the next day.Reproduced and verified locally, on the same shape of cache
My
~/.m2held the same stale jar —unzip -p .../fess-crawler-lasta-15.9.0-SNAPSHOT.jar crawler/client.xmlcounted 1s3Clientcomponent.After
mvn installof this branch and a singlemvn dependency:resolvein the Fess branch:Downloading from codelibs.org.snapshot: .../fess-crawler-lasta/15.9.0-SNAPSHOT/maven-metadata.xml, thenDownloaded20260910.024931-36appears in~/.m2s3Clientcomponents, 0s3:.*registrations, and names the plugin that took overThen, using nothing but published artifacts plus the new plugin jar, a LastaDi container on a plain JVM built
clientFactoryand mappeds3://bucket/keytoorg.codelibs.fess.crawler.client.s3.S3Client, withfile:unaffected — 9 checks, all passing. Without the plugin, which is what CI's "Run Fess" step does, the container still initialises. So #3428 was correct all along.Why here rather than
-Uin one workflowEvery consumer inherits this pom. The plugin repositories' CI checks out
codelibs/fess-parent, installs it from source, and then resolvesorg.codelibs.fess:fessfrom these repositories — sofess-sso-saml,fess-sso-spnego,fess-sso-entraid,fess-sso-oidc,fess-storage-s3andfess-storage-gcsare all in the same window whenever core publishes. Fixing it in one workflow would leave six.Cost
One metadata request per snapshot dependency per build — for Fess that is five artifacts.
-oskips it entirely, and a release build has no snapshot dependencies to check. The comment in the pom records the incident so the setting is not later removed as noise.