Skip to content

test: Fixing some flaky tests (#3644) - #3813

Open
ArneDeutsch wants to merge 1 commit into
eclipse-xtext:mainfrom
ArneDeutsch:ad-fix-flaky-tests
Open

ArneDeutsch wants to merge 1 commit into
eclipse-xtext:mainfrom
ArneDeutsch:ad-fix-flaky-tests

Conversation

@ArneDeutsch

Copy link
Copy Markdown
Contributor

AbstractProblemHoverTest: after creating the markers, waits for the editors initial validation job (testAnnotations failures)

XtextContentAssistTest: explicit build now runs after PDEs job and that build performs Xtexts pending full build, which indexes the jar grammars (flaky windows tests)

TestedWorkspaceWithJDT: job family handling in PDE has changed (PluginModelManager$UpdateClasspathsJob,
ClasspathComputer$UpdateClasspathsJob), old job detection no longer matches the new one (org.eclipse.xtext.builder.tests failures)

WorkbenchTestHelper: wait for PDE after project creation (org.eclipse.xtext.xbase.lib failures)

AbstractProblemHoverTest: after creating the markers, waits for the
editors initial validation job (testAnnotations failures)

XtextContentAssistTest: explicit build now runs after PDEs job and that
build performs Xtexts pending full build, which indexes the jar grammars
(flaky windows tests)

TestedWorkspaceWithJDT: job family handling in PDE has changed
(PluginModelManager$UpdateClasspathsJob,
ClasspathComputer$UpdateClasspathsJob), old job detection no longer
matches the new one (org.eclipse.xtext.builder.tests failures)

WorkbenchTestHelper: wait for PDE after project creation
(org.eclipse.xtext.xbase.lib failures)

Signed-off-by: Arne Deutsch <adeutsch@itemis.de>
@ArneDeutsch ArneDeutsch mentioned this pull request Sep 23, 2026
* @since 2.45
*/
@SuppressWarnings("restriction")
public static void waitForPdeClasspathUpdate() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ArneDeutsch we talked about this possible solution in the past to try to fix the tons of flaky tests!

I hadn't tried that myself.

I thought @cdietrich already tried this and it didn't work, but maybe I misremember or we were using the wrong job ids.

If this fixes flakyness for good, besides you becoming my hero ;) , I guess we should add such a wait in all the abstract test classes that need JDT/PDE.

@github-actions

Copy link
Copy Markdown

Test Results

  6 445 files  ±0    6 445 suites  ±0   3h 14m 51s ⏱️ + 19m 42s
 43 167 tests ±0   42 583 ✅ ±0    584 💤 ±0  0 ❌ ±0 
169 964 runs  ±0  167 627 ✅ ±0  2 337 💤 ±0  0 ❌ ±0 

Results for commit cda1978. ± Comparison against base commit 99768bd.

@ArneDeutsch

Copy link
Copy Markdown
Contributor Author

Hi. My commiter status has been removed, hence I can not merge these changes myself any more. It would be greatly appreciated if one of the commiters would go over my PRs and merge them (or provide feedback if changes are needed).

@LorenzoBettini

Copy link
Copy Markdown
Contributor

@ArneDeutsch how come your committer status has been removed?

I'm fine with the PR, though I'd rather use the new wait method implicitly in some parts; I guess it should be used in wait for jdt index (which in turn is used in wait for build).

@cdietrich @szarnekow what do you think?
Should we merge this now and experiment with its automatic usage in the future?

@ArneDeutsch

Copy link
Copy Markdown
Contributor Author

@LorenzoBettini I think it is because I have not contributed for quite some time. Would you also have a look at the other open PRs? Most of them are quite small and easy to check. Tamas can also not merge. He has reviewed the stuff, but can't merge as well.

@cdietrich

Copy link
Copy Markdown
Contributor

@LorenzoBettini wont find time before next tuesdays call

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants