Introduce SmartIntegrationArtifactType by mederly · Pull Request #784 · Evolveum/midpoint

mederly · 2026-09-03T10:50:19Z

What

Created SmartIntegrationArtifactType to store working data for smart integration: currently object statistics and cached schema matching data. It is indexed by scope (resource, object class, kind, intent, focus type) and carries the (unindexed) payload: statistics or schema matching.
Created 4 archetypes for these uses.
Renamed ShadowObjectClassStatisticsType to ObjectSetStatisticsType, because the name was outdated (the type is used in broad context).
Minor code restructuralization, mainly regarding utils.
Using ResourceObjectTypeIdentification instead of kind/intent pair at a couple of places.
ResourceObjectTypeIdentification has more human-friendly toString().

Why

Using specialized objects is much better than using GenericObjectType. Moreover, the use of GenericObjectType was considered an interim solution since the beginning. If the transition was not done in 4.11, later in 4.12+ we'd need to provide migration and so on, which is unnecessary hassle.

Note

Some tests in TestSmartIntegrationService fail, but they are failing on master as well.

Task: 11337 (11398)

What - Created SmartIntegrationArtifactType to store working data for smart integration: currently object statistics and cached schema matching data. It is indexed by scope (resource, object class, kind, intent, focus type) and carries the (unindexed) payload: statistics or schema matching. - Created 4 archetypes for these uses. - Renamed ShadowObjectClassStatisticsType to ObjectSetStatisticsType, because the name was outdated (the type is used in broad context). - Minor code restructuralization, mainly regarding utils. - Using ResourceObjectTypeIdentification instead of kind/intent pair at a couple of places. - ResourceObjectTypeIdentification has more human-friendly toString(). Why Using specialized objects is much better than using GenericObjectType. Moreover, the use of GenericObjectType was considered an interim solution since the beginning. If the transition was not done in 4.11, later in 4.12+ we'd need to provide migration and so on, which is unnecessary hassle. Note Some tests in TestSmartIntegrationService fail, but they are failing on master as well. Task: 11337 (11398)

Copilot

🟡 Changes recommended

Fresh database version markers, enum-filter values, and renamed test fixtures must be corrected before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Introduces dedicated smart-integration artifacts, replacing temporary generic-object storage.

Changes:

Adds artifact schema, archetypes, SQL persistence, and repository mappings.
Migrates statistics/schema-match services, activities, API, GUI, and tests.
Renames ShadowObjectClassStatisticsType to ObjectSetStatisticsType.

File summaries

File	Description
`repo/system-init/.../100-archetype-smart-integration-schema-match.xml`	Adds schema-match archetype.
`repo/system-init/.../100-archetype-smart-integration-resource-object-type-statistics.xml`	Adds object-type statistics archetype.
`repo/system-init/.../100-archetype-smart-integration-resource-object-class-statistics.xml`	Adds object-class statistics archetype.
`repo/system-init/.../100-archetype-smart-integration-focus-object-type-statistics.xml`	Adds focus statistics archetype.
`repo/repo-sqale/.../SqaleRepoAddDeleteObjectTest.java`	Tests artifact persistence and filtering.
`repo/repo-sqale/.../SqaleUtils.java`	Advances repository schema versions.
`repo/repo-sqale/.../SqaleRepositoryBeanConfig.java`	Registers the new mapping.
`repo/repo-sqale/.../QSmartIntegrationArtifactMapping.java`	Maps artifact fields to SQL.
`repo/repo-sqale/.../QSmartIntegrationArtifact.java`	Defines artifact query paths.
`repo/repo-sqale/.../MSmartIntegrationArtifact.java`	Defines artifact row data.
`repo/repo-sqale/.../MObjectType.java`	Registers the repository object type.
`model/smart-impl/.../TestSmartIntegrationServiceImpl.java`	Migrates integration tests.
`model/smart-impl/.../TestObjectTypesSuggestionOperation.java`	Migrates statistics lifecycle tests.
`model/smart-impl/.../TestMappingsSuggestionOperation.java`	Updates statistics type usage.
`model/smart-impl/.../TestCorrelatorSuggestions.java`	Updates correlation tests.
`model/smart-impl/.../AbstractSmartIntegrationTest.java`	Adds shared group object-class constant.
`model/smart-impl/.../StatisticsService.java`	Migrates statistics storage and queries.
`model/smart-impl/.../SmartIntegrationServiceImpl.java`	Updates service implementation signatures.
`model/smart-impl/.../SchemaMatchService.java`	Migrates schema-match persistence.
`model/smart-impl/.../PostSchemaMatchHeuristics.java`	Uses renamed statistics type.
`model/smart-impl/.../ObjectTypesSuggestionOperation.java`	Uses renamed statistics type.
`model/smart-impl/.../MappingSuggestionOperationFactory.java`	Updates factory statistics type.
`model/smart-impl/.../MappingsSuggestionOperation.java`	Updates operation statistics type.
`model/smart-impl/.../StatisticsAggregator.java`	Produces renamed statistics objects.
`model/smart-impl/.../SchemaMatchPreloadActivityRun.java`	Uses consolidated type identification.
`model/smart-impl/.../ObjectTypesSuggestionStatisticsComputationActivityRun.java`	Stores typed artifact references.
`model/smart-impl/.../ObjectTypesSuggestionObjectTypesActivityRun.java`	Loads statistics artifacts.
`model/smart-impl/.../ObjectTypeStatisticsComputer.java`	Returns renamed statistics type.
`model/smart-impl/.../ObjectTypeStatisticsComputationWorkDefinition.java`	Consolidates kind and intent.
`model/smart-impl/.../ObjectTypeStatisticsComputationActivityRun.java`	Persists typed statistics artifacts.
`model/smart-impl/.../ObjectClassStatisticsComputer.java`	Returns renamed statistics type.
`model/smart-impl/.../ObjectClassStatisticsComputationActivityRun.java`	Stores typed artifact references.
`model/smart-impl/.../FocusObjectStatisticsComputationWorkDefinition.java`	Consolidates type identification.
`model/smart-impl/.../FocusObjectStatisticsComputationActivityRun.java`	Persists focus statistics artifacts.
`model/smart-impl/.../MappingsSuggestionStatisticsComputationActivityRun.java`	Persists mapping statistics artifacts.
`model/smart-impl/.../MappingsSuggestionSchemaMatchingActivityRun.java`	Uses schema-match artifacts.
`model/smart-impl/.../FocusObjectStatisticsComputer.java`	Returns renamed statistics type.
`model/smart-impl/.../CorrelationSuggestionSchemaMatchingActivityRun.java`	Uses schema-match artifacts.
`model/smart-impl/.../AbstractObjectClassStatisticsComputationActivityRun.java`	Creates object-class artifacts.
`model/smart-api/.../SmartIntegrationService.java`	Updates public smart-integration API.
`model/smart-api/.../ObjectClassInfo.java`	Updates statistics record type.
`model/model-test/.../CommonInitialObjects.java`	Exposes archetypes to tests.
`model/model-intest/.../TestSmartIntegrationService.java`	Removes obsolete suppression.
`milestone-notes.adoc`	Documents upgrade requirements.
`infra/schema/.../model/extension-3.xsd`	Removes temporary generic extensions.
`infra/schema/.../common-tasks-3.xsd`	Retargets activity references.
`infra/schema/.../common-smart-integration-3.xsd`	Defines artifacts and renames statistics.
`infra/schema/.../common-core-3.xsd`	Registers archetype OIDs.
`infra/schema/.../SmartIntegrationArtifactUtil.java`	Adds artifact creation/extraction utilities.
`infra/schema/.../ShadowObjectTypeUtil.java`	Removes obsolete generic-object utility.
`infra/schema/.../ShadowObjectClassUtil.java`	Removes obsolete object-class utility.
`infra/schema/.../FocusObjectStatisticsTypeUtil.java`	Removes obsolete focus utility.
`infra/schema/.../ResourceObjectTypeIdentification.java`	Improves display and short dumps.
`infra/schema/.../SchemaConstants.java`	Removes temporary extension constants.
`infra/schema/.../ObjectTypes.java`	Registers the public object type.
`gui/admin-gui/.../SmartStatisticsPanel.java`	Uses renamed statistics model.
`gui/admin-gui/.../ObjectTypeStatisticsActions.java`	Loads typed statistics artifacts.
`gui/admin-gui/.../ObjectClassStatisticsActions.java`	Loads typed statistics artifacts.
`gui/admin-gui/.../FocusStatisticsActions.java`	Loads typed focus artifacts.
`config/sql/native/postgres.sql`	Adds artifact table and indexes.
`config/sql/native/postgres-upgrade.sql`	Adds repository upgrade changes.
`config/sql/native/postgres-audit.sql`	Adds audit object enum value.
`config/sql/native/postgres-audit-upgrade.sql`	Adds audit schema upgrade.

Review details

Suppressed comments (1)

model/smart-impl/src/main/java/com/evolveum/midpoint/smart/impl/StatisticsService.java:521

The deletion query has the same enum/string mismatch as retrieval: PATH_SCOPE_KIND uses an enum mapper, while kind is a String. This can prevent matching and leave stale statistics behind. Convert the lexical value to ShadowKindType before building the filter.

Files reviewed: 63/63 changed files
Comments generated: 6
Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

This reverts commit 1e795a5. Why It is no longer needed. Extensions for GenericObjectType are gone. (The test in question started to fail again, so it had to be fixed.) Task: 11337

Adapts test to recent changes in ResourceObjectTypeIdentification#toString() method. Task: 11337

This class was replaced by ObjectSetStatisticsType. The remains were probably benign, because relevant tests passed. But anyway... Task: 11337

Change level was updated in the code and in upgrade scripts but not in the "full" scripts. Task: 11337

What Replaced kind/intent pair with typeIdentification (ResourceObjectTypeIdentification) at various places in smart-impl module. Why Primarily because Github Copilot review pointed out that we are querying for String-typed kind at places where ShadowKindType (enum) is expected. That may cause troubles. But generally, using special data type for ResourceObjectTypeIdentification instead of a pair of raw values (sometimes even String for shadow kind) is much cleaner. Task: 11337

Task: 11337 Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

🟡 Changes recommended

Object-class queries normalize QNames inconsistently, and the repository test constructs a scope forbidden by the new schema.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

model/smart-impl/src/main/java/com/evolveum/midpoint/smart/impl/StatisticsService.java:482

This deletion query has the same QName canonicalization mismatch as the lookup: creation qualifies an unqualified object class with NS_RI, while deletion searches for the raw unqualified value. Such artifacts are left behind when deleteStatisticsForResource is called with the original unqualified QName. Normalize the filter value before comparing it.

Files reviewed: 69/69 changed files
Comments generated: 2
Review effort level: Balanced

Object class names and focus type names are stored as qualified. To be certain they are searched-for in the right way, we do the qualification when constructing the queries. Why This improves robustness. Task: 11337

Copilot

🟡 Changes recommended

Focus scopes can become unqueryable, the repository test creates an invalid scope, and legacy artifacts lack cleanup guidance.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

milestone-notes.adoc:23

Deleting the tasks does not delete the GenericObjectType artifacts referenced by their work state. Because the implementation now searches and expires only SmartIntegrationArtifactType and the old extension schema is removed, existing statistics/schema-match generic objects become permanent orphaned data. Add an explicit pre-upgrade cleanup step for the old smart-integration generic artifacts (or provide migration cleanup).

repo/repo-sqale/src/test/java/com/evolveum/midpoint/repo/sqale/func/SqaleRepoAddDeleteObjectTest.java:2828

This test constructs a schema-invalid scope: objectType and objectClass are mutually exclusive under the new xsd:choice. Split the assertions across valid object-type and object-class artifacts (with focusType on either as needed), otherwise the repository test validates a state that serialized artifacts are not allowed to represent.

Files reviewed: 69/69 changed files
Comments generated: 1
Review effort level: Balanced

tonydamage

The repository part looks OK to me.

Task: 11337 Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Task: 11337

mederly requested review from and-za and tonydamage and a balanced review from Copilot September 3, 2026 10:50

Copilot started reviewing on behalf of mederly September 3, 2026 10:51 View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

mederly and others added 8 commits September 3, 2026 13:02

Revert "Adapt schema tests"

f640de3

This reverts commit 1e795a5. Why It is no longer needed. Extensions for GenericObjectType are gone. (The test in question started to fail again, so it had to be fixed.) Task: 11337

Adapt TestConfigErrorReporter

e82fb8f

Adapts test to recent changes in ResourceObjectTypeIdentification#toString() method. Task: 11337

Fix mentions of ShadowObjectClassStatisticsType

94f9f56

This class was replaced by ObjectSetStatisticsType. The remains were probably benign, because relevant tests passed. But anyway... Task: 11337

Fix change level in "full" DB scripts

a133fc9

Change level was updated in the code and in upgrade scripts but not in the "full" scripts. Task: 11337

Fix typosin postgres.sql comments

e0b6b3f

Task: 11337 Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Fix copy error in artifact name for schema match

78976e1

Task: 11337 Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Fix a typo in XSD docs

a51fd7e

Task: 11337 Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

mederly requested a balanced review from Copilot September 3, 2026 11:48

Copilot started reviewing on behalf of mederly September 3, 2026 11:49 View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

Comment thread model/smart-impl/src/main/java/com/evolveum/midpoint/smart/impl/StatisticsService.java Outdated

Comment thread ...-sqale/src/test/java/com/evolveum/midpoint/repo/sqale/func/SqaleRepoAddDeleteObjectTest.java

Qualify QNames when searching in StatisticsService

a057a32

Object class names and focus type names are stored as qualified. To be certain they are searched-for in the right way, we do the qualification when constructing the queries. Why This improves robustness. Task: 11337

mederly requested a balanced review from Copilot September 3, 2026 12:05

Copilot started reviewing on behalf of mederly September 3, 2026 12:06 View session

Copilot AI reviewed Sep 3, 2026

View reviewed changes

Comment thread infra/schema/src/main/java/com/evolveum/midpoint/schema/util/SmartIntegrationArtifactUtil.java Outdated

tonydamage approved these changes Sep 3, 2026

View reviewed changes

mederly and others added 2 commits September 3, 2026 14:38

Qualify focusType in SmartIntegrationArtifactUtil

aa38a70

Task: 11337 Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Fix indentation

1b232ce

Task: 11337

and-za approved these changes Sep 3, 2026

View reviewed changes

Remove unused import

91a0067

Task: 11337

Navigation Menu

Uh oh!

Introduce SmartIntegrationArtifactType - #784

Introduce SmartIntegrationArtifactType#784
mederly wants to merge 13 commits into
masterfrom
feature/11337

mederly commented Sep 3, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

tonydamage left a comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

mederly commented Sep 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (2)

Uh oh!

Uh oh!

tonydamage left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants