Skip to content

Add a Learning Path for integrating a KleidiAI SME2 kernel into XNNPACK - #3706

Open
geesun wants to merge 1 commit into
ArmDeveloperEcosystem:mainfrom
geesun:xnnpack-kai-v3
Open

Add a Learning Path for integrating a KleidiAI SME2 kernel into XNNPACK#3706
geesun wants to merge 1 commit into
ArmDeveloperEcosystem:mainfrom
geesun:xnnpack-kai-v3

Conversation

@geesun

@geesun geesun commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds an advanced Learning Path that demonstrates how to integrate a KleidiAI SME2 matrix multiplication microkernel into XNNPACK's qd8_f16_qc4w fully connected operator.

What is included

  • An overview of the XNNPACK operator lifecycle and its QD8, QC4W, and FP16 data formats
  • Guidance for selecting a KleidiAI microkernel based on its complete quantization contract
  • Instructions for packing static QC4W weights into the KleidiAI RHS layout
  • Instructions for packing dynamic QD8 activations without requantization
  • SME2 runtime dispatch with the existing XNNPACK microkernels retained as a fallback
  • Support for normal and transposed QC4W weight layouts
  • Android NDK build and on-device correctness validation steps
  • A four-part patch series that readers can apply to the documented XNNPACK baseline revision

Key learning outcomes

After completing this Learning Path, readers will understand how to:

  • Match framework tensor formats to a compatible KleidiAI microkernel
  • Preserve per-row QD8 quantization parameters
  • Handle K-dimension padding correctly
  • Integrate KleidiAI packing and execution into XNNPACK
  • Select the SME2 path at runtime without breaking non-SME2 configurations
  • Build and test the integration on an SME2-capable Android device

Validation

The Learning Path documents validation using:

  • XNNPACK's FULLY_CONNECTED_NC_QD8_F16_QC4W correctness tests on an SME2-capable Android device
  • A fallback build with KleidiAI disabled

Submission checklist

Before submitting a pull request for a new Learning Path, please review Create a Learning Path.

  • [ x] I have reviewed Create a Learning Path

Please do not include any confidential information in your contribution. This includes confidential microarchitecture details and unannounced product information.

  • [ x] I have checked my contribution for confidential information

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution under the terms of the Creative Commons Attribution 4.0 International License.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant