Skip to content

[NEP10 NEP38 NEP54] SIMD and Iterator Performance #548

Description

@Nucs

Overview

NumPy's approach to SIMD optimization and iterator performance through universal intrinsics and runtime dispatch.


NEP 10: Optimizing Iterator/UFunc Performance

Status: Final | Full Text

Memory Layout Options

NPY_ANYORDER = -1    // F if all inputs F, else C
NPY_CORDER = 0       // C-contiguous (row-major)
NPY_FORTRANORDER = 1 // Fortran-contiguous
NPY_KEEPORDER = 2    // Match input layout (NEW)

Cache-Coherency (order='K')

Preserve input memory layout instead of forcing C-contiguous:

a.T + b.T  # 8.3x faster with order='K'

Dimension Coalescing

Merge adjacent contiguous dimensions for single-loop iteration:

If strides[i+1] * shape[i+1] == strides[i]:
    Merge dimensions i and i+1

Buffering

Copy chunks to cache-friendly buffer for poor memory layouts:

  • Up to 19x speedup for non-contiguous arrays

Casting Modes

NPY_NO_CASTING        // Identical types only
NPY_EQUIV_CASTING     // + byte-swapped
NPY_SAFE_CASTING      // Safe casts only
NPY_SAME_KIND_CASTING // + same-kind casts
NPY_UNSAFE_CASTING    // Any casts

NEP 38: SIMD Universal Intrinsics

Status: Final | Full Text

Three-Stage Mechanism

  1. Infrastructure: Abstract intrinsics in code
  2. Compile-time: Compiler converts to concrete intrinsics
  3. Runtime: CPU detection selects optimal loop

Universal Intrinsic Concept

npyv_load_u32  → vld1q_u32 (NEON) / _mm256_loadu_si256 (AVX2)
npyv_add_f32   → vaddq_f32 (NEON) / _mm256_add_ps (AVX2)

Supported Instruction Sets

  • x86_64: SSE3 (baseline), SSE4, AVX, AVX2, AVX-512
  • ARM: NEON
  • PowerPC: VSX

Build Options

  • --cpu-baseline: Minimum required features
  • --cpu-dispatch: Additional dispatch variants

NEP 54: Google Highway Adoption

Status: Accepted | Full Text

Why Highway

  • C++ (cleaner than C intrinsics)
  • Sizeless SIMD support (ARM SVE, RISC-V RVV)
  • Extensive documentation
  • Used by Chromium, JPEG XL

Code Improvement

// Old C
npyv_@sfx@ a5 = npyv_load_@sfx@(src1 + npyv_nlanes_@sfx@ * 4);

// New C++ with Highway
auto a5 = Load(src1 + nlanes * 4);

NumPy 2.0 Usage

Sorting functions accelerated via Highway/Intel x86-simd-sort.


Suggested Implementation for NumSharp

.NET SIMD Options

API .NET Version Portability Control
Vector<T> Core 2.0+ High Low
Vector128/256/512<T> Core 3.0+ Medium High

Priority Operations

Operation Expected Speedup
Element-wise (+, -, *, /) 4-8x
Comparisons 4-8x
Reductions (sum, mean) 2-4x
Dot/matmul 4-16x

Implementation Pattern

// Runtime dispatch
ISimdBackend backend = Avx2.IsSupported ? new Avx2Backend()
                     : Avx.IsSupported ? new AvxBackend()
                     : Sse2.IsSupported ? new Sse2Backend()
                     : new ScalarBackend();

Iterator Enhancements

  • Implement dimension coalescing
  • Add order='K' output allocation
  • Consider buffered iteration for non-contiguous arrays

Related Issues

Documentation

See docs/neps/NEP10.md, docs/neps/NEP38.md, docs/neps/NEP54.md

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

NumPy 2.x ComplianceAligns behavior with NumPy 2.x (NEPs, breaking changes)architectureCross-cutting structural changes affecting multiple componentscoreInternal engine: Shape, Storage, TensorEngine, iteratorsenhancementNew feature or requestperformancePerformance improvements or optimizations

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions