Skip to content

Optimize memory usage - #454

Open
Joannis wants to merge 5 commits into
ml-explore:mainfrom
wendylabsinc:jo/mlxarray-view
Open

Joannis wants to merge 5 commits into
ml-explore:mainfrom
wendylabsinc:jo/mlxarray-view

Conversation

@Joannis

@Joannis Joannis commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Proposed changes

  • Adds MLXArrayOf - a fixed-Scalar type wrapping an MLXArray instead of type-unsafe representations
  • Adds support for reading values as a Span

All this should increase safety, reduce allocations and reduce memory copies.

Checklist

Put an x in the boxes that apply.

  • I have read the CONTRIBUTING document
  • I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the necessary documentation (if needed)

view(_:) casts dtype + makes contiguous once, then vends zero-copy Span<Scalar>
(lifetime-tied via @_lifetime(borrow self)) plus single/range copy-out. Enables
the Lifetimes experimental feature on the MLX target.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JMq6N8WW3h7bGdgskwoFDQ

# Conflicts:
#	Package.swift
# Conflicts:
#	Source/MLX/Transforms+Compile.swift
# Conflicts:
#	Source/MLX/MLXArray+Bytes.swift
#	Tests/MLXTests/MLXArrayTests.swift
# Conflicts:
#	Source/MLX/Transforms+Eval.swift
@Joannis Joannis changed the title Jo/mlxarray view Optimize memory usage Aug 16, 2026
Comment thread Package.swift
Comment on lines 363 to +364
.enableExperimentalFeature("StrictConcurrency")
.swiftLanguageMode(.v6),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

build issue here -- missing a comma on 363

while true {
// offset relative to the span's own start -- always >= 0 by construction,
// since minSourceIndex is the true minimum reachable sourceIndex
let spanOffset = (sourceIndex - minSourceIndex) * itemSize

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it is maybe missing a commit -- minSourceIndex is not defined.

Comment on lines +129 to +130
traceState.stateInputs = inputs.flatMap { $0.innerState() }
traceState.argumentsCount = arguments.count

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to be done inside evalLock, otherwise concurrent use of the compiled function is mutating shared state.

evalLock is fine, but I wonder if the instance needs its own lock. Eventually (I hope) evalLock will go away -- we didn't need it in early versions of mlx because the mutation of shared state was done on the cpu/gpu worker threads. We need it now because some of that was hoisted into the calling thread.

Anyway, it might be better to express the intent here with a lock owned by trace state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants