Skip to content

Package.swift exclude list breaks when the mlx pin moves past v0.31.2 (jaccl restructured) #446

Description

@GoodOlClint

Summary

Package.swift's exclude: list for the Cmlx target enumerates individual files under the vendored mlx tree. Those paths are correct for the currently pinned mlx (v0.31.1), but mlx/distributed/jaccl was restructured in v0.31.2, so a bump past that point breaks the list in both directions at once. Filing it separately from the bump because the evidence is easy to lose track of.

What changed upstream

At v0.31.1, mlx/distributed/jaccl/ is flat source files, and Package.swift excludes four of them:

"mlx/mlx/distributed/jaccl/jaccl.cpp",
"mlx/mlx/distributed/jaccl/mesh.cpp",
"mlx/mlx/distributed/jaccl/ring.cpp",
"mlx/mlx/distributed/jaccl/utils.cpp",

At v0.31.2 (and still at v0.32.0) that directory is .gitignore, CMakeLists.txt, jaccl.cpp, jaccl.h, lib/, no_jaccl.cpp. So after a bump:

  1. Three excludes point at deleted filesmesh.cpp, ring.cpp and utils.cpp are gone. SwiftPM reports Invalid Exclude '…': File not found. for each. Warnings only, so the build still proceeds.
  2. mlx/distributed/jaccl/lib is a new vendored subtree and is not excluded — it carries its own CMakeLists.txt, README.md, examples/ and a nested jaccl/. Nothing in the current list covers it, so SwiftPM sweeps it into the Cmlx target as ordinary sources.

The second is the real problem; the first is just noise pointing at it.

Suggested fix

At bump time, drop the three dead entries and add the subtree:

"mlx/mlx/distributed/jaccl/jaccl.cpp",
"mlx/mlx/distributed/jaccl/lib",

Adding the lib exclude ahead of the bump is not useful on its own — the path does not exist at v0.31.1, so it would just emit an Invalid Exclude warning on every build until the pin moves.

If it is worth making this class of breakage structural rather than recurring, the distributed/ backends all follow one rule already: every implementation .cpp is excluded and only the no_*.cpp stub is compiled (mpi.cpp, ring.cpp, nccl.cpp, jaccl.cpp, plus nccl_stub). Deriving that in the manifest instead of listing it would cover all four backends against future restructuring. Happy to open a PR for either the minimal bump-time edit or the derived version, whichever you prefer — I did not want to guess which side of that tradeoff you would want.

Related

mlx-swift #445 fixes the sibling instance of this pattern in tools/update-mlx.sh, where the CUDA and Metal jit-source lists were hand-tracked and went stale on the same bump. Same failure shape, different file.

Verified against

The v0.31.1, v0.31.2 and v0.32.0 trees of ml-explore/mlx. Not reproducible at the current pin — this is a bump-time hazard, not a present-day break.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions