I paid for a GPU in my Mac. For two years I barely used it.

MLX is an open source machine learning framework built by Apple's research team.

It has arrays, neural networks, automatic differentiation, optimizers, and execution on both CPU and GPU. If you have worked with NumPy or PyTorch, a lot of it will feel familiar.

But that is not the interesting part. The interesting part is that it was built around the way your Mac actually works.

On classic GPU machines, the CPU and the GPU usually have separate memory. Want to get data to the GPU? You have to move it. Want it back? Move it again.

Which means copies, synchronization, memory management, and a few other words that usually show up right before a developer says: "forget it, I'll spin up an EC2".

Apple Silicon works differently. The CPU and the GPU share Unified Memory. One pool. MLX was designed around that from the start.

So instead of treating the GPU like an exotic appendix you ship data to and from, it can be a much more natural part of the computation.

But if your library does not know how to take advantage of that, then as far as you are concerned none of it exists. Which is exactly what happened to me.

Because for years, "run AI locally" usually meant one of three things:

1. 1It will run slowly on the CPU.
2. 2You need NVIDIA.
3. 3You somehow spent a Saturday with CUDA, drivers, and one specific Python version nobody is allowed to touch.

MLX offers Mac users another route. Projects like mlx-lm and mlx-whisper let you run LLMs and Whisper on a stack built specifically for Apple Silicon.

And that is interesting to more than just people who enjoy benchmarks. Local inference means:

• lower cloud costs
• sensitive data that stays with you
• work that continues without internet
• a much shorter feedback loop
• and in some cases, latency that is faintly ridiculous compared to what we got used to

Or in my case: finding out the app could be almost 6x faster without changing the model at all.

I maintain an open source Hebrew transcription tool. I built it for a very real need: transcribing episodes of "Developers Outside the Box".

I hand it an audio or video file, and it transcribes with an ivrit.ai build of Whisper Large V3 Turbo, produces text and SRT, and can run diarization to separate speakers.

In this case, the input was a 32-minute episode.

The original version used faster-whisper, which runs on CTranslate2. It is an excellent engine. On NVIDIA it knows how to use CUDA. But on Apple Silicon, on the path I was using, it ran on the ARM CPU.

Which means that while I sat waiting for a full episode to transcribe, the GPU in my Mac was barely in the game.

So I asked the obvious question: what happens if I let it work?

I added a second backend with mlx-whisper and an MLX build of the same Hebrew model. Then, instead of simply deciding MLX "should" be faster, I let the tool run both and measure.

The first time the tool runs on a supported Mac, it takes a 60-second sample and runs it twice:

1. 1Once with CTranslate2.
2. 2Once with MLX.
3. 3Compares the times.
4. 4Saves the faster backend.
5. 5And uses it on subsequent runs.

If MLX is unavailable, it uses CTranslate2. If MLX fails, it falls back to CTranslate2.

That might sound like a small implementation detail. It is probably the most important thing I built in this whole story.

I ran both backends on a MacBook Pro with an Apple M5 Max, an 18-core processor and 36GB of Unified Memory. The same 60 seconds of audio. The same workload. The models were already local before I measured.

This is not an improvement you need a magnifying glass to see. In a later test, a full 32-minute episode went from something that "works locally" to something that feels nearly instant.

And that is the difference that matters to me. There is an enormous gap between "you can run this on your machine" and "wait, it finished already?"

No, this is not a scientific benchmark.

I did not prove MLX will be 5.81x faster for you. I did not prove the output is identical word for word. And no, your M1 or M2 will not necessarily produce the same number.

But that was not the question either. My question was: on this machine, for this workload, which backend should I use?

I thought the story here was MLX. It is not.

The story is not picking a backend. At least not permanently.

Hardware changes. Libraries change. Models change. What was true on an M2 is not necessarily true on an M5. And what I see on my machine is not necessarily true on your user's.

So instead of hardcoding something like this:

Mac            = MLX
Everything else = CTranslate2

It is far more interesting to say: check. On the user's machine. With a real workload. Then save the result. It is the same habit I wrote about in building an agent that actually works: decide from what actually happens, not from what is supposed to happen.

It is a pattern I am taking with me to other things:

• a portable backend by default
• a hardware-optimal backend where possible
• a real benchmark
• a cached decision
• a manual override
• always a fallback

It is less "elegant" than one hardcoded rule. But users do not pay us for elegance. They pay us for software that runs fast.

If you develop on Apple Silicon and do anything AI-related: at least check.

• Especially if your current workload runs on the CPU
• Especially if you care about latency, privacy or cloud cost
• And especially if your model already has a good MLX implementation

• This does not mean MLX replaces CUDA
• It does not mean you should throw out PyTorch
• And it certainly does not mean every workload is suddenly 6x faster

But if you have a Mac with a GPU and you are still running all of your AI on the CPU alone, there is a good chance you are leaving a lot of performance on the table.

For years we got used to thinking of the Mac as the computer you write AI on. The real computation happens somewhere else. A GPU server. AWS. NVIDIA. The cloud.

MLX is starting to undermine that assumption.

And the funny part? I did not change Whisper. I did not train a model. I did not buy a GPU. I did not even switch computers.

I just found out the computer I already bought was far faster than I was letting it be.