A repeated iOS build still waits in the Mac queue, while cache storage and transfer costs keep rising.
Fastest fix: do not enable Bazel 9 remote caching globally yet; start with a fixed Mac baseline and a read-only cache trial, then expand only when reproducibility, cross-node reuse, and network evidence support it.
Who should read this: You are an engineering productivity leader running a large Bazel-based iOS project, an IT or FinOps owner deciding between cache investment and more Mac capacity, or a platform and security engineer governing cache writes, artifact isolation, and build supply-chain risk.
Last updated September 1, 2026. Version and compatibility checks should be repeated against the Bazel release notes, the rules_apple repository, and current Apple platform requirements before production approval.
Start with the decision, not the cache server
Bazel 9 remote caching is worth a controlled trial only when the same actions recur across builds and machines, the actions are reproducible, and the cache path is stable enough that downloading an output does not cost more than rebuilding it. The cache stores and retrieves Bazel action outputs. It does not turn a Linux or generic remote worker into an executor for Xcode, Apple simulators, code signing, or other Mac-only tasks. Bazel documents the remote cache as a shared store for action results and related objects, not as a replacement for the execution platform. See the official remote caching model.
Use this first-pass decision table before requesting budget:
| Observed condition | Recommended decision | Evidence to collect |
|---|---|---|
| Repeated actions are reproducible, cross-node reuse is visible, and transfer failures are rare | Proceed with a read-only pilot, then a controlled writer group | Action-level hit data, download and upload logs, artifact verification |
| Builds repeat work, but toolchains or environment inputs drift between Macs | Pause deployment and fix reproducibility first | Xcode, SDK, Bazel, rules, environment, path, and host-tool differences |
| Most queue time comes from signing, simulator tests, archives, or cache misses | Prioritize Mac capacity rather than cache expansion | Queue segments by target type, miss reason, and Mac occupancy |
| Cache objects are large, remote, or unreliable across regions | Keep the scope narrow or use a same-region cache | Transfer duration, failure logs, storage growth, and cold-start behavior |
| Sensitive targets cannot share artifacts safely | Exclude them from the shared cache | Target classification, credentials, provenance, and deletion controls |
First metric: prove that actions can be reused
The first question is not whether Bazel can connect to a cache. It is whether the action key remains meaningful across the machines that will consume it.
A cache hit is useful only when the action inputs, command, environment, tool paths, and declared platform properties represent the same build semantics. For an iOS pipeline, this makes the following items part of the cache review:
- Bazel version and startup configuration.
rules_appleandrules_swiftrevisions.- Xcode version, SDK selection, and selected developer directory.
- macOS image, CPU architecture, and installed command-line tools.
- Environment variables that influence compilation or packaging.
- Absolute paths and generated files included outside declared inputs.
- External tools invoked by a rule or script.
- Signing, provisioning, archive, and simulator targets that should not share generic outputs.
rules_apple project are useful compatibility references, but a repository statement or release note is not the same as production validation. Check whether the exact Bazel 9 release, rule revision, Xcode version, and SDK combination is documented. If it is not, classify the combination as a trial condition rather than a supported production baseline.
Collect these baseline metrics before changing the build architecture:**Operational warning:** a cache hit can be fast and still be wrong if an undeclared input or host-specific tool changed. Treat reproducibility and artifact verification as release controls, not as optional performance tuning.
- The proportion of repeated actions across comparable builds.
- The proportion of clean builds rather than incremental local rebuilds.
- Queue waiting time separated from execution time.
- The share of work performed by compile, link, test, archive, signing, and packaging actions.
- The overlap of target graphs across Mac nodes.
- The percentage of actions that depend on machine-local tools or credentials.
Second metric: separate caching from Mac execution
Teams often compare remote caching with adding Mac nodes as if they were interchangeable. They are not.
Remote caching reuses completed outputs. Remote execution changes where eligible actions run. A local Xcode compilation cache is controlled by Xcode and its build environment. Adding a Mac node increases the capacity available for misses, tests, signing, and host-bound work. These four mechanisms solve different bottlenecks.
For iOS CI, a cached compile action may remove repeated compiler work, but the pipeline can still require a real Mac for:
- Xcode-driven build steps.
- Simulator boot and test execution.
- Apple platform validation.
- Code signing and provisioning operations.
- Archive creation and export.
- Toolchain verification after an Xcode or SDK update.
- Scripts that invoke host utilities or rely on local paths.
A useful internal report labels every action as one of four categories:
- Reusable and safe to share.
- Reusable only inside a fixed toolchain pool.
- Mac-bound and not replaceable by cache retrieval.
- Sensitive or release-specific and excluded from shared cache use.
Third metric: measure network and storage efficiency
Remote caching moves data over a network. It does not make transfer free. Compare a cache hit with the counterfactual rebuild on the same Mac configuration, using enterprise logs rather than a vendor-wide assumption about speed.
Record each of these fields for the pilot:
| Metric | Required interpretation | Production question |
|---|---|---|
| Remote hit and miss by action class | Shows which work is actually reusable | Are the expensive actions hitting, or only small intermediates? |
| Download and upload duration | Separates compute saved from transfer time | Is retrieval faster than rebuilding on the target Mac? |
| Transfer failure and retry events | Shows reliability under normal load | Does a cache outage block or merely bypass reuse? |
| Object growth and retention | Captures storage operations and deletion work | Can the team forecast storage without retaining everything? |
| Cold-cache behavior | Establishes the cost of an empty or rebuilt cache | Can the Mac pool absorb the first wave after invalidation? |
| Queue and concurrency impact | Finds contention caused by cache access | Does cache traffic compete with CI source and artifact traffic? |
Do not publish a speed percentage unless your own logs support it. The official Bazel documentation defines the protocol and troubleshooting model, but it does not establish a universal iOS acceleration rate for your repository, Xcode image, target graph, or network.
Storage policy needs the same discipline. Define retention, namespace ownership, object deletion, and behavior after a toolchain change. A cache that grows indefinitely is not a fixed infrastructure cost. It also creates more material to inspect during an incident.
Fourth metric: govern writers, readers, and sensitive targets
A shared cache expands the trust boundary of the build system. Anyone who can write may influence what later jobs retrieve, especially when identities, namespaces, or action inputs are poorly controlled.
Use a permission matrix rather than a single “CI has access” credential:
| Node or actor | Build role | Cache access | Audit evidence | Incident action |
|---|---|---|---|---|
| Pinned release Mac | Archive, signing, release validation | Prefer read-only or isolated namespace | Machine identity, commit, toolchain, target | Disable access and preserve provenance |
| Controlled CI writer | Reproducible compile and test actions | Write only to approved namespace | Job identity, digest, rule revision | Revoke credential and quarantine objects |
| General developer Mac | Local development and diagnosis | Read-only where needed | User, branch, action diagnostics | Remove access if environment is unverified |
| Ephemeral or newly rebuilt node | Trial or recovery work | Read-only until verified | Image hash, provisioning record | Reimage and repeat validation |
| Sensitive target runner | Signing or confidential inputs | Excluded or isolated cache | Target policy and release record | Purge namespace and review artifacts |
The key distinction is between a corrupted cache object and an incorrect but validly stored output. The first may indicate infrastructure failure. The second can indicate undeclared inputs, a compromised writer, or a policy defect. Your response plan should cover both.
A safe pilot includes an invalidation exercise. Disable a writer, remove or quarantine the relevant namespace, rebuild from a clean Mac, and compare the resulting artifact and test evidence. If the team cannot explain how to recover from a bad object, the cache is not ready for release-critical use.
Fifth metric: calculate TCO against real Mac demand
Do not put an invented savings percentage into the business case. Build a variable model that finance and engineering can audit:
Cache TCO = cache infrastructure + storage + network transfer + credentials and monitoring + maintenance time + incident response.
Mac expansion TCO = Mac lease or purchase cost + node provisioning + maintenance + connectivity + idle capacity + replacement and recovery work.
Hybrid TCO = controlled cache cost + baseline Mac pool + peak Mac capacity + operations effort for both systems.
The cache can improve effective capacity when reusable actions account for a meaningful share of expensive work and the remaining Mac-bound tasks do not already saturate the pool. It can fail economically when large outputs cross regions, misses remain high, storage retention is uncontrolled, or engineers spend more time diagnosing inconsistent environments than the cache saves.
Use the following rule set for the investment review:
- If repeated action overlap is high, toolchains are pinned, and retrieval is consistently cheaper than rebuilding, choose a read-only cache pilot.
- If read-only reuse is credible but writers are not yet trusted, choose a controlled writer group and isolate release targets.
- If cache misses dominate queue time, choose additional Mac capacity before expanding storage.
- If signing, simulator, archive, or validation work dominates, choose Mac nodes because those tasks remain Mac-bound.
- If regional transfer or outage behavior is unacceptable, choose a same-region cache, a narrower namespace, or no shared cache.
- If the organization cannot prove artifact provenance, choose no production cache for sensitive targets until governance is fixed.
Run the A/B trial with fixed evidence
A useful trial compares the same submission, target set, toolchain, and Mac class under three conditions:
| Trial lane | Cache policy | What it proves |
|---|---|---|
| A | Cache disabled | Baseline execution, queue, and artifact behavior |
| B | Read-only cache | Whether existing outputs can be reused safely |
| C | Controlled read/write cache | Whether approved producers improve reuse without contamination |
Record action-level hits and misses, transfer events, queue intervals, Mac occupancy, test outcomes, artifact digests, and the identity of every writer. Retain enough evidence for another engineer to reproduce the decision.
The release gate should be explicit:
| Result | Action |
|---|---|
| Reuse is reproducible, access is controlled, and Mac-bound work remains within capacity | Expand gradually by namespace or target class |
| Reuse is promising but misses or toolchain drift are unexplained | Fix the environment and repeat the trial |
| Cache improves compilation but release work still queues | Keep the cache narrow and add or rent Mac capacity |
| Outputs differ, provenance is incomplete, or sensitive data crosses boundaries | Stop the rollout and quarantine the affected path |
| Cache outages block delivery rather than falling back safely | Redesign failure handling before production use |
rules_apple, rules_swift, Bazel, Xcode, or an SDK automatically makes old objects safe or unsafe. Define a version-change policy: namespace separation, explicit invalidation, or a verified compatibility path. Check the current [Apple Xcode system requirements](https://developer.apple.com/xcode/system-requirements/) before treating a Mac image as a valid baseline.
Use the final architecture score
Score each option against your own recorded evidence, not generic claims. A simple internal review can use five dimensions: reuse, reproducibility, network reliability, security readiness, and Mac capacity relief.
| Option | Strong fit when | Weak fit when | Main risk |
|---|---|---|---|
| Remote cache first | Repeated actions and pinned environments dominate | Work is mostly signing, simulator, or archive work | False confidence from partial reuse |
| More Mac nodes | Queue time comes from cache misses or Mac-bound tasks | Macs are idle because duplicate compilation dominates | Paying for capacity that duplication wastes |
| Hybrid cache plus Mac pool | The workload has reusable compile actions and unavoidable Mac tasks | Neither workload split nor ownership is measurable | Two systems with unclear fallback rules |
| No shared cache yet | Toolchain drift, sensitive inputs, or poor provenance exists | Repeated action demand is already measurable | Continued duplicate work and queue pressure |
Enterprise FAQ
Can remote caching replace more Mac build nodes?
No. It can reduce repeated Bazel action execution, but it cannot execute Xcode, simulator, signing, archive, or host-specific tasks. Size Mac capacity from cache misses and Mac-bound work after the trial, not from the total build graph.
Why does rules_apple fail to hit across Macs?
Check the complete action environment: Bazel and rule revisions, Xcode and SDK selection, environment variables, absolute paths, external tools, undeclared inputs, and platform properties. Different Mac models are not automatically the cause, and identical models do not guarantee equivalent action keys.
When should you choose cache expansion over another Mac?
Choose cache expansion when repeated actions are verified, transfer is reliable, and the queue is dominated by duplicate compilation. Choose another Mac when signing, simulator tests, archives, validation, or misses consume the queue. Compare measured transfer and rebuild records rather than relying on a general acceleration claim.
How do you verify that cache outputs are not contaminated?
Run independent cache-disabled and cache-enabled builds from the same commit, compare artifact digests and test results, and retain producer provenance. Restrict writers, quarantine suspicious namespaces, revoke credentials, and rebuild on a clean Mac when an undeclared input or untrusted producer is discovered.
Which nodes should write to the shared cache?
Use a small set of pinned, clean, auditable CI nodes. Keep developer machines and unverified ephemeral agents read-only. Exclude signing and confidential targets unless your security review approves an isolated namespace, explicit credentials, and a tested deletion and recovery process.
If your current setup uses only additional Mac hosts, you may still be paying for duplicate compilation, idle off-peak capacity, manual image maintenance, and queue spikes that could be measured before expansion. If it relies only on a shared cache, it may still lack enough isolated Mac capacity for signing, simulator execution, release archives, and cache misses. After the A/B trial, MACGPU can be evaluated as a periodic baseline or elastic remote Mac pool when you need controlled capacity without committing to a permanent hardware purchase. Explore the available remote Mac options from MACGPU only after your queue and cache evidence identifies the capacity gap.