Apple system models

On Apple silicon, Knowledge can use the operating system's model APIs through a small local Swift helper. There is no model download managed by Knowledge. Apple Intelligence must be enabled and its system model available for generation or image description. Apple manages any required system assets. Apple Foundation Models

Role Backend
Generation, RAG answers and optional enrichment On-device SystemLanguageModel on macOS 26 or later. Knowledge validates the RAG response and citations before returning it.
Image and video-frame descriptions On-device SystemLanguageModel with image input on macOS 27 or later, plus Apple Vision text recognition.
Text embeddings Apple Natural Language sentence embeddings for the configured language and revision. The English revision 1 model returned 512 dimensions on the tested Mac.
Raw image/audio embeddings and speech transcription Choose a separately configured model for these roles. Apple image descriptions and OCR are text representations and can be embedded by the selected text encoder.

Build the helper with Xcode and the macOS 27 SDK, then select the included YAML profile. The sample uses the existing SQLCipher key from the local security setup and stores a separate index for Apple vectors. Sample configuration

sh scripts/build-apple-models.sh
knowledge --config configs/apple.yml models
knowledge --config configs/apple.yml index
knowledge --config configs/apple.yml ask "How often is the sensor calibrated?"

Each role can be selected independently. For example, enable Apple generation with an HTTP or GGUF embedding model, or use Apple text embeddings with remote transcription. The config rejects multiple providers for the same Apple role. Apple sentence embeddings use a separate space, so switching from a previous encoder stages a new index. Language, embedding revision, configured revision and macOS build participate in vector identity. An OS upgrade can therefore trigger re-indexing. Changing models

The helper receives inputs over a private process pipe. It uses the local system model API and a local vector model. It runs one Apple call at a time per model instance, with byte and time limits. Image inputs are bounded and downscaled before model execution. Apple controls its own hardware allocation. When model inference is unavailable to the service account, the call returns an error. The macOS 27 fm CLI is a separate option with machine-wide terms acceptance and is outside this integration. Apple CLI and framework overview

Local verification

On the M3 Mac running macOS 27.2, the helper indexed three OKF pages and an image, and SQLite reported four ready sources. The image produced an indexed description. Vector search for a calibration question ranked the navigation page first and the calibration procedure ahead of the unrelated jam page. Hybrid context placed the calibration procedure first. Apple generation returned “The workshop's carbon dioxide sensor is calibrated every thirty days. [S1]” with S1 resolving to the calibration page.

A sandboxed probe could load the sentence model but the system generation service returned an error. The same synthetic generation and the full Knowledge indexing and RAG run succeeded outside the workspace sandbox. Check availability from the account and launch environment that will run Knowledge.

Read Markdown →