Files, media and referenced URLs
The source remains an OKF folder. A normal concept is UTF-8 Markdown with YAML type and optional title/resource/tags/provenance; links and metadata build the graph. Reserved index.md/log.md are navigation/history rather than concepts. Binary assets can be linked from concepts. File changes are hashed and only affected publications are replaced; unchanged vectors/model outputs are reused from cache.
Supported adapters
| Input | Derived representations | Extra requirements |
|---|---|---|
.md, .txt |
Source text, chunks, authored metadata/links | None |
.html, .htm |
Static UTF-8 readable text | Rust HTML parser; no browser/JavaScript |
| PNG/JPEG/WebP | Raw image vectors; optional caption/OCR text | Compatible native decoder/projector or HTTP media encoder; optional vision model |
| WAV/MP3/FLAC/OGG/M4A | Raw audio vectors; optional timestamped transcript | Backend must support the codec; native decoder's tested formats are WAV/MP3/FLAC |
| MP4/M4V/MOV/MKV/WebM/AVI | Sampled frame vectors/descriptions; segmented audio vectors/transcripts | video.enabled, FFmpeg/ffprobe and selected model roles |
| Animated GIF | Timestamped sampled frames | video.enabled and FFmpeg/ffprobe |
| Text chunks per page; optional page image vectors/OCR | pdf.enabled, Poppler; optional projector/vision model |
Recognition is case-insensitive. Recognition does not guarantee every codec in a container is supported. Invalid or unsupported replacements fail visibly and retain previous published content. No unsupported file is silently treated as a successful content embedding. Generic asset metadata is explicitly marked content_processed:false when content models are disabled.
Video and animation
video:
enabled: true
ffmpeg: ffmpeg
ffprobe: ffprobe
frame_interval_ms: 1000
audio_segment_secs: 10
max_frames: 60
max_audio_segments: 60
max_duration_secs: 600
frame_max_edge: 512
max_output_bytes: 67108864
process_timeout_secs: 60
extract_audio: true
Defaults are shown except enabled is false and frame interval is 10000 ms. Frames sample the complete clip at interval boundaries beginning at zero. Audio is decoded to mono 16 kHz PCM WAV segments. Missing audio is allowed. A frame's start_ms is its sample position; end_ms is the next sampling interval boundary, not proof that its visual content persists that long. GIF uses the same sampler; use a short interval for short animations. This is sampled retrieval, not optical flow, temporal action recognition or continuous video understanding.
Budgets that would require truncation are rejected. Limits include duration (up to 24 hours by configuration), counts (up to 1000 each), audio segment length (1–30 seconds), decoded output bytes and per-process wall time. Input video resolution is capped at 33,177,600 pixels (8K). FFmpeg uses fixed arguments, limited demuxers, local file/pipe protocols and one decoding thread; no shell, manifests or remote streams are passed to it. Total adapter time can span multiple per-process timeouts; use host process/container limits for overall native CPU/RAM isolation.
pdf:
enabled: true
pdfinfo: pdfinfo
pdftotext: pdftotext
pdftoppm: pdftoppm
max_pages: 100
render_pages: false
render_max_edge: 768
max_output_bytes: 67108864
process_timeout_secs: 60
These are defaults except enabled is false. Text extraction keeps page boundaries and inserts locator.page (one-based). Empty pages retain their numbering. Poppler performs layout-aware text extraction; complicated tables/columns/encodings may still need another adapter. Password/permission failures are errors, not bypassed restrictions.
Enable rendering for raw page-image vectors or vision OCR. A scanned PDF has no textual RAG context until an OCR/vision backend supplies it; image vectors alone support visual retrieval. A no-text PDF with rendering disabled fails explicitly. Rendering every page may be expensive and is bounded by page/output limits. Native external decoders temporarily receive original bytes in private OS temporary directories; SQLCipher does not encrypt those files. Use encrypted temporary storage/host controls where required.
URL references and snapshots
urls:
enabled: true
allowed_origins: [https://docs.example.com]
allow_loopback_http: false
max_resources_per_document: 16
max_resource_bytes: 16777216
max_total_bytes: 67108864
refresh_secs: 3600
timeout_secs: 30
URL fetching defaults to disabled. Explicit Markdown links/images, top-level resource, and sources[].resource HTTP(S) references are candidates. Scope descriptors, computation/executor/attester code and arbitrary metadata strings are not fetched. Disallowed origins remain graph references with a warning; allowed resources that fail prevent replacing the owning publication. processing.fetch_urls: false opts out per document. Fetching is one-hop: downloaded pages do not become crawlers and no embedded scripts execute.
Exact origins include scheme, host and optional port. Only HTTPS is accepted remotely. Optional loopback HTTP is a deliberate local-service setting. DNS answers are checked and pinned; private/special networks are blocked except explicitly configured literal/localhost loopback. No redirects, cookies, proxy environment, or compressed HTTP bodies are used. HTML parsing is UTF-8-only, at most 2 MiB, 100,000 DOM nodes and depth 64; script/style/head/hidden content is excluded. Static text extraction is not layout rendering, authenticated browsing, CSS evaluation or JavaScript execution.
Watch reconciliation reschedules referencing documents on aligned refresh_secs buckets. Each resource is revalidated once per bucket with ETag/Last-Modified when available; an HTTP 304 reuses bytes. A folder event can trigger processing sooner, but fresh bucket cache entries are reused. Refresh requires the watcher or repeated index invocation. Prior raw URL versions remain in encrypted SQLite tables when encryption is configured. There is no history/cache garbage collection.
Downloaded chunks belong to the referencing OKF concept and carry source_url, remote_sha256 and remote_version_id, plus PDF pages/media timestamps. Original fetched bytes live in remote_versions; remote_resources tracks validation/cache state; transactional remote_references binds active owner publications to versions. Removing a link removes its active derived chunks/references at the next successful publication while preserving history. This is distinct from API capture, which explicitly writes a portable archived page/HTML/assets into the OKF folder.
Whole-bundle snapshots and source/scan limits still apply. Large corpora are currently read into memory during scan; raw media and history consume disk. See limitations and measured evidence.