Files, media and referenced URLs

The source remains an OKF folder. A normal concept is UTF-8 Markdown with YAML type and optional title/resource/tags/provenance; links and metadata build the graph. Reserved index.md/log.md are navigation/history rather than concepts. Binary assets can be linked from concepts. File changes are hashed and only affected publications are replaced; unchanged vectors/model outputs are reused from cache.

Supported adapters

Input Derived representations Extra requirements
.md, .txt Source text, chunks, authored metadata/links None
.html, .htm Static UTF-8 readable text Rust HTML parser; no browser/JavaScript
PNG/JPEG/WebP Raw image vectors; optional caption/OCR text Compatible native decoder/projector or HTTP media encoder; optional vision model
WAV/MP3/FLAC/OGG/M4A Raw audio vectors; optional timestamped transcript Backend must support the codec; native decoder's tested formats are WAV/MP3/FLAC
MP4/M4V/MOV/MKV/WebM/AVI Sampled frame vectors/descriptions; segmented audio vectors/transcripts video.enabled, FFmpeg/ffprobe and selected model roles
Animated GIF Timestamped sampled frames video.enabled and FFmpeg/ffprobe
PDF Text chunks per page; optional page image vectors/OCR pdf.enabled, Poppler; optional projector/vision model

Recognition is case-insensitive. Recognition does not guarantee every codec in a container is supported. Invalid or unsupported replacements fail visibly and retain previous published content. No unsupported file is silently treated as a successful content embedding. Generic asset metadata is explicitly marked content_processed:false when content models are disabled.

Video and animation

video:
  enabled: true
  ffmpeg: ffmpeg
  ffprobe: ffprobe
  frame_interval_ms: 1000
  audio_segment_secs: 10
  max_frames: 60
  max_audio_segments: 60
  max_duration_secs: 600
  frame_max_edge: 512
  max_output_bytes: 67108864
  process_timeout_secs: 60
  extract_audio: true

Defaults are shown except enabled is false and frame interval is 10000 ms. Frames sample the complete clip at interval boundaries beginning at zero. Audio is decoded to mono 16 kHz PCM WAV segments. Missing audio is allowed. A frame's start_ms is its sample position; end_ms is the next sampling interval boundary, not proof that its visual content persists that long. GIF uses the same sampler; use a short interval for short animations. This is sampled retrieval, not optical flow, temporal action recognition or continuous video understanding.

Budgets that would require truncation are rejected. Limits include duration (up to 24 hours by configuration), counts (up to 1000 each), audio segment length (1–30 seconds), decoded output bytes and per-process wall time. Input video resolution is capped at 33,177,600 pixels (8K). FFmpeg uses fixed arguments, limited demuxers, local file/pipe protocols and one decoding thread; no shell, manifests or remote streams are passed to it. Total adapter time can span multiple per-process timeouts; use host process/container limits for overall native CPU/RAM isolation.

PDF

pdf:
  enabled: true
  pdfinfo: pdfinfo
  pdftotext: pdftotext
  pdftoppm: pdftoppm
  max_pages: 100
  render_pages: false
  render_max_edge: 768
  max_output_bytes: 67108864
  process_timeout_secs: 60

These are defaults except enabled is false. Text extraction keeps page boundaries and inserts locator.page (one-based). Empty pages retain their numbering. Poppler performs layout-aware text extraction; complicated tables/columns/encodings may still need another adapter. Password/permission failures are errors, not bypassed restrictions.

Enable rendering for raw page-image vectors or vision OCR. A scanned PDF has no textual RAG context until an OCR/vision backend supplies it; image vectors alone support visual retrieval. A no-text PDF with rendering disabled fails explicitly. Rendering every page may be expensive and is bounded by page/output limits. Native external decoders temporarily receive original bytes in private OS temporary directories; SQLCipher does not encrypt those files. Use encrypted temporary storage/host controls where required.

URL references and snapshots

urls:
  enabled: true
  allowed_origins: [https://docs.example.com]
  allow_loopback_http: false
  max_resources_per_document: 16
  max_resource_bytes: 16777216
  max_total_bytes: 67108864
  refresh_secs: 3600
  timeout_secs: 30

URL fetching defaults to disabled. Explicit Markdown links/images, top-level resource, and sources[].resource HTTP(S) references are candidates. Scope descriptors, computation/executor/attester code and arbitrary metadata strings are not fetched. Disallowed origins remain graph references with a warning; allowed resources that fail prevent replacing the owning publication. processing.fetch_urls: false opts out per document. Fetching is one-hop: downloaded pages do not become crawlers and no embedded scripts execute.

Exact origins include scheme, host and optional port. Only HTTPS is accepted remotely. Optional loopback HTTP is a deliberate local-service setting. DNS answers are checked and pinned; private/special networks are blocked except explicitly configured literal/localhost loopback. No redirects, cookies, proxy environment, or compressed HTTP bodies are used. HTML parsing is UTF-8-only, at most 2 MiB, 100,000 DOM nodes and depth 64; script/style/head/hidden content is excluded. Static text extraction is not layout rendering, authenticated browsing, CSS evaluation or JavaScript execution.

Watch reconciliation reschedules referencing documents on aligned refresh_secs buckets. Each resource is revalidated once per bucket with ETag/Last-Modified when available; an HTTP 304 reuses bytes. A folder event can trigger processing sooner, but fresh bucket cache entries are reused. Refresh requires the watcher or repeated index invocation. Prior raw URL versions remain in encrypted SQLite tables when encryption is configured. There is no history/cache garbage collection.

Downloaded chunks belong to the referencing OKF concept and carry source_url, remote_sha256 and remote_version_id, plus PDF pages/media timestamps. Original fetched bytes live in remote_versions; remote_resources tracks validation/cache state; transactional remote_references binds active owner publications to versions. Removing a link removes its active derived chunks/references at the next successful publication while preserving history. This is distinct from API capture, which explicitly writes a portable archived page/HTML/assets into the OKF folder.

Whole-bundle snapshots and source/scan limits still apply. Large corpora are currently read into memory during scan; raw media and history consume disk. See limitations and measured evidence.

Read Markdown →