Skip to content

Configuration parameters

This page documents field ownership, types, defaults and value constraints, grouped by the component that reads them. For backend selection and configuration methods, see Cache Configuration in the developer guide.

This structure is for vLLM UCM YAML. The SGLang entry format is in its quickstart.

ucm_connectors:
  - ucm_connector_name: UcmPipelineStore
    ucm_connector_config:
      store_pipeline: "Cache|Posix"
      storage_backends: /mnt/ucm-cache
enable_metrics: true

Root fields configure engine integration; fields inside ucm_connector_config configure the selected Store. Configure only stages in use; backend guides describe other settings.

Engine Connector

These fields belong at the YAML root. Descriptions identify model- or engine-specific restrictions.

Parameter Required Type Value Range Description
use_layerwise Optional bool Default: true Enable layer-wise (per-layer) load/save mode. Recommended true; DeepSeek V4 series recommends false.
enable_event_sync Optional bool Default: true Performance optimization switch. Recommended to enable.
persist_token_threshold Optional int 0 When request length < persist_token_threshold, UCM does not process the request.
wa_dump_block_wise Optional bool true Only used in FAWA connector. true: every block's WA cache is dumped (high frequency); false: only dump last block's WA cache of each chunk prefill (low frequency).
load_tokens_threshold Optional int 0 Minimum token threshold for triggering KV cache loading. Only effective for DeepSeek V4 series. When external hit tokens > load_tokens_threshold, triggers KV Cache loading.
enable_record_traces Optional bool false Record request information (timestamps, input length, output length, etc.).
enable_metrics Optional bool Default: true Whether to enable metrics collection.
use_lite Optional bool false Enable UCM Lite. Does not save/load KV Cache data, only saves and queries metadata. Used to evaluate KV Cache hit rate — no acceleration effect.
metrics_config_path Optional string User-configured Custom metrics config file path. Enables UCM online monitoring via toolkit. Reference config: examples/metrics/metrics_configs.yaml.

Store selection and tasks

Select ucm_connector_name in ucm_connectors[]; the following fields belong inside its ucm_connector_config. store_pipeline applies only to UcmPipelineStore.

Parameter Required Type Value Range Description
store_pipeline Required for Pipeline Store string See valid values below Registered pipeline name; set it explicitly in vLLM YAML. SGLang selects Posix in its adapter.
timeout_ms Optional int Default: 30000 Cache transfer and Posix I/O task timeout in milliseconds. Place it inside ucm_connector_config, not at YAML root.

Posix filesystem stage

These options configure Posix in a Pipeline. Provide storage_backends when Posix is used.

Parameter Required Type Value Range Description
storage_backends Required string User-configured, multiple mount points separated by : Local directory or mount point. Multiple mount points are separated by colons.
io_direct Optional bool Default: true Enable Direct I/O (bypass OS page cache). false: uses PageCache; true: skips PageCache.
posix_io_engine Optional string Default: psync File I/O mode. psync: synchronous; aio: asynchronous, requires io_direct=true.
posix_data_trans_concurrency Optional int Default: 128 Read/write threads per card in psync mode. NFS over RDMA: 128/card. Not used in aio mode.
posix_open_concurrency Optional int Default: 32 File open threads in aio mode. Not applicable in psync.
posix_commit_concurrency Optional int Default: 4 File rename threads in aio mode. Not applicable in psync.
posix_lookup_concurrency Optional int Default: 16 Threads for checking file existence at mount point.

Cache host-buffer stage

These fields apply to pipelines containing Cache. Budget shared buffers per sharing scope and private buffers across all workers on the host.

Parameter Required Type Value Range Description
cache_buffer_capacity_gb Optional int See description Native Cache Store defaults to 256 GiB with shared buffers, or 32 GiB per worker when share_buffer_enable is false. The vLLM connector supplies 128 GiB when shared buffers are enabled and capacity is omitted. Shared buffers default on for MLA. A positive explicit capacity overrides the native defaults; budget all unshared workers on the host.
use_dpc Optional bool Default: false Host buffer allocation in non-shared-buffer mode. Defaults to huge pages (legacy DirectIO path); set true to use the driver memory API (DPC path).
cache_sdma_direct Optional bool Depends on build env: true when PLATFORM=ascend-a3, false otherwise Enable SDMA H2D/D2H transfer. Only effective on A3 devices. Recommended to disable.
cache_load_backend_only Optional bool Default: false Force load from SSD even on cache hit. Test only.
cache_io_aggregation Optional bool Default: false, auto-enabled when PLATFORM=ascend and model is V4 Enable IO aggregation H2D transfer. Only effective on A2 devices.
share_buffer_enable Optional bool MLA: default enabled; GQA: default disabled Enable shared memory. MLA without shm or GQA with shm causes performance degradation.

Posix capacity reclamation

Capacity settings do not reserve filesystem space. Coordinate GC ownership for shared namespaces instead of letting independent instances reclaim against separate budgets.

Parameter Required Type Value Range Description
posix_capacity_gb Optional int Default: 0 (no GC); must not exceed mounted filesystem available capacity Max disk storage capacity (GB). Triggers GC when used >= posix_capacity_gb * posix_gc_trigger_threshold_ratio. In multi-instance deployments sharing the same filesystem, only one instance should enable GC; others should not.
posix_gc_trigger_threshold_ratio Conditional float Default: 0.7, 0~1. Not set when posix_capacity_gb is not configured GC trigger threshold ratio. Used with posix_capacity_gb.
posix_gc_recycle_percent Optional float Default: 0.1, 0~1. Not set when posix_capacity_gb is not configured Ratio of current capacity deleted per GC round.
posix_gc_max_recycle_count_per_shard Optional int Default: 50000, >0. Not recommended to modify. Not set when posix_capacity_gb is not configured Max file deletion count per directory per GC round.
posix_gc_shard_sample_ratio Optional float Default: 0.1, 0~1. Not set when posix_capacity_gb is not configured Sample 10% directories to estimate total capacity.
posix_gc_check_interval_sec Optional int Default: 30, >0. Not set when posix_capacity_gb is not configured GC sampling and trigger interval.
posix_gc_concurrency Optional int Default: 16, >0. Not set when posix_capacity_gb is not configured GC thread pool worker count.
posix_gc_task_timeout_ms Optional int Default: 300000, >0. Not set when posix_capacity_gb is not configured Single directory task timeout watchdog. 0 = disabled.
posix_gc_precise_mode Optional bool Default: true. Not set when posix_capacity_gb is not configured true: precise mode (global coldest); false: performance mode (per-directory coldest).

Pipeline health checks

Place these fields in ucm_connector_config.store_health. See storage health for probing and recovery.

Parameter Required Type Value Range Description
enabled Optional bool Default: true Master switch for storage isolation. Adds circuit breaker for disk KV cache.
health_check_interval_s Optional number Default: 10 Disk health check interval (sec). Must be >0 and > health_check_timeout_s.
health_check_timeout_s Optional number Default: 3 Single probe timeout (sec). Must be >0 and < health_check_interval_s.
health_window_size Optional int Default: 8 Fault statistics window. Must be positive and >= failure_threshold.
failure_threshold Optional int Default: 2 Fault trigger threshold. Must be positive and <= health_window_size.

Registered pipelines

Check that the build contains the selected stages. Empty/Fake and their combinations are test configurations, not persistence backends.

Pipeline Purpose
Cache|Posix Normal use case
Cache|Empty MLA pure Cache test
Cache|Fake MLA/GQA pure Cache test (precision risk)
Empty All Store interfaces empty; engine never hits
Fake No actual load/dump, stores metadata only; tests peak performance
Mooncake Mooncake memory pool; vllm-ascend only
Mooncake|Posix Mooncake memory pool with disk persistence
YuanRong YuanRong memory pool
YuanRong|Posix YuanRong memory pool with disk persistence
Cache|Ds3fs I/O through the 3FS client
Cache|Compress|Posix Compress payloads before filesystem storage

Other backend options are in the 3FS, Mooncake, compression guides.

Defaults come from engine integration, stage global_config.h files and store_health_config.h. See how caching works for ownership and filesystem configuration for operation.