VLLM

VLLM backend configuration for LightOn text extraction.

LightOnTextVLLMConfig

Bases: BaseModel

VLLM backend config for LightOn text extraction.

Uses VLLM for high-throughput GPU inference with: - PagedAttention for efficient KV cache - Continuous batching - Optimized CUDA kernels

Example

from omnidocs.tasks.text_extraction import LightOnTextExtractor
from omnidocs.tasks.text_extraction.lighton import LightOnTextVLLMConfig

extractor = LightOnTextExtractor(
    backend=LightOnTextVLLMConfig(
        tensor_parallel_size=1,
        gpu_memory_utilization=0.85,
    )
)
result = extractor.extract(image)