RagIQ.RuntimeEngine

Welcome to the official developer documentation for **`RagIQ.RuntimeEngine`**, a high-performance, native C++ inference runner designed specifically for executing Small Language Models (SLMs) and extracting vector embeddings locally on the CPU.

Built directly on top of the industry-leading **llama.cpp** core, RagIQ.RuntimeEngine compiles to a fully standalone, statically linked executable. This ensures zero dependency conflicts or dynamic dll errors on standard Windows environments.

Why choose RagIQ-RuntimeEngine?

It enables fast, offline inference and semantic embedding extraction directly from your scripts, CLI tools, or background services without needing complex python environments or massive GPU installations.

Installation

RagIQ.RuntimeEngine is officially distributed through Microsoft WinGet. This allows you to install and keep the engine up-to-date with a single command.

Install via WinGet CLI

PS >winget install RagIQ.RuntimeEngine

Static Binary Verification

Unlike standard programs that require installing Microsoft Visual C++ redistributable packages, the WinGet binary is compiled statically (`+crt-static`). This guarantees it runs immediately on clean Windows installations without additional DLL prompts.

Quick Start

Once installed, you can start running native GGUF inference directly. These examples are scoped to IT infrastructure and operations use-cases — perfect for on-prem environments, server monitoring, and incident triage workflows.

1. Server Incident Triage — Analyze Logs Offline

Feed raw syslog or event log excerpts directly to the model for root cause analysis and actionable insights, no cloud API required:

PowerShell / CMD

RagIQ-RuntimeEngine --model "D:\Models\qwen2_1.5b_q4km.gguf" --prompt "Analyze this Windows Event Log error and suggest root cause: Event ID 7034, Service Control Manager, The 'SQL Server' service terminated unexpectedly." --max-tokens 200 --threads 4

2. IT Policy Q&A — Query Internal Runbooks

Query natural-language summaries of incident runbooks or SOP documents stored as text files, fully offline:

PowerShell / CMD

RagIQ-RuntimeEngine --model "D:\Models\qwen2_1.5b_q4km.gguf" --file "D:\Runbooks\disk_alert_sop.txt" --prompt "What are the immediate steps when disk utilization exceeds 90% on a production server?" --max-tokens 150 --threads 4

3. Semantic Indexing — Embed Infrastructure Alerts

Extract high-dimensional vector embeddings from alert descriptions or CMDB notes for offline semantic search and RAG indexing pipelines:

PowerShell / CMD

RagIQ-RuntimeEngine --model "D:\Models\qwen2_1.5b_q4km.gguf" --prompt "Critical: Network interface eth0 packet loss 35% on APP-PROD-02 in rack 4B, datacenter east wing." --embedding

CLI Parameters Reference

Use the search input below to filter through the CLI parameters supported by the `RagIQ-RuntimeEngine` executable:

Parameter Flag	Description	Default Value
-m, --modelRequired	Path to the local GGUF model file on disk.	None
-p, --prompt	The raw prompt string (takes precedence over prompt file option).	None
-f, --file	Path to a file containing the prompt text.	None
-t, --threads	Number of CPU threads to map. Set to physical cores for optimal speed.	2
-c, --ctx-size	Context size limit (number of tokens in memory).	2048
-e, --embedding	Enables high-performance text embedding extraction mode.	Disabled
-n, --max-tokens	Maximum output tokens allowed for text generation.	256
--temp	Temperature for text generation sampling (set to 0 for greedy decoding).	0.7
--top-p	Top-p sampling parameter.	0.9
--repeat-penalty	Repeat penalty parameter for text generation.	1.1
--no-display-prompt	Disables printing the original prompt to stdout.	Disabled

Code Integration

Integrate RagIQ.RuntimeEngine into your automation workflows. Below are working examples in both Python and PowerShell — ideal for IT scripts, monitoring agents, and on-prem pipelines.

Python — Automated Log Triage Script

Invoke the engine from Python to analyze Windows Event Logs or syslog entries and route the result to a ticketing system or Slack alert:

Python Script Example

import subprocess

MODEL_PATH = r"D:\Models\qwen2_1.5b_q4km.gguf"

def analyze_log_entry(log_text: str) -> str:
    """Send a raw log entry to RagIQ-RuntimeEngine and return the AI triage summary."""
    prompt = f"You are an IT SRE assistant. Analyze this log entry and suggest a resolution:\n\n{log_text}"
    cmd = [
        "RagIQ-RuntimeEngine",
        "--model", MODEL_PATH,
        "--prompt", prompt,
        "--max-tokens", "150",
        "--threads", "4",
        "--no-display-prompt"
    ]
    result = subprocess.run(cmd, capture_output=True, text=True, encoding="utf-8")
    if result.returncode == 0:
        return result.stdout.strip()
    raise RuntimeError(f"[RagIQ-RuntimeEngine] Inference failed: {result.stderr}")

# --- Usage ---
log = "Event ID 7034: The 'MSSQLSERVER' service terminated unexpectedly. It has done this 3 time(s)."
print(analyze_log_entry(log))

PowerShell — Server Health Check Automation

Run RagIQ-RuntimeEngine directly from a PowerShell maintenance script to generate natural-language summaries of CPU, disk, and memory alerts across your server fleet:

PowerShell Script Example

# --- RagIQ-RuntimeEngine IT Infra Health Analyzer ---
$ModelPath = "D:\Models\qwen2_1.5b_q4km.gguf"
$Threads   = 4

function Invoke-RagIQAnalysis {
    param (
        [string]$Prompt,
        [int]$MaxTokens = 120
    )
    $output = & RagIQ-RuntimeEngine `
        --model $ModelPath `
        --prompt $Prompt `
        --max-tokens $MaxTokens `
        --threads $Threads `
        --no-display-prompt
    return $output
}

# Example: Analyze disk alert
$DiskAlert = "Server: APP-PROD-07 | Drive: C:\ | Used: 93% | Free: 14 GB"
$prompt    = "Summarize this disk utilization alert and recommend immediate action for an IT admin: $DiskAlert"
$result    = Invoke-RagIQAnalysis -Prompt $prompt

Write-Host "[RagIQ.RuntimeEngine analysis]" -ForegroundColor Cyan
Write-Host $result

# Optionally write result to a report file
$result | Out-File -FilePath "D:\Reports\disk_alert_summary.txt" -Encoding UTF8

Troubleshooting & FAQ

Common questions about hardware compatibility, performance, model support, and runtime troubleshooting for RagIQ.RuntimeEngine:

📊 Real-World CPU Benchmark (Qwen2 1.5B, Q4_K, 4 threads)

Prompt Prefill Eval Speed 159.07 tokens/sec

Text Generation Speed (Q4_K Quantized) 4.37 tokens/sec

What is a "modern CPU" — and which Intel/AMD models support AVX2? +

⚡ AVX2 — 256-bit Vector Math

AVX2 (Advanced Vector Extensions 2) allows the CPU to process 8 floating-point values simultaneously per cycle using 256-bit registers. It directly accelerates every matrix multiply operation inside the SLM attention and MLP layers, delivering 4–8× faster inference versus scalar code.

• Intel: Supported since 2013 on all Haswell (4th Gen Core i3/i5/i7) and newer — i.e. any Core processor from 2013 onward.

• AMD: Supported since 2015 on Excavator and fully optimized across all Zen / Ryzen processors (1000 series and newer, from 2017 onward).

In short: any laptop or desktop bought after 2014 almost certainly supports AVX2.

What is FMA3, and which CPUs support it? +

⚡ FMA3 — Fused Multiply-Add

FMA3 (Fused Multiply-Add, 3-operand) merges multiply and add into a single CPU instruction (A × B + C), halving the instruction count for every GEMM operation and providing up to 2× throughput gain for quantized model inference.

• Intel: Supported since 2013 (Haswell, 4th Gen Core). Any modern Intel laptop or desktop supports FMA3.

• AMD: Supported since 2012 on Piledriver (FX / A-series APUs), and across all Zen / Ryzen architectures from 2017 onward.

Together, AVX2 + FMA3 deliver the 5–8× speedup over plain scalar CPU inference that RagIQ-RuntimeEngine is benchmarked at.

What happens if I don't set --threads? Does the runtime auto-detect? +

✅ Yes — Auto-Detection is Built In

When --threads is not set (or set to 0), RagIQ-RuntimeEngine's Hardware Profile Resolver automatically detects the optimal physical core count for your machine. You do not need to set it manually on most systems.

Here is exactly what happens under the hood when --threads is omitted:

• Step 1: available_parallelism() is called to get total logical thread count from the OS.

• Step 2: RagIQ.RuntimeEngine applies the physical-core heuristic — if logical threads ≤ 4, use all; otherwise use logical ÷ 2 to strip hyperthreaded siblings.

• Step 3: RAM is also detected. If no --ram-gb is provided, it falls back to a safe estimate: 8 GB for ≤ 4-core machines, 16 GB for larger.

• Step 4: The full hardware tier is classified (Low / Medium / High) and batch size, context size, and max tokens are all set automatically from the tier.

Use --threads only when you want to override for a specific benchmark or workload. Otherwise, leave it unset and let the runtime self-tune.

How do I fix DLL missing errors (e.g. MSVCP140.dll)? +

This is already solved in RagIQ.RuntimeEngine. The executable is statically linked with MSVC runtime libraries, so no separate redistributable install is needed. If you are upgrading from an older build, run a clean update: winget upgrade RagIQ.RuntimeEngine

Does RagIQ.RuntimeEngine require a GPU? +

No. RagIQ.RuntimeEngine is purpose-built for high-efficiency CPU-only inference. It leverages AVX2 and FMA3 vectorized instruction sets to execute GGUF models entirely on system RAM — making it ideal for on-prem servers, edge nodes, and air-gapped environments without dedicated GPUs.

What models are supported? +

RagIQ.RuntimeEngine is optimized for the GGUF model format. Compatible families include Qwen2, Llama3, Mistral, and IBM Granite. Simply download the GGUF variant of any supported model and point --model at the file path.

What makes RagIQ.RuntimeEngine different from stock llama.cpp? +

RagIQ.RuntimeEngine is not just a repackaged llama.cpp binary. It ships with a custom Rust-based orchestration layer on top of the llama.cpp core that adds several features absent from the stock CLI tool:

🤖 1. Hardware Profile Auto-Resolver
A built-in resolve_hardware_profile() engine classifies your machine into Low / Medium / High tier by reading physical cores and RAM, then automatically selects optimal batch size, context size, and token limits. Stock llama-cli requires manual tuning of every parameter.

💬 2. Embedding Mode Auto-Detection
RagIQ.RuntimeEngine detects embedding mode not just from --embedding but also from the executable filename itself. If the binary is named with "embedding" in its path, the mode activates automatically — enabling drop-in compatibility for any script that invokes it by name.

🧠 3. Smart JSON Prompt Recovery
When a prompt ends with a closing brace } (e.g. a partially-constructed JSON object from a workflow), RagIQ.RuntimeEngine automatically appends a structured ### Response: continuation trigger. This prevents models from stalling or repeating the prompt in structured-output pipelines.

🤐 4. Silent Logging Mode by Default
Stock llama.cpp prints verbose internal debug logs to stderr. RagIQ.RuntimeEngine suppresses all internal llama.cpp logging by default via a registered silent_log_callback. This makes stdout output clean and directly pipeable into scripts, parsers, or ticketing integrations. Use --verbose to re-enable full diagnostic output.

📌 5. Prompt File Support (-f)
RagIQ.RuntimeEngine natively accepts a --file flag to load the prompt from a text file on disk. This is critical for IT automation scenarios where prompts are dynamically constructed by PowerShell or batch scripts and passed as temp files.

🧱 6. Zero-Dependency Static Binary
The WinGet binary is compiled with static CRT linkage, bundling all Microsoft runtime libraries directly. Unlike stock llama.cpp releases which may require MSVC redistributables, RagIQ-RuntimeEngine runs immediately on a clean Windows install with no pre-requisites.