Hello, I am currently playing with ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf on my 16G Vram 9070XT and I was wondering if I am doing a mistake. I used to use the model with q8/q4 and 128k context. It worked more or less well but when compaction triggers it was blocking everything for 2-3 minutes and it was also slowing down when reaching +- 60% context. I am now trying 64k ctx with q8/q8 and https://github.com/elpapi42/pi-observational-memory It looks like it is working as good despite the obvious more regular/faster compaction but I am wondering if I am fooling myself and missing something... DEVICE="Vulkan0" MODELS_DIR="$HOME/Documents/models" NGL=99 CTX=65536 # Test with pi-observational-memory otherwise 128K and q4 v cache PORT=8080 HOST="0.0.0.0" HEALTH_URL="http://127.0.0.1:${PORT}/health" HEALTH_TIMEOUT=300 USE_MTP="true" CACHE_K="q8_0" CACHE_V="q8_0" # q4_0 if 128k context THREADS=8 # Ryzen 7 5800X: 8 CCD threads REASONING_EFFORT="xhigh" TEMP=1.0 TOP_P=0.95 TOP_K=20 MIN_P=0.0 REPO="ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF" BASE_MODEL="Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf" llama-server \ -m "$MODELS_DIR/$MODEL" \ --device "$DEVICE" \ -c "$CTX" \ -ngl "$NGL" \ -t "$THREADS" \ --threads-batch "$THREADS" \ --flash-attn on \ --cache-type-k "$CACHE_K" \ --cache-type-v "$CACHE_V" \ --jinja \ --reasoning-preserve \ --reasoning-effort "$REASONING_EFFORT" \ --host "$HOST" \ --port "$PORT" \ -np 1 \ --metrics \ --temp "$TEMP" --top-p "$TOP_P" --top-k "$TOP_K" --min-p "$MIN_P" \ "${SPEC_ARGS[@]}" &   submitted by   /u/Tibuski [link]   [comments]