Best Thaluna Models | OpenRouter, Ollama, KoboldCpp

Short answer

If you want the best quality with the least setup, use OpenRouter + Gemini 2.5 Flash Lite.
If you want a simple local setup, use Ollama + gemma3:4b.
If you want the fastest advanced local setup, use KoboldCpp, llama.cpp server, or LM Studio through Thaluna's Local API mode.
If you only want to test the workflow without setup, use Thaluna's built-in offline models, but do not expect the same quality as larger LLMs.

Quick Recommendations

You want...	Use this	Why
Best overall experience	OpenRouter + Gemini 2.5 Flash Lite	Strong quality, low setup, reasonable cost for short OCR text.
Good cloud alternatives	GPT-4o mini or GPT-4.1 nano	Reliable options when Gemini is not preferred or unavailable.
Cheapest cloud option	GPT-OSS 20B on OpenRouter	Lower price, useful for budget testing, but quality may be less consistent.
Easiest local setup	Ollama + gemma3:4b	Good quality/simplicity balance for local translation.
Fastest advanced local setup	KoboldCpp, llama.cpp, or LM Studio + Qwen3 GGUF	More tuning control and often lower latency than a simple Ollama setup.
Basic no-setup testing	Thaluna built-in offline models	Convenient and offline, but intentionally lightweight.

OpenRouter Model Choices

Cloud models are usually the easiest way to improve translation quality because OCR translation requests are short. Prices can change, so always check the provider page before adding credits.

Model	Input / Output price	Best use
Gemini 2.5 Flash Lite	$0.10 / $0.40 per 1M tokens	Recommended default for quality, speed, and cost balance.
GPT-4.1 nano	$0.10 / $0.40 per 1M tokens	Good compact OpenAI option for fast translation.
GPT-4o mini	$0.15 / $0.60 per 1M tokens	Reliable general-purpose alternative.
GPT-OSS 20B	$0.029 / $0.14 per 1M tokens	Budget option when cost matters more than maximum quality.

Local Model Choices

Thaluna built-in offline models

Best for trying the workflow and basic translation without setup. These models are lightweight and mostly below 1B parameters, so rare languages, complex jokes, slang, or visual novel context may be weak.

Ollama + gemma3:4b

The safest local recommendation for most users who want better quality than the built-in models without learning llama.cpp flags. Ollama can have a cold start: the first line may take longer while the model loads, then later translations are usually faster.

Qwen3 4B through Local API

A good speed-focused option for KoboldCpp, llama.cpp server, or LM Studio. A practical starting point is Qwen_Qwen3-4B-Instruct-2507-Q4_K_M.gguf.

Qwen3 1.7B

Useful for weak GPUs, CPU fallback, or very low-latency testing. Quality is lower than 4B+ models, but it can be enough for simple UI and short dialogue.

Gemma 3 4B via llama.cpp/KoboldCpp

Advanced users can try Gemma 3 4B GGUF through a Local API backend. It may be faster than Ollama with similar quality, depending on configuration.

Gemma 3 12B and larger

Better quality is possible on strong hardware, but latency matters for games. For live gameplay, a smaller fast model can feel better than a larger slow model.

Rough VRAM Guidance

These are practical estimates, not guarantees. Real VRAM use depends on quantization, context size, KV cache, backend overhead, GPU drivers, and how the server is configured.

Hardware	Reasonable local choices	Notes
No GPU / weak laptop	OpenRouter, or Qwen3 1.7B if fully local is required	CPU-only local LLMs can be slow. Cloud may feel better.
4-6 GB VRAM	Qwen3 4B Q4_K_M, Gemma 3 4B Q4_K_M, Qwen3 1.7B Q8_0	Use short context such as 4096-8192. If unstable, go smaller.
8 GB VRAM	Qwen3 4B Q4/Q5/Q6, Gemma 3 4B Q4/Q5	Q8 may work, but longer context can make it tight.
12 GB VRAM	Qwen3 4B Q8, Gemma 3 4B Q8, Gemma 3 12B Q4	Good point for higher local quality, but test latency.
16-24 GB VRAM	Gemma 3 12B, larger Qwen models, higher quantizations	For real-time OCR, speed can matter more than maximum model size.

Model file size is not full VRAM usage

A 4 GB model file can need more than 4 GB of VRAM once the backend adds KV cache, context, buffers, and runtime overhead. For Thaluna translation, huge context windows are usually unnecessary. Start with 4096 or 8192 context.

Recommended Settings by Workflow

Fast game dialogue

Use a small capture region, Instant/Fast OCR where appropriate, Game Lens with a low auto-refresh delay, and a fast Local API backend or OpenRouter model.

Best translation quality

Use OpenRouter with Gemini 2.5 Flash Lite, GPT-4.1 nano, or GPT-4o mini. Keep OCR language and target language selected correctly.

Fast local translation

Use Local API / OpenAI-compatible mode with KoboldCpp, llama.cpp server, or LM Studio. Start with Qwen3 4B Q4_K_M and context 4096-8192.

Simple local translation

Use Ollama with gemma3:4b. It is not always the fastest option, but it is easier to set up than manual GGUF servers.

Rare or low-resource languages

Built-in offline model quality can vary. If output becomes mixed-language, unstable, or too literal, use Ollama, OpenRouter, or a stronger Local API model.

RTX 5000 series

If GPU OCR or built-in GPU models hang, set Thaluna OCR and Translation Device to CPU, then use an external backend such as KoboldCpp, llama.cpp, LM Studio, Ollama, or OpenRouter.

Common Mistakes

Judging Thaluna's translation quality only from the built-in offline models. They are convenient, but not the highest-quality option.
Using a huge context window for short OCR lines. It can waste VRAM and slow down local inference.
Running a slow model for fast game subtitles and then blaming OCR timing. The translation backend is often the bottleneck.
Using the wrong OCR language. OCR language is the language on the screen before translation, not the target language.
Expecting RTX 5000 GPU acceleration to work like older RTX cards before upstream framework support is stable.

Need setup steps?

Use the dedicated guides if you already know which backend you want to use.

Local API Guide OpenRouter Guide Hardware Compatibility Complete Guide