Real-Time Screen OCR — Drag a Frame Over Any Text, Get Live Translation in 109 Languages Update 28.03.2026
Real-Time Screen OCR — Drag a Frame Over Any Text, Get Live Translation in 109 Languages Update 28.03.2026 A draggable frame that reads any text on your screen and translates it in real time. No copy-paste. No browser extensions. Just hover and read.
109 languages. Real-time OCR. Works on literally anything visible on your screen.
Think of it as a magnifying glass that also speaks every language. You drag a green frame over text — a game menu, a locked PDF, a foreign website, a DRM-protected app — and it reads what’s there, figures out what language it is, and spits out the translation live. No clipboard tricks, no screen recording, no API keys. It just watches and translates.
🧠 How It Actually Works — Plain English, No CS Degree Required
The magic is a loop that runs every ~0.2 seconds:
Step 1 — Screenshot. The green overlay frame captures whatever’s underneath it. Think of it as a camera pointed at a tiny piece of your screen.
Step 2 — OCR (Optical Character Recognition). Tesseract (Google’s open-source text reader) scans the image and pulls out the text. The script runs multiple image processing passes — contrast boosting, edge sharpening, noise filtering — to squeeze out the best possible read. If the frame is large, it splits into tiles and reads each chunk separately, then stitches the results together.
Step 3 — Language Detection. It doesn’t ask you what language the text is in. It checks the characters — Cyrillic? Russian. Kanji? Japanese. Devanagari? Hindi. Arabic script? Arabic. Mixed scripts get scored and the best match wins.
Step 4 — Translation. Google Translate handles the heavy lifting. Results get cached so repeated text doesn’t re-translate. Big blocks get split into safe chunks to avoid API limits.
Step 5 — Display. The translation shows up in a separate window, live-updating as the source text changes. Ctrl+scroll to resize the text. Right-click to copy.
| Component | What It Does |
|---|---|
| Tesseract OCR | Reads text from screenshots — 15+ language packs built in |
| Google Translate | Translates between 109 languages via deep-translator |
| mss | Takes fast screenshots of just the overlay area |
| Pillow | Image processing — contrast, sharpening, binarization |
| Tkinter | The green overlay frame + translation output window |
| LRU Cache | Stores recent translations so it doesn’t re-translate identical text |
The anti-gibberish engine: 1,261 lines of code aren’t just for show. Multiple confidence checks, script detection, transliteration noise filters, and gibberish detectors run on every single OCR pass. If the result looks like random characters, it retries with different image processing. This is why it actually works where simpler tools spit out garbage.
🎯 What You Can Actually Do With This — Fun & Profitable Use Cases
| Use Case | How It Helps |
|---|---|
| Foreign games | Japanese RPGs, Chinese MMOs, Korean visual novels — drag the frame over dialogue, play in your language |
| Locked PDFs | DRM-protected documents that block copy-paste? The screen doesn’t care about DRM. OCR reads the pixels. |
| Secure websites | Banking portals, paywalled content rendered as images, right-click-disabled pages — if you can see it, this reads it |
| Language learning | Keep the frame on foreign content, see translations live. Passive immersion while you browse |
| Foreign software | That Russian dev tool with no English UI? Frame it. Translated. |
| Manga & comics | Overlay on scanned pages, read translations as you scroll |
| Live streams | Foreign Twitch chat, YouTube live subtitles, anything on-screen in real time |
| Research | Non-English academic papers, foreign forums, untranslated documentation |
If text is visible on your monitor, this tool can read it and translate it. No exceptions.
📱 The Mobile-Style Interface — Why It Works So Well
The overlay is a transparent green frame you can drag and resize — just like the translate bubble on Android phones. Move it over text, resize it to focus on specific lines, and the translation window updates live.
Pro tip: If a section translates poorly, shrink the frame to just that chunk of text. Smaller frame = cleaner OCR = better translation. The program adapts its recognition strategy based on frame size and aspect ratio automatically.
⚙️ Setup — 10 Minutes, Then It Just Works
Step 1 — Install Tesseract OCR
Download from UB Mannheim’s Tesseract builds (Windows). During install, check the box for additional language packs — grab everything you might need.
Step 2 — Install Python dependencies
pip install pytesseract Pillow mss deep-translator
tkinter comes bundled with Python on Windows. On Linux: sudo apt install python3-tk.
Step 3 — Run it
Save the script below, then: python ocr_translator.py
The green overlay appears. The translation window opens. Drag the frame. Done.
| Requirement | Details |
|---|---|
| Python | 3.9 or newer |
| Tesseract OCR | CLI binary — auto-detected in Program Files or PATH |
| Language packs | eng, rus, chi_sim, jpn, hin, ara, deu, fra, spa, por, ita, vie, ukr, hye, kat — or any you need |
| Python packages | pytesseract, Pillow, mss, deep-translator |
| Internet | Required for Google Translate. Offline OCR works without it (text capture only) |
💻 The Full Script — 1,500 + Lines, Copy-Paste Ready
For non-coders: Don’t touch this. Save it as a
.pyfile and run it. The code handles everything — Tesseract detection, image processing, language guessing, translation, caching, the overlay UI, all of it.
For coders: The OCR pipeline uses 6 image preprocessing variants (grayscale scaling, median denoising, contrast enhancement, edge sharpening, soft binarization, hard binarization) with multi-PSM Tesseract passes. Confidence scoring weighs script detection, word-level confidence, and text plausibility. Tiled OCR kicks in automatically for large capture areas. The translation layer pools
GoogleTranslatorinstances and chunks long text to stay under API limits. LRU cache holds 1,500 entries. Frame signature hashing skips OCR when nothing changed. It’s genuinely well-engineered.
import hashlib
import os
import queue
import re
import shutil
import sys
import threading
import time
import tkinter as tk
from collections import OrderedDict, deque
from statistics import mean
from tkinter import Menu
import pytesseract
from deep_translator import GoogleTranslator
from mss import mss
from PIL import Image, ImageEnhance, ImageFilter, ImageChops, ImageStat, ImageOps
def find_tesseract():
path = shutil.which("tesseract")
if path:
return path
for c in (
r"C:\Program Files\Tesseract-OCR\tesseract.exe",
r"C:\Program Files (x86)\Tesseract-OCR\tesseract.exe",
):
if os.path.exists(c):
return c
return None
tess = find_tesseract()
if not tess:
print("❌ Tesseract OCR not found")
sys.exit(1)
pytesseract.pytesseract.tesseract_cmd = tess
MIN_ACCEPTED_OCR_CONF = 44.0
MIN_STABLE_FRAMES = 1
MAX_SYMBOL_RATIO = 0.45
MAX_CHUNK = 400
CONTEXT_BLOCK_CHARS = 1100
CAPTURE_INTERVAL = 0.22
CAPTURE_INTERVAL_INTERACT = 0.09
RETRY_INTERVAL_EMPTY = 0.12
RENDER_FORCE_REFRESH = 1.6
AUTO_OCR_REFRESH_SEC = 6.0
TILE_FALLBACK_MIN_ASPECT = 1.9
TILE_FALLBACK_MIN_PIXELS = 280_000
FAST_CONFIDENCE_SHORT_CIRCUIT = 72.0
MAX_OCR_RUNTIME_SEC = 1.8
MAX_OCR_RUNTIME_SEC_LARGE = 1.15
MIN_WORD_CONF = 26.0
MOTION_DIFF_THRESHOLD = 16.0
MOTION_MIN_CONF = 58.0
LAST_GOOD_HOLD_SEC = 1.8
TEXT_LIKELIHOOD_MIN = 0.12
TILE_TEXT_LIKELIHOOD_MIN = 0.14
GEOMETRY_CHANGE_RELAX_SEC = 0.9
OCR_LANGS_BY_SOURCE = {
"en": "eng",
"zh-CN": "chi_sim",
"ja": "jpn",
"hi": "hin",
"vi": "vie",
"de": "deu",
"fr": "fra",
"es": "spa",
"pt": "por",
"it": "ita",
"ar": "ara",
"ru": "rus+eng",
"uk": "ukr+rus+eng",
"hy": "hye",
"ka": "kat",
}
SCRIPT_CANDIDATES = {
"Latin": ["en", "de", "fr", "es", "pt", "it", "vi"],
"Arabic": ["ar"],
"Han": ["zh-CN", "ja"],
"Hiragana": ["ja"],
"Katakana": ["ja"],
"Devanagari": ["hi"],
"Cyrillic": ["ru", "uk"],
"Armenian": ["hy"],
"Georgian": ["ka"],
}
DEFAULT_SOURCE_CANDIDATES = ["en", "ru", "uk", "hy", "ka", "zh-CN", "ja", "hi", "vi", "de", "fr", "es", "pt", "it", "ar"]
TARGET_LABELS = {
"af": "Afrikaans",
"sq": "Albanian",
"am": "Amharic",
"ar": "Arabic",
"hy": "Armenian",
"az": "Azerbaijani",
"eu": "Basque",
"be": "Belarusian",
"bn": "Bengali",
"bs": "Bosnian",
"bg": "Bulgarian",
"ca": "Catalan",
"ceb": "Cebuano",
"ny": "Chichewa",
"zh-CN": "Chinese (Simplified)",
"zh-TW": "Chinese (Traditional)",
"co": "Corsican",
"hr": "Croatian",
"cs": "Czech",
"da": "Danish",
"nl": "Dutch",
"en": "English",
"eo": "Esperanto",
"et": "Estonian",
"tl": "Filipino",
"fi": "Finnish",
"fr": "French",
"fy": "Frisian",
"gl": "Galician",
"ka": "Georgian",
"de": "German",
"el": "Greek",
"gu": "Gujarati",
"ht": "Haitian Creole",
"ha": "Hausa",
"haw": "Hawaiian",
"he": "Hebrew",
"hi": "Hindi",
"hmn": "Hmong",
"hu": "Hungarian",
"is": "Icelandic",
"ig": "Igbo",
"id": "Indonesian",
"ga": "Irish",
"it": "Italian",
"ja": "Japanese",
"jw": "Javanese",
"kn": "Kannada",
"kk": "Kazakh",
"km": "Khmer",
"rw": "Kinyarwanda",
"ko": "Korean",
"ku": "Kurdish",
"ky": "Kyrgyz",
"lo": "Lao",
"la": "Latin",
"lv": "Latvian",
"lt": "Lithuanian",
"lb": "Luxembourgish",
"mk": "Macedonian",
"mg": "Malagasy",
"ms": "Malay",
"ml": "Malayalam",
"mt": "Maltese",
"mi": "Maori",
"mr": "Marathi",
"mn": "Mongolian",
"my": "Myanmar (Burmese)",
"ne": "Nepali",
"no": "Norwegian",
"or": "Odia",
"ps": "Pashto",
"fa": "Persian",
"pl": "Polish",
"pt": "Portuguese",
"pa": "Punjabi",
"ro": "Romanian",
"ru": "Russian",
"sm": "Samoan",
"gd": "Scots Gaelic",
"sr": "Serbian",
"st": "Sesotho",
"sn": "Shona",
"sd": "Sindhi",
"si": "Sinhala",
"sk": "Slovak",
"sl": "Slovenian",
"so": "Somali",
"es": "Spanish",
"su": "Sundanese",
"sw": "Swahili",
"sv": "Swedish",
"tg": "Tajik",
"ta": "Tamil",
"tt": "Tatar",
"te": "Telugu",
"th": "Thai",
"tr": "Turkish",
"tk": "Turkmen",
"uk": "Ukrainian",
"ur": "Urdu",
"ug": "Uyghur",
"uz": "Uzbek",
"vi": "Vietnamese",
"cy": "Welsh",
"xh": "Xhosa",
"yi": "Yiddish",
"yo": "Yoruba",
"zu": "Zulu",
}
EXTRA_RARE_LABELS = {
"ce": "Chechen",
"ba": "Bashkir",
"cv": "Chuvash",
}
def build_runtime_target_labels():
base = dict(TARGET_LABELS)
try:
supported = GoogleTranslator.get_supported_languages(as_dict=True)
allowed = set(supported.values())
for code, label in EXTRA_RARE_LABELS.items():
if code in allowed:
base[code] = label
except Exception:
pass
return base
TARGET_LABELS = build_runtime_target_labels()
LANG_LABELS = {code: code.split("-")[0].upper() for code in TARGET_LABELS}
class LRUCache:
def __init__(self, max_items=300):
self.data = OrderedDict()
self.max = max_items
def get(self, k):
if k in self.data:
self.data.move_to_end(k)
return self.data[k]
return None
def set(self, k, v):
self.data[k] = v
self.data.move_to_end(k)
if len(self.data) > self.max:
self.data.popitem(last=False)
def clear(self):
self.data.clear()
cache = LRUCache(max_items=1500)
translator_pool = {
("en", "ru"): GoogleTranslator(source="en", target="ru"),
("en", "uk"): GoogleTranslator(source="en", target="uk"),
("ru", "en"): GoogleTranslator(source="ru", target="en"),
("ru", "uk"): GoogleTranslator(source="ru", target="uk"),
("uk", "ru"): GoogleTranslator(source="uk", target="ru"),
("uk", "en"): GoogleTranslator(source="uk", target="en"),
("hy", "ru"): GoogleTranslator(source="hy", target="ru"),
("hy", "en"): GoogleTranslator(source="hy", target="en"),
("ka", "ru"): GoogleTranslator(source="ka", target="ru"),
("ka", "en"): GoogleTranslator(source="ka", target="en"),
}
def text_hash(text: str) -> str:
return hashlib.md5(text.encode("utf-8")).hexdigest()
def fast_frame_signature(image: Image.Image) -> str:
"""Fast frame hash to skip OCR when the capture region is unchanged."""
thumb = image.resize((64, 36), Image.Resampling.BILINEAR).convert("L")
return hashlib.md5(thumb.tobytes()).hexdigest()
def frame_motion_score(prev_thumb: Image.Image, current_thumb: Image.Image) -> float:
if prev_thumb is None or current_thumb is None:
return 0.0
try:
diff = ImageChops.difference(prev_thumb, current_thumb)
return float(ImageStat.Stat(diff).mean[0])
except Exception:
return 0.0
def text_likelihood_score(image: Image.Image) -> float:
try:
gray = image.convert("L").resize((160, 90), Image.Resampling.BILINEAR)
std = ImageStat.Stat(gray).stddev[0] / 64.0
edges = gray.filter(ImageFilter.FIND_EDGES)
edge_bin = edges.point(lambda px: 255 if px > 44 else 0)
edge_ratio = (sum(edge_bin.histogram()[200:256]) / max(1, 160 * 90))
dark_bin = gray.point(lambda px: 255 if px < 170 else 0)
dark_ratio = (sum(dark_bin.histogram()[200:256]) / max(1, 160 * 90))
score = 0.45 * min(std, 1.0) + 0.4 * min(edge_ratio * 5.0, 1.0) + 0.15 * min(dark_ratio * 2.0, 1.0)
return max(0.0, min(1.0, score))
except Exception:
return 0.0
def stabilize_temporal_text(current_text: str, history: deque) -> str:
if not current_text:
return current_text
history.append(current_text)
if len(history) < 3:
return current_text
counts = {}
sample = {}
for t in history:
h = text_hash(t)
counts[h] = counts.get(h, 0) + 1
sample[h] = t
best_h = max(counts, key=counts.get)
if counts[best_h] >= 2:
return sample[best_h]
return current_text
def _extract_script_name(osd_text: str) -> str:
for line in osd_text.splitlines():
if line.lower().startswith("script:"):
return line.split(":", 1)[1].strip()
return ""
def get_ocr_candidates(image: Image.Image):
try:
osd = pytesseract.image_to_osd(image)
script = _extract_script_name(osd)
candidates = SCRIPT_CANDIDATES.get(script)
if candidates:
return candidates
except Exception:
pass
return DEFAULT_SOURCE_CANDIDATES
def detect_source_lang(text: str) -> str:
if re.search(r"[\u3040-\u30ff]", text):
return "ja"
if re.search(r"[\u4e00-\u9fff]", text):
return "zh-CN"
if re.search(r"[\u0900-\u097f]", text):
return "hi"
if re.search(r"[\u0600-\u06ff]", text):
return "ar"
if re.search(r"[А-Яа-яЁё]", text):
return "ru"
if re.search(r"[-֏]", text):
return "hy"
if re.search(r"[Ⴀ-ჿ]", text):
return "ka"
return "en"
def stabilize_source_hint(text: str, source_hint: str) -> str:
"""Refine OCR source language to reduce false DE/FR guesses on Cyrillic text."""
normalized = normalize_confusable_cyrillic(text)
detected = detect_source_lang(normalized)
latin_family = {"en", "de", "fr", "es", "pt", "it", "vi"}
if source_hint in latin_family and detected in {"ru", "uk", "hy", "ka", "ar", "hi", "ja", "zh-CN"}:
return detected
letters = sum(ch.isalpha() for ch in normalized)
if letters >= 14 and detected != source_hint:
return detected
return source_hint or detected
def _script_penalty(text: str) -> float:
letters = [c for c in text if c.isalpha()]
if not letters:
return 0.0
latin = sum("a" <= c.lower() <= "z" or "À" <= c <= "ÿ" for c in letters)
cyr = sum("а" <= c.lower() <= "я" or c in "ёЁ" for c in letters)
if latin and cyr:
ratio = min(latin, cyr) / max(latin, cyr)
return 25.0 * ratio
return 0.0
def normalize_confusable_cyrillic(text: str) -> str:
cyr_hits = len(re.findall(r"[А-Яа-яЁё]", text))
lat_hits = len(re.findall(r"[A-Za-z]", text))
total = cyr_hits + lat_hits
if total == 0 or lat_hits == 0:
return text
if cyr_hits < 4 or (cyr_hits / total) < 0.34:
return text
table = str.maketrans({
"A": "А", "a": "а", "B": "В", "C": "С", "c": "с", "E": "Е", "e": "е", "H": "Н", "K": "К", "k": "к", "M": "М",
"O": "О", "o": "о", "P": "Р", "p": "р", "T": "Т", "X": "Х", "x": "х", "Y": "У", "y": "у",
})
return text.translate(table)
def min_conf_threshold_for_text(text: str) -> float:
letters = sum(ch.isalpha() for ch in text)
if letters <= 22:
return 26.0
if letters <= 48:
return 34.0
return MIN_ACCEPTED_OCR_CONF
def preprocess_variants_for_ocr(image: Image.Image):
gray = image.convert("L")
w, h = gray.size
base_scale = 2.25
max_side = 2600
max_pixels = 3_600_000
side_scale = min(max_side / max(w, 1), max_side / max(h, 1))
area_scale = (max_pixels / max(w * h, 1)) ** 0.5
adaptive_scale = min(base_scale, side_scale, area_scale)
adaptive_scale = max(1.0, adaptive_scale)
scaled = gray.resize(
(max(1, int(w * adaptive_scale)), max(1, int(h * adaptive_scale))),
Image.Resampling.LANCZOS,
)
denoised = scaled.filter(ImageFilter.MedianFilter(size=3))
autocontrast = ImageOps.autocontrast(denoised, cutoff=1)
contrast = ImageEnhance.Contrast(autocontrast).enhance(1.75)
sharp = ImageEnhance.Sharpness(contrast).enhance(2.15)
edge = sharp.filter(ImageFilter.EDGE_ENHANCE_MORE)
gamma_dark = autocontrast.point(lambda px: int(((px / 255.0) ** 0.85) * 255))
gamma_bright = autocontrast.point(lambda px: int(((px / 255.0) ** 1.25) * 255))
soft = contrast.point(lambda px: 255 if px > 176 else 0)
hard = edge.point(lambda px: 255 if px > 152 else 0)
inv_hard = ImageOps.invert(gamma_bright).point(lambda px: 255 if px > 150 else 0)
return [scaled, denoised, autocontrast, contrast, edge, gamma_dark, soft, hard, inv_hard]
def is_plausible_text(text: str) -> bool:
if len(text.strip()) < 8:
return False
letters_digits = sum(ch.isalnum() for ch in text)
symbols = sum(not ch.isalnum() and not ch.isspace() for ch in text)
total = max(len(text), 1)
if symbols / total > MAX_SYMBOL_RATIO or letters_digits < 6:
return False
chunks = re.findall(r"[A-Za-zА-Яа-яЁё\u0600-\u06ff\u0900-\u097f\u3040-\u30ff\u4e00-\u9fff]+", text)
if not chunks:
return False
return sum(1 for c in chunks if len(c) == 1) <= len(chunks) // 2
def looks_like_translit_noise(text: str) -> bool:
bad_patterns = [r"[A-Za-z][А-Яа-яЁё]", r"[А-Яа-яЁё][A-Za-z]"]
mixed_hits = sum(len(re.findall(p, text)) for p in bad_patterns)
return mixed_hits >= 8
def looks_like_gibberish(text: str, avg_conf: float = 0.0) -> bool:
stripped = text.strip()
if len(stripped) < 8:
return True
letters = sum(ch.isalpha() for ch in stripped)
digits = sum(ch.isdigit() for ch in stripped)
spaces = stripped.count(" ")
weird = len(re.findall(r"[@#$%^&*_+=~`|\/<>]", stripped))
if letters < 5 and weird >= 2:
return True
if re.search(r"(.)\1{4,}", stripped):
return True
total = max(len(stripped), 1)
if (weird / total) > 0.20:
return True
words = [w for w in stripped.split() if w]
if not words:
return True
one_char_words = sum(1 for w in words if len(w) == 1)
if len(words) >= 5 and one_char_words / len(words) > 0.55:
return True
if len(stripped) <= 18 and avg_conf < 58:
return True
if letters and digits > letters * 1.5:
return True
if any(len(w) > 34 for w in words) and spaces < 2:
return True
return False
def expected_script_for_source(source: str) -> str:
return {
"ru": "cyrillic", "ar": "arabic", "hi": "devanagari", "zh-CN": "han", "ja": "japanese", "hy": "armenian", "ka": "georgian"
}.get(source, "latin")
def score_ocr_result(text: str, confidences, source_lang: str):
letters = sum(ch.isalpha() for ch in text)
if letters < 8:
return -9999.0
valid_conf = [c for c in confidences if c > 0]
conf_score = mean(valid_conf) if valid_conf else 0.0
length_bonus = min(len(text), 320) / 12.0
punctuation_penalty = text.count(" ") * 0.5
expected = expected_script_for_source(source_lang)
low = text.lower()
latin_hits = len(re.findall(r"[a-zà-ÿ]", low))
cyr_hits = len(re.findall(r"[а-яё]", low))
ar_hits = len(re.findall(r"[\u0600-\u06ff]", text))
hi_hits = len(re.findall(r"[\u0900-\u097f]", text))
ja_hits = len(re.findall(r"[\u3040-\u30ff]", text))
han_hits = len(re.findall(r"[\u4e00-\u9fff]", text))
script_bonus = 0.0
if expected == "latin":
script_bonus = latin_hits * 0.03 - (cyr_hits + ar_hits + hi_hits) * 0.05
elif expected == "cyrillic":
script_bonus = cyr_hits * 0.05 - latin_hits * 0.03
elif expected == "arabic":
script_bonus = ar_hits * 0.05 - latin_hits * 0.03
elif expected == "devanagari":
script_bonus = hi_hits * 0.05 - latin_hits * 0.03
elif expected == "han":
script_bonus = han_hits * 0.04 - latin_hits * 0.03
elif expected == "japanese":
script_bonus = (ja_hits + han_hits) * 0.04 - latin_hits * 0.03
return conf_score + length_bonus + script_bonus - _script_penalty(text) - punctuation_penalty
def _extract_confident_words(data, min_conf: float = MIN_WORD_CONF):
words = []
confs = []
raw_words = data.get("text", [])
raw_confs = data.get("conf", [])
for i, w in enumerate(raw_words):
word = (w or "").strip()
if not word:
continue
try:
conf = float(raw_confs[i]) if i < len(raw_confs) else -1.0
except Exception:
conf = -1.0
if conf >= min_conf:
words.append(word)
confs.append(conf)
if len(words) < 4:
words = []
confs = []
for i, w in enumerate(raw_words):
word = (w or "").strip()
if not word:
continue
try:
conf = float(raw_confs[i]) if i < len(raw_confs) else -1.0
except Exception:
conf = -1.0
if conf > 0:
words.append(word)
confs.append(conf)
return words, confs
def _merge_text_parts(parts):
merged = []
for part in parts:
p = clean_ocr(part)
if not p:
continue
if not merged:
merged.append(p)
continue
prev = merged[-1]
if p == prev or p in prev:
continue
max_ol = min(60, len(prev), len(p))
overlap = 0
for n in range(max_ol, 9, -1):
if prev[-n:] == p[:n]:
overlap = n
break
if overlap:
p = p[overlap:].strip()
if not p:
continue
merged.append(p)
return " ".join(merged).strip()
def _fast_tile_ocr(tile: Image.Image, candidates):
if text_likelihood_score(tile) < TILE_TEXT_LIKELIHOOD_MIN:
return "", (candidates[0] if candidates else "en"), 0.0
best_text = ""
best_source = candidates[0] if candidates else "en"
best_conf = 0.0
best_score = -10_000.0
variants = preprocess_variants_for_ocr(tile)[:2]
langs = candidates[:3] if candidates else ["en"]
for variant in variants:
for source in langs:
tess_lang = OCR_LANGS_BY_SOURCE[source]
try:
data = pytesseract.image_to_data(
variant,
lang=tess_lang,
config="--oem 3 --psm 6",
output_type=pytesseract.Output.DICT,
timeout=0.7,
)
words, confs = _extract_confident_words(data)
text = normalize_confusable_cyrillic(" ".join(words))
source = stabilize_source_hint(text, source)
if not text or not is_plausible_text(text) or looks_like_translit_noise(text):
continue
avg_conf = mean(confs) if confs else 0.0
if looks_like_gibberish(text, avg_conf):
continue
score = score_ocr_result(text, confs, source)
if score > best_score:
best_score = score
best_text = text
best_source = source
best_conf = avg_conf
except Exception:
continue
return best_text, best_source, best_conf
def perform_ocr_tiled(image: Image.Image):
w, h = image.size
candidates = get_ocr_candidates(image)
area = w * h
cols = 2
rows = 2
if w >= h * 1.8:
cols, rows = 4, 2
elif h >= w * 1.8:
cols, rows = 2, 4
elif area >= 520_000:
cols, rows = 3, 3
x_step = w / cols
y_step = h / rows
x_ov = int(x_step * 0.14)
y_ov = int(y_step * 0.14)
tiles = []
for ry in range(rows):
for cx in range(cols):
left = max(0, int(cx * x_step) - x_ov)
right = min(w, int((cx + 1) * x_step) + x_ov)
top = max(0, int(ry * y_step) - y_ov)
bottom = min(h, int((ry + 1) * y_step) + y_ov)
tiles.append((left, top, right, bottom, ry, cx))
texts = []
confs = []
source = "en"
tiles.sort(key=lambda t: (t[4], t[5]))
for left, top, right, bottom, _ry, _cx in tiles:
tile = image.crop((left, top, right, bottom))
t, s_lang, c = _fast_tile_ocr(tile, candidates)
if not t:
continue
texts.append(t)
confs.append(c)
source = s_lang
if not texts:
return "", source, 0.0
merged = _merge_text_parts(texts)
source = stabilize_source_hint(merged, source)
return merged, source, (mean(confs) if confs else 0.0)
def _token_signature(token: str) -> str:
token = normalize_confusable_cyrillic(token)
return re.sub(r"[^\wЀ-ӿ-ۿऀ-ॿ-ヿ一-鿿]", "", token.lower())
def build_consensus_text(candidates):
if not candidates:
return "", "en", 0.0
sorted_candidates = sorted(candidates, key=lambda c: c["score"], reverse=True)
base = sorted_candidates[0]
token_lists = [c["text"].split() for c in sorted_candidates[:5] if c["text"]]
if not token_lists:
return base["text"], base["source"], base["conf"]
max_len = max(len(toks) for toks in token_lists)
consensus_tokens = []
for i in range(max_len):
votes = {}
originals = {}
for cand, toks in zip(sorted_candidates[:5], token_lists):
if i >= len(toks):
continue
tok = toks[i]
sig = _token_signature(tok)
if not sig:
continue
weight = max(1.0, cand["score"] / 25.0)
votes[sig] = votes.get(sig, 0.0) + weight
if sig not in originals or len(tok) > len(originals[sig]):
originals[sig] = tok
if votes:
best_sig = max(votes, key=votes.get)
consensus_tokens.append(originals[best_sig])
elif i < len(base["text"].split()):
consensus_tokens.append(base["text"].split()[i])
consensus = clean_ocr(" ".join(consensus_tokens))
if not consensus:
consensus = base["text"]
source = stabilize_source_hint(consensus, base["source"])
conf = mean([c["conf"] for c in sorted_candidates[:5]]) if sorted_candidates else base["conf"]
return consensus, source, conf
def perform_ocr(image: Image.Image):
best_text = ""
best_source = "en"
best_score = -10_000.0
best_conf = 0.0
started_at = time.time()
candidate_pool = []
candidates = get_ocr_candidates(image)
variants = preprocess_variants_for_ocr(image)
w, h = image.size
large_frame = (w * h) >= 900_000
runtime_limit = MAX_OCR_RUNTIME_SEC_LARGE if large_frame else MAX_OCR_RUNTIME_SEC
if large_frame:
variants = variants[:6]
primary_langs = candidates[:4] if len(candidates) > 4 else candidates
stages = [
(variants[:4], primary_langs, (6,)),
([variants[1], variants[2], variants[4], variants[5]], primary_langs, (6, 11)),
]
if not large_frame:
stages.append((variants, candidates, (6, 4, 11)))
for stage_variants, stage_langs, psm_modes in stages:
for variant in stage_variants:
for source in stage_langs:
tess_lang = OCR_LANGS_BY_SOURCE[source]
for psm in psm_modes:
if time.time() - started_at > runtime_limit:
if candidate_pool:
return build_consensus_text(candidate_pool)
return best_text, best_source, best_conf
try:
data = pytesseract.image_to_data(
variant,
lang=tess_lang,
config=f"--oem 3 --psm {psm}",
output_type=pytesseract.Output.DICT,
timeout=1.0,
)
words, confs = _extract_confident_words(data)
text = normalize_confusable_cyrillic(" ".join(words))
source = stabilize_source_hint(text, source)
if not text or not is_plausible_text(text) or looks_like_translit_noise(text):
continue
score = score_ocr_result(text, confs, source)
avg_conf = mean(confs) if confs else 0.0
if looks_like_gibberish(text, avg_conf):
continue
candidate_pool.append({"text": text, "source": source, "score": score, "conf": avg_conf})
if len(candidate_pool) > 18:
candidate_pool = sorted(candidate_pool, key=lambda c: c["score"], reverse=True)[:18]
if score > best_score:
best_score = score
best_text = text
best_source = source
best_conf = avg_conf
if best_conf >= FAST_CONFIDENCE_SHORT_CIRCUIT and len(best_text) >= 35 and len(candidate_pool) >= 3:
return build_consensus_text(candidate_pool)
except Exception:
continue
if candidate_pool:
return build_consensus_text(candidate_pool)
return best_text, best_source, best_conf
def clean_ocr(text: str) -> str:
lines = []
for l in text.splitlines():
l = l.strip()
if len(l) < 3:
continue
if re.fullmatch(r"[+\-*•]?\s*\d{3,4}\s*[—\-]?\s*", l):
continue
lines.append(normalize_confusable_cyrillic(l))
cleaned = re.sub(r"\s{2,}", " ", " ".join(lines)).strip()
cleaned = re.sub(r"([@#$%^&*_+=~`|\/<>]){2,}", " ", cleaned)
return re.sub(r"\s{2,}", " ", cleaned).strip()
def split_paragraphs(text: str):
parts = [p.strip() for p in re.split(r"(?<=[.!?。！？])\s{1,}", text) if len(p.strip()) >= 8]
if parts:
return parts
words = text.split()
if len(words) < 4:
return []
chunk_size = 28
chunks = []
for i in range(0, len(words), chunk_size):
chunk = " ".join(words[i:i + chunk_size]).strip()
if len(chunk) >= 8:
chunks.append(chunk)
return chunks
def split_safe_chunks(text: str):
sentences = re.split(r"(?<=[.!?。！？])\s+", text)
chunks, current = [], ""
def flush_long(part: str):
words = part.split()
if not words:
return []
out, cur = [], ""
for w in words:
if len((cur + " " + w).strip()) <= MAX_CHUNK:
cur = (cur + " " + w).strip()
else:
if cur:
out.append(cur)
cur = w
if cur:
out.append(cur)
return out
for snt in sentences:
snt = snt.strip()
if not snt:
continue
if len(snt) > MAX_CHUNK:
if current:
chunks.append(current)
current = ""
chunks.extend(flush_long(snt))
continue
if len(current) + len(snt) <= MAX_CHUNK:
current = f"{current} {snt}".strip()
else:
if current:
chunks.append(current)
current = snt
if current:
chunks.append(current)
return chunks
def postprocess_text(text: str) -> str:
return re.sub(r"\s+([,.!?])", r"\1", text).strip()
def get_translator(source_lang: str, target_lang: str):
key = (source_lang, target_lang)
tr = translator_pool.get(key)
if tr:
return tr
try:
tr = GoogleTranslator(source=source_lang, target=target_lang)
except Exception:
tr = GoogleTranslator(source="auto", target=target_lang)
translator_pool[key] = tr
return tr
def translate_one(text: str, source_lang: str, target_lang: str) -> str:
if source_lang == target_lang:
return text
key = f"{source_lang}->{target_lang}:{text}"
cached = cache.get(key)
if cached:
return cached
out = []
for part in split_safe_chunks(text):
translated_part = ""
try:
tr = get_translator(source_lang, target_lang).translate(part)
translated_part = postprocess_text(tr)
except Exception:
translated_part = ""
if (not translated_part) or (translated_part.strip().lower() == part.strip().lower() and source_lang != target_lang):
try:
tr = GoogleTranslator(source="auto", target=target_lang).translate(part)
translated_part = postprocess_text(tr)
except Exception:
pass
if translated_part:
out.append(translated_part)
result = " ".join(out).strip() or "[translation unavailable]"
cache.set(key, result)
return result
def split_context_blocks(paragraphs):
blocks = []
current = ""
for p in paragraphs:
p = p.strip()
if not p:
continue
candidate = (current + "\n\n" + p).strip() if current else p
if len(candidate) <= CONTEXT_BLOCK_CHARS:
current = candidate
else:
if current:
blocks.append(current)
current = p
if current:
blocks.append(current)
return blocks
def coherence_postprocess(text: str) -> str:
text = re.sub(r"\s{2,}", " ", text)
text = re.sub(r"\s*\n+\s*", "\n", text)
text = re.sub(r"\n{3,}", "\n\n", text)
return text.strip()
def translate_coherent_paragraphs(paragraphs, source_hint: str = "", target_lang: str = "ru"):
joined = "\n\n".join([p for p in paragraphs if p.strip()]).strip()
if not joined:
return {"source": source_hint or "en", "target": target_lang, "translation": ""}
source_lang = source_hint or detect_source_lang(joined)
key = f"coherent:{source_lang}->{target_lang}:{text_hash(joined)}"
cached = cache.get(key)
if cached:
return {"source": source_lang, "target": target_lang, "translation": cached}
blocks = split_context_blocks(paragraphs)
out = []
for block in blocks:
out.append(translate_one(block, source_lang, target_lang))
result = coherence_postprocess("\n\n".join(out))
cache.set(key, result)
return {"source": source_lang, "target": target_lang, "translation": result}
class Overlay(tk.Tk):
def __init__(self):
super().__init__()
self.overrideredirect(True)
self.attributes("-topmost", True)
self.attributes("-alpha", 0.45)
self.configure(bg="#00ff88")
self.geometry("460x220+260+220")
self.min_w, self.min_h = 120, 60
self.border_width = 4
self.inner = tk.Frame(self, bg="black")
self.inner.place(x=self.border_width, y=self.border_width, relwidth=1.0, relheight=1.0, width=-(self.border_width * 2), height=-(self.border_width * 2))
self.hint = tk.Label(self.inner, text="", bg="black", fg="#00ff88", font=("Segoe UI", 10, "bold"))
self.hint.place_forget()
self.margin = 8
self.edge = ""
self.mode = None
self.start = None
self.interacting = False
self.bind("<Motion>", self.detect_edge)
self.bind("<ButtonPress-1>", self.start_action)
self.bind("<B1-Motion>", self.perform_action)
self.bind("<ButtonRelease-1>", self.end_action)
self.bind("<Leave>", lambda _e: self.config(cursor="arrow"))
def detect_edge(self, e):
w, h = self.winfo_width(), self.winfo_height()
self.edge = ""
if e.x < self.margin:
self.edge += "w"
if e.x > w - self.margin:
self.edge += "e"
if e.y < self.margin:
self.edge += "n"
if e.y > h - self.margin:
self.edge += "s"
cursor_map = {
"n": "top_side",
"s": "bottom_side",
"e": "right_side",
"w": "left_side",
"ne": "top_right_corner",
"nw": "top_left_corner",
"se": "bottom_right_corner",
"sw": "bottom_left_corner",
}
self.config(cursor=cursor_map.get(self.edge, "fleur"))
def start_action(self, e):
w, h, x, y = self.parse_geometry(self.geometry())
self.mode = "resize" if self.edge else "move"
self.start = {"x": x, "y": y, "w": w, "h": h, "xr": e.x_root, "yr": e.y_root}
if not self.interacting:
self.interacting = True
self.event_generate("<<OverlayInteractionStart>>", when="tail")
def perform_action(self, e):
if not self.start:
return
dx = e.x_root - self.start["xr"]
dy = e.y_root - self.start["yr"]
w0, h0, x0, y0 = self.start["w"], self.start["h"], self.start["x"], self.start["y"]
if self.mode == "move":
self.geometry(f"{w0}x{h0}+{x0 + dx}+{y0 + dy}")
return
left, top = x0, y0
right, bottom = x0 + w0, y0 + h0
if "w" in self.edge:
left = x0 + dx
if "e" in self.edge:
right = x0 + w0 + dx
if "n" in self.edge:
top = y0 + dy
if "s" in self.edge:
bottom = y0 + h0 + dy
if right - left < self.min_w:
if "w" in self.edge:
left = right - self.min_w
else:
right = left + self.min_w
if bottom - top < self.min_h:
if "n" in self.edge:
top = bottom - self.min_h
else:
bottom = top + self.min_h
new_w = int(max(self.min_w, right - left))
new_h = int(max(self.min_h, bottom - top))
self.geometry(f"{new_w}x{new_h}+{int(left)}+{int(top)}")
def end_action(self, _):
self.start = None
self.mode = None
if self.interacting:
self.interacting = False
self.event_generate("<<OverlayInteractionEnd>>", when="tail")
@staticmethod
def parse_geometry(geo):
g, p = geo.split("+", 1)
w, h = map(int, g.split("x"))
x, y = map(int, p.split("+"))
return w, h, x, y
class OCRReader:
def __init__(self):
self.running = True
self.overlay = Overlay()
self.output = tk.Toplevel()
self.output.title("Translation")
self.output.geometry("960x580+760+240")
self.output.attributes("-topmost", True)
controls = tk.Frame(self.output)
controls.pack(fill="x", padx=8, pady=(8, 4))
tk.Label(controls, text="Target language:").pack(side="left")
self.language_picker = None
self.target_entries = sorted(TARGET_LABELS.items(), key=lambda item: item[1].lower())
self.target_display_to_code = {
f"{idx:03d}. {label}": code
for idx, (code, label) in enumerate(self.target_entries, start=1)
}
default_display = next((k for k, v in self.target_display_to_code.items() if v == "ru"), next(iter(self.target_display_to_code), "001. English"))
self.target_var = tk.StringVar(value=default_display)
self.target_menu = tk.Button(
controls,
textvariable=self.target_var,
width=36,
anchor="w",
command=self.open_language_picker,
)
self.target_menu.pack(side="left", padx=(8, 0))
self.font_size = 13
self.text = tk.Text(self.output, wrap="word", font=("Segoe UI", self.font_size))
self.text.pack(expand=True, fill="both")
self.text.bind("<Control-MouseWheel>", self.scale_text)
self.text.insert(tk.END, "Waiting for OCR...")
self.menu = Menu(self.output, tearoff=0)
self.menu.add_command(label="Copy", command=self.copy)
self.menu.add_command(label="Select all", command=self.select_all)
self.text.bind("<Button-3>", self.show_menu)
self.text.bind("<Control-c>", self.copy_shortcut)
self.text.bind("<Control-C>", self.copy_shortcut)
self.last_hash = None
self.last_update = 0.0
self.pending_hash = None
self.pending_hits = 0
self.last_frame_sig = ""
self.last_ocr_result = ("", "en", 0.0)
self.capture_area = self._snapshot_area()
self.force_refresh = True
self.last_render_payload = ""
self.last_ocr_time = 0.0
self.last_render_time = time.time()
self.prev_thumb = None
self.last_good_cleaned = ""
self.last_good_source = "en"
self.last_good_conf = 0.0
self.last_good_time = 0.0
self.recent_cleaned_history = deque(maxlen=6)
self.overlay_interacting = False
self.latest_requested_version = 0
self.last_geometry_change = time.time()
self.translation_jobs = queue.Queue(maxsize=1)
self.render_queue = queue.Queue(maxsize=2)
self.overlay.bind("<Control-Shift-Escape>", self.on_close)
self.output.bind("<Control-Shift-q>", self.on_close)
self.output.bind("<Control-Shift-Q>", self.on_close)
self.overlay.bind("<Configure>", self._on_overlay_configure)
self.overlay.bind("<<OverlayInteractionStart>>", self._on_overlay_interaction_start)
self.overlay.bind("<<OverlayInteractionEnd>>", self._on_overlay_interaction_end)
self.overlay.protocol("WM_DELETE_WINDOW", self.on_close)
self.output.protocol("WM_DELETE_WINDOW", self.on_close)
self.keep_topmost = True
self.topmost_pause_until = 0.0
self._enforce_topmost()
threading.Thread(target=self.translation_worker, daemon=True).start()
threading.Thread(target=self.loop, daemon=True).start()
self.output.after(80, self._flush_render_queue)
def _enforce_topmost(self):
if not self.running or not self.keep_topmost:
return
if time.time() < self.topmost_pause_until:
self.output.after(220, self._enforce_topmost)
return
try:
self.overlay.attributes("-topmost", True)
self.output.attributes("-topmost", True)
except Exception:
pass
self.output.after(550, self._enforce_topmost)
def _flush_render_queue(self):
if not self.running:
return
try:
while True:
payload = self.render_queue.get_nowait()
if payload and payload != self.last_render_payload:
self.text.delete("1.0", tk.END)
self.text.insert(tk.END, payload)
self.last_render_payload = payload
self.last_render_time = time.time()
except queue.Empty:
pass
self.output.after(30, self._flush_render_queue)
def _snapshot_area(self):
border = max(0, int(getattr(self.overlay, "border_width", 0)))
width = max(2, self.overlay.winfo_width() - border * 2)
height = max(2, self.overlay.winfo_height() - border * 2)
return {
"top": self.overlay.winfo_y() + border,
"left": self.overlay.winfo_x() + border,
"width": width,
"height": height,
}
def _on_overlay_configure(self, _=None):
new_area = self._snapshot_area()
if new_area == self.capture_area:
return
self.capture_area = new_area
self.last_frame_sig = ""
self.force_refresh = True
self.last_geometry_change = time.time()
def _on_overlay_interaction_start(self, _=None):
self.overlay_interacting = True
self.force_refresh = True
def _on_overlay_interaction_end(self, _=None):
self.overlay_interacting = False
self.pending_hash = None
self.pending_hits = 0
self.force_refresh = True
def open_language_picker(self):
if self.language_picker and self.language_picker.winfo_exists():
self.language_picker.lift()
self.language_picker.focus_force()
return
self.topmost_pause_until = time.time() + 3600.0
self.language_picker = tk.Toplevel(self.output)
self.language_picker.title("Select target language")
self.language_picker.geometry("420x520")
self.language_picker.transient(self.output)
self.language_picker.attributes("-topmost", True)
self.language_picker.protocol("WM_DELETE_WINDOW", self.close_language_picker)
frame = tk.Frame(self.language_picker)
frame.pack(fill="both", expand=True, padx=8, pady=8)
scroll = tk.Scrollbar(frame)
scroll.pack(side="right", fill="y")
self.lang_listbox = tk.Listbox(frame, yscrollcommand=scroll.set, activestyle="none")
self.lang_listbox.pack(side="left", fill="both", expand=True)
scroll.config(command=self.lang_listbox.yview)
for item in self.target_display_to_code.keys():
self.lang_listbox.insert(tk.END, item)
current = self.target_var.get().strip()
keys = list(self.target_display_to_code.keys())
if current in self.target_display_to_code:
idx = keys.index(current)
self.lang_listbox.selection_set(idx)
self.lang_listbox.see(max(0, idx - 4))
self.lang_listbox.bind("<Double-Button-1>", self.pick_language_from_list)
self.lang_listbox.bind("<Return>", self.pick_language_from_list)
btns = tk.Frame(self.language_picker)
btns.pack(fill="x", padx=8, pady=(0, 8))
tk.Button(btns, text="Select", command=self.pick_language_from_list).pack(side="right")
tk.Button(btns, text="Close", command=self.close_language_picker).pack(side="right", padx=(0, 8))
self.language_picker.focus_force()
def pick_language_from_list(self, _=None):
if not self.language_picker or not self.language_picker.winfo_exists():
return
cur = self.lang_listbox.curselection()
if not cur:
return
selected = self.lang_listbox.get(cur[0])
self.target_var.set(selected)
self.on_target_change()
self.close_language_picker()
def close_language_picker(self):
if self.language_picker and self.language_picker.winfo_exists():
self.language_picker.destroy()
self.language_picker = None
self.topmost_pause_until = time.time() + 0.6
def on_target_menu_open(self, _=None):
self.topmost_pause_until = time.time() + 2.0
def get_target_lang(self):
selected = self.target_var.get().strip()
return self.target_display_to_code.get(selected, "ru")
def on_target_change(self, _=None):
self.last_hash = None
self.topmost_pause_until = time.time() + 0.6
def on_close(self, _=None):
self.running = False
self.keep_topmost = False
cache.clear()
self.close_language_picker()
try:
if self.output.winfo_exists():
self.output.destroy()
except Exception:
pass
try:
if self.overlay.winfo_exists():
self.overlay.destroy()
except Exception:
pass
def scale_text(self, event):
self.font_size += 1 if event.delta > 0 else -1
self.font_size = max(10, min(30, self.font_size))
self.text.config(font=("Segoe UI", self.font_size))
def show_menu(self, e):
self.topmost_pause_until = time.time() + 2.0
try:
self.menu.tk_popup(e.x_root, e.y_root)
finally:
self.menu.grab_release()
def copy_shortcut(self, _=None):
self.copy()
return "break"
def copy(self):
try:
sel = self.text.get(tk.SEL_FIRST, tk.SEL_LAST)
self.output.clipboard_clear()
self.output.clipboard_append(sel)
except tk.TclError:
pass
def select_all(self):
self.text.tag_add(tk.SEL, "1.0", tk.END)
def area(self):
return dict(self.capture_area)
def _format_translated_block(self, translated_paragraphs):
blocks = []
for item in translated_paragraphs:
src = LANG_LABELS.get(item["source"], item["source"].upper())
tgt = LANG_LABELS.get(item["target"], item["target"].upper())
blocks.append(f"{tgt} ({src}→{tgt}): {item['translation']}")
return "\n\n".join(blocks)
def translation_worker(self):
while self.running:
try:
version, target_lang, source_hint, paragraphs = self.translation_jobs.get(timeout=0.3)
except queue.Empty:
continue
if version < self.latest_requested_version:
continue
unique = []
seen = set()
for p in paragraphs:
p = p.strip()
if not p or p in seen:
continue
seen.add(p)
unique.append(p)
translated_item = translate_coherent_paragraphs(unique, source_hint=source_hint, target_lang=target_lang)
translated = [translated_item] if translated_item.get("translation") else []
payload = self._format_translated_block(translated) if translated else self.last_render_payload
if version < self.latest_requested_version:
continue
try:
if self.render_queue.full():
self.render_queue.get_nowait()
self.render_queue.put_nowait(payload)
except queue.Full:
pass
def loop(self):
with mss() as sct:
while self.running:
try:
now = time.time()
img = sct.grab(self.area())
image = Image.frombytes("RGB", img.size, img.rgb)
thumb = image.resize((64, 36), Image.Resampling.BILINEAR).convert("L")
motion_score = frame_motion_score(self.prev_thumb, thumb)
self.prev_thumb = thumb
motion_heavy = motion_score >= MOTION_DIFF_THRESHOLD
frame_text_likelihood = text_likelihood_score(image)
a = self.area()
frame_sig = f"{a['left']}:{a['top']}:{a['width']}:{a['height']}:{fast_frame_signature(image)}"
periodic_refresh_due = (now - self.last_ocr_time) >= AUTO_OCR_REFRESH_SEC
should_run_ocr = self.force_refresh or self.overlay_interacting or periodic_refresh_due or frame_sig != self.last_frame_sig
if should_run_ocr:
raw, source_hint, avg_conf = perform_ocr(image)
self.last_frame_sig = frame_sig
self.last_ocr_result = (raw, source_hint, avg_conf)
self.last_ocr_time = now
else:
raw, source_hint, avg_conf = self.last_ocr_result
cleaned = clean_ocr(raw)
source_hint = stabilize_source_hint(cleaned, source_hint)
geometry_change_recent = (now - self.last_geometry_change) <= GEOMETRY_CHANGE_RELAX_SEC
if frame_text_likelihood < TEXT_LIKELIHOOD_MIN and not self.last_good_cleaned and not self.overlay_interacting:
time.sleep(RETRY_INTERVAL_EMPTY)
continue
if motion_heavy and avg_conf < MOTION_MIN_CONF and not geometry_change_recent:
time.sleep(RETRY_INTERVAL_EMPTY)
continue
if looks_like_gibberish(cleaned, avg_conf):
if (not geometry_change_recent) and self.last_good_cleaned and (now - self.last_good_time) <= LAST_GOOD_HOLD_SEC:
cleaned = self.last_good_cleaned
source_hint = self.last_good_source
avg_conf = max(avg_conf, self.last_good_conf)
if cleaned and not looks_like_gibberish(cleaned, avg_conf):
cleaned = stabilize_temporal_text(cleaned, self.recent_cleaned_history)
self.last_good_cleaned = cleaned
self.last_good_source = source_hint
self.last_good_conf = avg_conf
self.last_good_time = now
if not cleaned:
raw_fallback = re.sub(r"\s+", " ", (raw or "")).strip()
if sum(ch.isalnum() for ch in raw_fallback) >= 4:
cleaned = raw_fallback
area = self.area()
area_pixels = area["width"] * area["height"]
aspect = max(area["width"], area["height"]) / max(1, min(area["width"], area["height"]))
need_tile_fallback = area_pixels >= TILE_FALLBACK_MIN_PIXELS and aspect >= TILE_FALLBACK_MIN_ASPECT
if (not self.overlay_interacting) and need_tile_fallback and (not cleaned or avg_conf < MIN_ACCEPTED_OCR_CONF or len(cleaned) < 60 or area_pixels > 700_000):
tiled_raw, tiled_source, tiled_conf = perform_ocr_tiled(image)
tiled_cleaned = clean_ocr(tiled_raw)
base_letters = sum(ch.isalpha() for ch in cleaned)
tiled_letters = sum(ch.isalpha() for ch in tiled_cleaned)
if tiled_letters > base_letters + 12 or len(tiled_cleaned) > len(cleaned) + 20:
cleaned = tiled_cleaned
source_hint = tiled_source
avg_conf = max(avg_conf, tiled_conf)
required_conf = min_conf_threshold_for_text(cleaned)
if self.overlay_interacting:
required_conf = max(22.0, required_conf - 8.0)
if not cleaned or avg_conf < required_conf:
if not self.last_render_payload and (now - self.last_render_time) > 2.5:
try:
if self.render_queue.full():
self.render_queue.get_nowait()
self.render_queue.put_nowait("Waiting for readable text...")
except Exception:
pass
time.sleep(RETRY_INTERVAL_EMPTY)
continue
paragraphs = split_paragraphs(cleaned)
if not paragraphs:
time.sleep(RETRY_INTERVAL_EMPTY)
continue
stable_window = cleaned
h = text_hash(stable_window)
if self.pending_hash == h:
self.pending_hits += 1
else:
self.pending_hash = h
self.pending_hits = 1
min_stable_frames = 1 if self.overlay_interacting else MIN_STABLE_FRAMES
if not self.force_refresh and self.pending_hits < min_stable_frames:
time.sleep(0.1)
continue
target_lang = self.get_target_lang()
render_scope = ""
if self.overlay_interacting:
render_scope = f":{area['left']}:{area['top']}:{area['width']}:{area['height']}"
render_key = text_hash(f"{target_lang}:{stable_window}{render_scope}")
if self.force_refresh or render_key != self.last_hash or now - self.last_update > RENDER_FORCE_REFRESH:
if self.translation_jobs.full():
try:
self.translation_jobs.get_nowait()
except queue.Empty:
pass
self.latest_requested_version += 1
self.translation_jobs.put_nowait((self.latest_requested_version, target_lang, source_hint, paragraphs))
self.last_hash = render_key
self.last_update = now
self.force_refresh = False
except Exception as e:
print("OCR error:", e)
time.sleep(CAPTURE_INTERVAL_INTERACT if self.overlay_interacting else CAPTURE_INTERVAL)
if __name__ == "__main__":
OCRReader()
tk.mainloop()
⚠️ Tips & Gotchas — Read This Before You Rage-Quit
| Issue | Fix |
|---|---|
| Translation is wrong/choppy | Shrink the frame to just the text you need. Smaller area = cleaner OCR read |
| Tesseract not found | Install from UB Mannheim, make sure it’s in C:\Program Files\Tesseract-OCR\ or in your PATH |
| No translation appearing | Check internet connection — Google Translate needs it. OCR capture works offline but translation doesn’t |
| Wrong language detected | The auto-detection is good but not magic. If it keeps guessing wrong, manually resize the frame to isolate text in one language |
| Gibberish output | Normal on complex backgrounds, small fonts, or decorative text. The 1,261-line engine filters most of it — but screenshots with heavy graphics behind text will always be harder |
The frame is your remote control. Move it, resize it, focus it. The tighter you frame the text, the better the read. Think of it like focusing a camera lens.
Quick Hits
| Want | Do |
|---|---|
 Play foreign games in your language |
→ Drag frame over dialogue, pick target language, play |
 Read locked PDFs |
→ Frame over the page, OCR ignores DRM |
 Translate any website |
→ Works on right-click-disabled and image-rendered pages too |
 Offline text capture |
→ Works without internet (translation needs it, OCR doesn’t) |
If you can see it on your screen, this reads it. 109 languages. Zero copy-paste. Just hover.
!