Bilingual archive search with cited answers
A regional publisher needed research answers its editors could trace to dependable sources. Over 18 weeks we built the retrieval pipeline, search and reading interface, and evaluation tools for its Turkish and English archive.
Screens
The challenge
Plausible answers often came with unusable citations. Turkish word forms also caused keyword search to miss relevant material, making source reliability the central problem.
Our approach
We combined keyword and semantic retrieval with a Turkish-aware analyser and a reranker trained on examples labelled by the client's editors. The system declines low-confidence questions. Each release runs against a 600-question evaluation set, split equally between the two languages and expanded when editors find a bad answer.
Outcome
- Citation precision improved from 61% to 89% across both languages
- Low-confidence questions declined rather than answered with unsupported sources
- Release evaluations running in the client's CI after model and index changes
- Archive research time reduced by about a third
“Evals over vibes turned out to be the whole argument. It says it does not know often enough that we believe it when it does answer.”Deniz Arslan — Archive editor, Mürekkep Yayıncılık
Original interfaces and workflows are protected by NDA. Images are UI recreations using synthetic data and figures. Client identity is shared with permission.