Project · Research preview
Embedding and reranking models for local semantic search and RAG — trained, audited, and published with the mistakes left in the write-up.
Swedish RAG often starts from multilingual encoders that treat Swedish as a minority language, or from strong public Swedish baselines you may not want to depend on for every deployment story. I wanted a European-foundation, Swedish-first retrieval path that runs locally: fast candidate recall, then optional cross-encoder precision — before a chat model like Europa answers.
Both models are explicitly labelled research preview. They are not marketed as production defaults or best-in-class Swedish embeddings.
Pipeline
Start from EuroBERT-210m (Apache 2.0) — a multilingual encoder built for European languages.
Masked-language training on cleaned Swedish web text, Swedish Wikipedia, and English replay — then drop the MLM head.
Contrastive query/document learning with in-batch and hard negatives; companion cross-encoder for reranking.
Early retrieval metrics looked excellent. An audit showed why: many paraphrased passages about the same facts sat across train/dev/test. Splits grouped exact documents but not semantic duplicates, so the model was rewarded for topic leakage, not true generalization.
That table stayed in the model card for transparency, but it was no longer treated as a release claim. I rebuilt the data into semantic/factual groups before splitting, retrained from the Swedish-adapted encoder (not from the contaminated checkpoint), and re-evaluated on the corrected hold-out.
Independent Swedish MTEB checks (SweFAQ / SweDN) then became the public honesty check: large gains over the adapted EuroBERT base, still clearly behind KBLab’s strong Swedish sentence-BERT baseline. Publishing both sides is intentional.
Results (summary)
Corrected internal eval and MTEB both show large lifts after retrieval tuning — the training objective transferred.
Still a substantial gap on SweFAQ and SweDN nDCG@10. That blocks any “best Swedish embedding” claim.
Documented CPU and GPU query latency / throughput on the embedding card — useful for on-prem sizing, not a service SLA.
diabase-reranker-1 is the precision stage after embedding recall — same product story as the models page pipeline.
Decisions
A European open encoder plus controlled Swedish adaptation was a faster path to a usable retrieval stack than a from-scratch foundation — with license clarity (Apache 2.0) and a clear ablation story.
Bi-encoder for recall, cross-encoder for precision. Matches how production RAG is usually operated and keeps the embedding model deployable on modest hardware.
Hiding the leakage episode would make the story prettier and less useful. Senior reviewers care that you can invalidate your own numbers.
Topic-safe splits as default, multi-positive labels, a human-reviewed Swedish document benchmark, stronger baseline comparisons, cross-lingual regression checks, and production latency budgets — listed on the model card, not waved away.
The Hugging Face cards are the detailed lab notebook. This page is the executive case study for people who will not open every training table.
