use-benchmark-67-cpu-1318-gpu

IN premise — summaries/2026/08/24/reimers-2019-sentence-bert-s8-conclusion.md

Created 2026-08-25T02:58:30+00:00

Universal Sentence Encoder achieves 67 sentences/second on CPU and 1318 sentences/second on GPU in the SBERT paper's benchmark.

Summary

The Universal Sentence Encoder can process roughly 67 sentences per second on a CPU but about 1,300 per second on a GPU, a roughly 20x speedup. This matters because it tells us that embedding large batches of text is highly GPU-dependent, so any system relying on this encoder needs to budget compute and plan for GPU availability to meet throughput targets.