Page History

...

Test	Device	Score (FPS ↑)	Score (ms ↓)
Face Detection FP16	CPU	2.16	2735.36
Person Detection FP16	CPU	22.00	272.29
Person Detection FP32	CPU	21.84	274.21
Vehicle Detection FP16	CPU	144.39	41.47
Face Detection FP16-INT8	CPU	7.95	751.10
Face Detection Retail FP16	CPU	596.99	10.01
Road Segmentation ADAS FP16	CPU	64.82	92.45
Vehicle Detection FP16-INT8	CPU	387.23	15.44
Weld Porosity Detection FP16	CPU	225.04	70.89
Face Detection Retail FP16-INT8	CPU	1321.78	11.93
Road Segmentation ADAS FP16-INT8	CPU	165.55	36.15
Machine Translation EN To DE FP16	CPU	27.37	218.78
Weld Porosity Detection FP16-INT8	CPU	837.45	18.99
Person Vehicle Bike Detection FP16	CPU	310.99	19.23
Noise Suppression Poconet-Like FP16	CPU	419.29	37.90
Handwritten English Recognition FP16	CPU	125.88	126.94
Person Re-Identification Retail FP16	CPU	320.22	18.67
Age Gender Recognition Retail 0013 FP16	CPU	6972.63	2.24
Handwritten English Recognition FP16-INT8	CPU	143.45	111.37
Age Gender Recognition Retail 0013 FP16-INT	CPU	16385.56	0.92

llama-cpp-2.1.1

TPS - Tokens Per Second

Backend	Model	Test		Text Generation 128, TPS	Prompt Processing 512, TPS	Prompt Processing 1024, TPSScore
CPU BLAS	Llama-3.1-Tulu-3-8B-Q8_0	Text Generation 128		6.47	22.27	21.766.45
CPU BLAS	LlamaMistral-3.17B-TuluInstruct-v0.3-8B-Q8_0	Prompt Processing 512	6.81	22.22	21.8122.73
CPU BLAS	Llamagranite-3.10-Tulu3b-3a800m-8Binstruct-Q8_0	Prompt Processing 1024	22.59	25.36	97.04	89.25	CPU BLAS	Mistral-7B-Instruct-v0.3-Q8_0

FPS - Tokens Per Second

FPS - Tokens Per Second

Test	Score, FPS
AV1 Video Encoding	122

...