Accuracy scores on LVBench.
# | Model | Throughput | LLM Params |
Date | Overall (%) | ER (%) | EU (%) | KIR (%) | TG (%) | Rea (%) | Sum (%) |
---|---|---|---|---|---|---|---|---|---|---|---|
1 | Gemini 1.5 Pro | 3600 | - | 2024-06-11 | 33.1 | 32.1 | 30.9 | 39.3 | 31.8 | 27 | 32.8 |
2 | LLaVA-NeXT-Video-DPO (34B) | 32 | 34B | 2024-06-11 | 32.2 | 30.1 | 31.2 | 34.1 | 31.4 | 35 | 27.6 |
3 | GPT-4o | 10 | - | 2024-06-11 | 27 | 26.5 | 23.7 | 28.3 | 21.4 | 28 | 32.8 |
4 | PLLaVA 34B | 16 | 34B | 2024-06-11 | 26.1 | 25.0 | 24.9 | 26.2 | 21.4 | 30.0 | 25.9 |
5 | LWM | >3600 | 7B | 2024-06-11 | 25.5 | 24.7 | 24.8 | 26.5 | 28.6 | 30.5 | 22.4 |
6 | LLaMA-VID | >10800 | 13B | 2024-06-11 | 23.9 | 25.4 | 21.7 | 23.4 | 26.4 | 26.5 | 17.2 |
7 | MovieChat | >10000 | 7B | 2024-06-11 | 22.5 | 21.3 | 23.1 | 25.9 | 22.3 | 24.0 | 17.2 |
8 | TimeChat | >96 | 7B | 2024-06-11 | 22.3 | 21.9 | 21.7 | 25.9 | 22.7 | 25.0 | 24.1 |