Saturday, 12 April, 2025

Meta’s Maverick AI Model Falls Short on Benchmark After Using Optimized Variant

Meta's Llama 4 Maverick AI model has come under scrutiny after its "vanilla" version ranked significantly lower on the LM Arena benchmark compared to competitors like OpenAI's GPT-4o and Google's Gemini 1.5 Pro. Earlier, Meta had submitted an "experimental" version optimized for conversational tasks, which achieved a high ranking. Following criticism, LM Arena updated its policies to ensure fair evaluations.