Google’s new small AI model Gemma 2 2B surpasses OpenAI’s GPT-3.5

RZ Nitin Star Quester - 2025

2024-08-02 11:55

Hey Questers ❤️ 




The new model has recorded MMLU and MBPP scores 10 per cent higher than its predecessor.


Google has just unveiled its latest AI model, Gemma 2 2B a lightweight AI model that has reportedly outperformed larger models like GPT-3.5 and Mistral 8x7B on key benchmarks. Weeks after it released the Gemma 2 best-in-class models, Google has unveiled Gemma 2 2B with built-in safety advancements. Along with Gemma 2 2B, Google has also announced new additions ShieldGemma and Gemma Scope. 

Figure 1, view larger image


With these additions, researchers and developers can now create safer customer experiences, gain unprecedented insights into our models, and confidently deploy powerful AI responsibly, right on the device, unlocking new possibilities for innovation,” Google said in its official release.


What is Gemma 2 2B?

According to Google, Gemma 2 2B is a lightweight model that produces outsized results by learning from larger models through distillation. The company claims that the Gemma 2 2B surpasses all GPT-3.5 models on Chatbot Arena, showcasing exceptional conversational AI abilities. 


Google has said that the new model can efficiently run on a wide range of hardware, from edge devices and laptops to robust cloud deployments with Vertex AI and Google Kubernetes Engine (KGE). The model is also optimised with the NVIDIA TensorRT-LLM library to further enhance its speed. Gemma 2 2B seamlessly integrates with Keras, JAX, Hugging Face, NVIDIA NeMo, Ollama, Gemma.cpp and it will be available soon on the MediaPipe platform for streamlined development. The model is open and accessible. 

Figure 2, view larger image


How is it different?

Gemma 2 2B comes with just 2.6B parameters, however, it was trained on a massive 2 trillion token dataset. On Chatbot Arena, the model scored 1130 which matches the scores by GPT-3.5 Turbo and Mixtral – 8x7b which are significantly larger than Gemma 2 2B.


When it comes to MMLU (Massive Multitask Language Understanding) the model has secured 56.1 and a score of 36.6 on MBPP (Mostly Basic Python Problems). With these scores the new model has reportedly beat its predecessor by over 10 per cent. Since it is open-source, developers can download the model from Google’s announcement page. 





Follow for such more informative threads ➡️ @RZ Nitin

Figure 3, view larger image
Tech