Llama 3.1 405B vs GPT-4o: Which model is better?

Evaluating Llama 3.1–405B and GPT-4o Across Key Performance Metrics to Determine the Superior AI Model for Users and Developers.

For the open-source version of this article, please visit this link.

Author

• Zijian Yang (ORCID: 0009–0006–8301–7634)

Introduction

To compare the strengths and intelligence of Llama-3.1–405B and GPT-4o from a practical usage perspective, this article designed test cases covering five scenarios: mathematics, coding, tool usage, JSON extraction, and creative writing. We conducted a comparison test between the strongest open-source and closed-source models.

Due to resource limitations, we tested Llama-3.1–405B-Instruct and GPT-4o-2024–08–06 on publicly available platforms lmsys.org and huggingface.co/chat/.

Introduction to the Llama 3.1 Model

On July 24, Meta officially open-sourced Llama 3.1, which includes versions with 8B, 70B, and 405B parameters. The specific information for the three models is detailed in the table below. In addition to the BF16 precision, the 405B model also has an FP8 quantized version. An extra content safety…