Open-source Large Language Models Leaderboard

Open-source Large
Language Models Leaderboard

Large Language Models (LLMs) have revolutionized natural language processing and have shown impressive results in various language tasks.
The Problem: Several LLMs are available in the market. But, relevant information about these models is scattered on the internet, and it is extremely difficult to evaluate these models.
The Solution: We created this leaderboard to help researchers easily identify the best open-source LLM with an intuitive leadership quadrant graph. We evaluate the performance of open-source LLMs to rank them based on their capabilities and market adoption.View Models

Leaders

LLAMA2, LLAMA1, and T5, based on our scoring methodology, these models scored 89, 87, and 81 points, respectively. The scoring methodology is explained below. The current leader LLAMA2 is a collection of pretrained and fine-tuned large language models (LLMs) that range in scale from 7 billion to 70 billion parameters. The fine-tuned LLMs, or Llama 2-Chat, are specifically optimized for dialogue applications. These models surpass the performance of most open-source chat models on the benchmarks they were tested on. Llama2 70B model outperforms all open-source models.

Leaderboard

RankModelSizeArchitectureOrganizationAdoption Rating
Calculated based on the number of forks and stars on the official model repo.
Capability Rating
Calculated based on the number of tasks and downstream tasks of the model.
Score
A weighted average of the adoption and capability score of the model.
#1LLaMA270BTransformer, AutoregressiveMeta AI869289
#2LLaMA65BTransformer, AutoregressiveMeta AI858987
#3T511BTransformerGoogle748881
#4Galactica120BTransformerMeta AI475853
#5LongT511BTransformerGoogle376250
#6GLM130BTransformer, AutoregressiveTsinghua University485049
#7OPT175BTransformerMeta AI494145
#8BERT340MTransformerGoogle572742
#9UL220BTransformerGoogle521936
#10GPT-NeoX20BTransformer, AutoregressiveEleutherAI423338
11Switch1.6TTransformerGoogle271823
#12H32.7BState Space ModelStanford University272124
#13FLAN-T511BTransformerGoogle252425
#14Pythia12BDecoder-only autoregressiveEleuther.ai212121

Need help with Generative AI?

If you have any questions or need a helping hand, don't hesitate to reach out.

Let’s Get Started

The first step towards greatness begins now, let's embark on this journey.

Help us Help you.

Share more information with us, and we'll send relevant information that cater to your unique needs.

Final Touch

Kindly share some details about your company to help us identify the best-suited person to contact you.

Contact Details

Next

Project Details

Next

Company Information

Submit

Ranking Methodology

We only considered prominent and open-source LLMs to create this leaderboard. Note that this leaderboard can only be considered a high-level indicator of overall performance. Depending on the specific use case and business requirements, a detailed analysis is required to choose the right model. The key parameters we used for the scoring are;

  1. Benchmark results
  2. Model forks

Capability Rating(CR) is calculated based on a weighted sum of benchmark results(BR) published in the Model's research paper.

Rank weights;
For performance ranks #1 to #5, rank weight = 3.

For performance ranks #6 to #10, rank weight = 2.

For performance ranks #11 to #20, rank weight = 1.

Adoption Rating (AR) is calculated based on Model forks (MF) and penalizing that value against model performance. To calculate the adoption rating, we calculate the sum of the normalized value of Forks and Capability score.  Then normalize the resulting value to 100. The Model score is simply the average of scores Adoption Rating and Capability Rating. 

Generative AI Adoption Framework

This whitepaper will explore generative AI and identify business growth opportunities it offers. We aim to provide business owners with a comprehensive guide to using AI to unlock new opportunities and achieve sustainable growth. We will explore how generative AI can be used to analyze data and identify patterns, as well as how it can be used to generate new ideas and solutions.

Free Download

Frequently Asked Questions

Here are some of the most common questions we get asked. If you have a question that isn't on this list, please don't hesitate to contact us. We're always happy to help! We'll get back to you within 24 hours.