Accubits Technologies, in collaboration with Bud Ecosystem, has open-sourced their fifth Large Language Model (LLM) – GenZ 70B, an instruction-fine-tuned model offering a commercial license. The model is primarily fine-tuned for enhanced reasoning, roleplaying, and writing capabilities. In our initial evaluations, the model exhibited outstanding results. The model scored 70.32 on the MMLU benchmark, which is higher than LLaMA2 70B’s score of 69.83. Moreover, GenZ 70B’s exceptional score of 7.34 on the MT bench benchmark is the SOTA result for open-source LLMs with commercial licenses.
GenZ 70B is an auto-regressive language model with an optimized transformer architecture. We fine-tuned the model with curated datasets using the Supervised Fine-Tuning (SFT) method. The mode is fine-tuned with OpenAssistant’s instruction fine-tuning dataset and Thought Source for the Chain Of Thought (CoT) approach. With extensive fine-tuning, the model has acquired additional skills and capabilities beyond what a pretrained model can offer.
| Model Name | MT Bench | MMLU | Human Eval | Hellaswag | BBH |
|---|---|---|---|---|---|
| Genz 13B v1 | 6.12 | 53.62 | 17.68 | 77.38 | 37.76 |
| Genz 13B v2 | 6.79 | 53.68 | 21.95 | 77.48 | 38.1 |
| Genz 70B | 7.34 | 70.32 | 37.8 | - | 54.69 |
We are committed to continuously enhancing the capabilities of GenZ by regularly fine-tuning our models with various curated datasets to make them even better. Our goal is to reach the state of the art and beyond – and we’re committed to staying the course until we get there.
In our journey of innovation, while our previously released smaller models of GenZ displayed commendable performance, we realized that complex business applications might require larger models to guarantee excellence. This realization propelled us to launch our fine-tuned model with 70 billion parameters. Such a vast parameter set improves the model’s reasoning capabilities and considerably enhances its knowledge base, rendering it apt for advanced business applications. By open-sourcing GenZ 70B, we are extending an invitation to developers, organizations, and businesses worldwide to experiment, innovate, and construct groundbreaking solutions, harnessing the formidable capabilities of this model.
Intended Use
When we created GenZ 70B, we had a clear vision of how it could be used to push the boundaries of what’s possible with large language models. We also understand the importance of using such models responsibly. Here’s a brief overview of the intended and out-of-scope uses for GenZ 70B.
Direct Use:
GenZ 13B is designed to GenZ 70B is designed to be a powerful tool for research on large language models. It’s also an excellent foundation for further specialization and fine-tuning for specific use cases, such as Text summarization, Text generation, Chatbot creation, And much more!
Out-of-Scope Use
The GenZ 70B, though a versatile tool, has limitations in its applicability. Users are cautioned against deploying it for production without thoroughly assessing potential risks and implementing mitigation strategies. It should not be utilized for purposes that could be deemed irresponsible or harmful, nor should it be employed in a way that breaches any prevailing laws or trade compliance regulations. Additionally, adhering to the Acceptable Use Policy and Licensing Agreement for Llama 2 is paramount. It’s imperative to note that GenZ 70B, as with many large language models, is rooted in extensive web corpora and might thus reflect online biases and stereotypes.
Recommendations
We recommend users of GenZ 70B consider fine-tuning it for the specific set of tasks of interest. Appropriate precautions and guardrails should be taken for any production use. Using GenZ 70B responsibly is key to unlocking its full potential while maintaining a safe and respectful environment.
