Grok-1 is Now Open-Source, Here’s How You Can Set It Up

Grok-1 LLM by Elon Musk’s xAI is now open-source, a new rival to OpenAI’s ChatGPT. So, what’s new comes with this release and how can developers get started to set up Grok-1? Let’s get right into it!

Grok-1 Open-Source Update

On March 17, 2024, Elon Musk and xAI openly released Grok-1, an enormous 314B parameter language model. They released the model’s weights and architecture for the public under an open-source Apache 2.0 license.

The model weights and architecture are now available on GitHub. Developers can set it up and install the model locally. Note that the Device needs multiple GPUs to run the 314B parameter Grok-1 model.

Here’s what they shared in their official statement:

“We are releasing the base model weights and network architecture of Grok-1, our large language model. Grok-1 is a 314 billion parameter Mixture-of-Experts model trained from scratch by xAI.”

By base model, it means that this is the same model that was announced in October 2023. The model hasn’t been fine-tuned for any specific application-based purposes such as Dialogues.

How to Set Up Grok-1?

To enable more efficient calculation, Grok-1 is a Mixture-of-Experts model that only activates 25% of its weights for any given input token.

xAI has openly released Grok-1’s model weights and architecture on a GitHub repository which developers can access. You can set up the model by following the instructions mentioned in the repository. To publish the model, you can also refer to HuggingFace, where xAI has provided detailed instructions to do so.

Here we have provided the detailed steps that you can follow to set up the model. First, you need to clone the GitHub repository which contains the model weights. To clone the repo perform the following steps:

• Open Git Bash
• To create a cloned directory, move the current working directory to the desired location.
• Type the following code

git clone https://github.com/xai-org/grok-1.git && cd grok-1

• Press enter to create your local clone.

You have successfully set up the GitHub repository clone. The next step is to download the int8 checkpoint to the checkpoints directory by executing this command in the repo root directory:

huggingface-cli download xai-org/grok-1 --repo-type model --include ckpt-0/* --local-dir checkpoints --local-dir-use-symlinks False

Now you are good to run the final snippet. Run the code below and you should be good to go:

pip install -r requirements.txt
python run.py

Congratulations! You have successfully set up Grok-1 on your device. On a test input, the script loads the model’s samples and checkpoint.

What are the Requirements?

To test the model using the example code, a system with sufficient GPU memory is needed due to the model’s massive size (314B parameters).

NVIDIA’s Ampere Tesla A40 is a good GPU to run Grok-1 as it comes with a video memory of 48 GB.

You can also download its all-model weights by using a torrent client and this magnet link:

magnet:?xt=urn:btih:5f96d43576e3d386c9ba65b883210a393b68210e&tr=https%3A%2F%2Facademictorrents.com%2Fannounce.php&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce

Future Enhancements

According to xAI, there is no specific application for this model, such as having conversations with it. Grok-1 was trained on a “custom” stack, the business stated, although it did not provide further information. The Apache Licence 2.0, which allows for commercial use cases, governs the model’s licensing.

Nonetheless, a few AI-driven tool developers have already discussed integrating this new AI into their products. The CEO of Perplexity, Aravind Srinivas, announced on X that Grok will be enhanced for conversational search and made accessible to Pro users.

Some of the AI models from many well-known firms, such as Meta’s LLaMa, Mistral, Falcon, and AI2, are available to the public. Google also unveiled two brand-new open models in February: Gemma2B and Gemma7B.

Conclusion

We finally have Grok-1 on our hands. Even though it’s still a base model of October 2023 and it doesn’t lead the race for benchmarks, it is still an advancement, considering its ability to be answerable to more harmless questions. This tool also continues Elon Musk’s challenge towards OpenAI -4 in developing something beneficial to society!