Nearly two weeks after the launch of Elon Musk's xAI open AI model behind Grok for the public, its AI chatbot is set to get an upgrade.
company announced Grok-1.5 on Thursday and claimed that its latest model can understand longer documents, handle more complex queries and perform more advanced reasoning.
While Grok-1.5 appears to be a step up from the original 1.0 version with improvements in coding and math capabilities, its announcement post shows that it still lags behind Google's Gemini Pro 1.5 AI, OpenAI's GPT-4, and Claude 3 Opus of Anthropic in several landmarks. tests, outperforming OpenAI in a key HumanEval test.
Connected: Meet Grok: Elon Musk unveils 'spicy' AI Chatbot filled with 'sarcasm' and 'humour'
Grok-1.5 scored more than GPT-4 in HumanEval benchmark, which consists of 164 challenging programming problems that are not included in the AI model training data. The GPT-4 had a score of 67% and the Gemini Pro 1.5 scored 71.9%, while the Grok-1.5 got 74.1%.
Elon Musk's company xAI is set to release a new version of the Grok AI chatbot, a competitor to ChatGPT. Photo by Jaap Arriens/NurPhoto via Getty Images.
With a result of 81.3% in MMLU test, which covers knowledge of 57 subjects from elementary to advanced level, Grok-1.5 performed close to Google Gemini's score (83.7%).
It also scored close to the GPT-4 score of 52.9% with a score of 50.6% in Mathematics test, a benchmark that covers math competition problems from middle school to high school.
Musk announced on Friday post on social networks that Grok 1.5 should be available on X, formerly Twitter, within the next week.
Owner X has high expectations for the next-generation Grok, writing that the next step after the Grok-1.5 will be better than the currently available AI “in all metrics.” Grok 2 is “in training now”, he wrote in the post.
Grok AI is currently only available those with a $16 per month or higher Premium+ subscription on X.
Musk ignorant OpenAI, a competitor of xAI, earlier this month and sought a court order that would force OpenAI to make public the research and technology behind the AI.