- \n
- xAI revealed Grok 1.5V, its first multimodal model capable of immense visual processing.<\/li>\n\n\n\n
- A standout feature is its ability to convert logical diagrams into executable code.<\/li>\n\n\n\n
- Grok 1.5V outperforms its peers in the new RealWorldQA benchmark that measures real-world spatial understanding.<\/li>\n<\/ul>\n\n\n\n
The Capabilities of Grok 1.5V<\/strong><\/h2>\n\n\n\n
Grok-1.5V is their first-generation multimodal model that can now process visual information as well.<\/strong> It will be available existing Grok users soon.<\/p>\n\n\n\n
This release comes mere weeks after xAI released their updated chatbot model Grok 1.5 featuring enhanced reasoning capabilities and a 128,000 tokens context. The model showed advanced coding and math capabilities and was praised for its minimal censoring and ability to give answers to controversial questions.<\/p>\n\n\n\n
Don’t forget that Grok 1 is also open-source<\/a> now!<\/p>\n\n\n\n
Grok 1.5V outperforms GPTt-4 in text-reading, mathematics, and real-world question-answer. The real-world QA capabilities might just be the most impressive feature Grok has displayed yet.<\/p>\n\n\n\n
The company described 7 vision-based use cases for Grok in their blog from building Python code from a flowchart to counting calories based on the nutritional information on the packets. Here is an overview of the possible uses of this model in different domains.<\/p>\n\n\n\n
Let\u2019s take a look at all the use cases!<\/p>\n\n\n\n
1) Writing Code from Diagrams<\/strong><\/h3>\n\n\n\n
This development is so impactful because it enables any person with a strong logical base to become a programmer without detailed knowledge of any language! <\/p>\n\n\n
\n![\"Code](\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image4-5-1024x518.png\")
<\/figure>\n<\/div>\n\n\nIt empowers people to start building immediately without having to learn the finer intricacies of code.<\/p>\n\n\n\n
2) Calculation and Real-world understanding<\/strong><\/h3>\n\n\n\n
Far too many times AI models make errors in calculation even when all information is provided to them in text form. Here the model extracts information from the image and performs accurate mathematical calculations:<\/p>\n\n\n
\n![\"](\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image3-4-1024x516.png\")
<\/figure>\n<\/div>\n\n\nWe wonder if it will be able to analyze further more complicated mathematics like percentages and taxes. If so, imagine how efficient doing your taxes will be, simply by uploading records and asking the model to evaluate it!<\/p>\n\n\n\n
3) Understanding Images<\/strong><\/h3>\n\n\n\n
The next three examples show the depth of image understanding shown by Grok. From a simple child’s drawing, it inferred the elements in the drawing and built a story around those elements:<\/p>\n\n\n
\n![\"Understaing](\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image9-1-1024x466.png\")
<\/figure>\n<\/div>\n\n\nIt even explained a meme in a pop culture context and analyzed a defective wood image to diagnose the problem:<\/p>\n\n\n
\n![\"meme](\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image5-3-1024x302.png\")
<\/figure>\n<\/div>\n\n\nThe possible use cases of this particular application in industries like healthcare after fine-tuning are endless.<\/p>\n\n\n\n
4) Data extraction and problem-solving<\/strong><\/h3>\n\n\n\n
Grok also shows great ability to extract data from images and convert it to the required format as a CSV file or as a dataframe. It can give detailed solutions to competitive coding problems including the test cases:<\/p>\n\n\n
\n![\"Grok](\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image2-2-1024x351.png\")
<\/figure>\n<\/div>\n\n\nThe possibility that this ability will be misused in competitive coding competitions is very high, which seems a problem that platforms like leetcode will have to solve as soon as possible:<\/p>\n\n\n
\n![\"Solving](\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image1-6-1024x530.png\")
<\/figure>\n<\/div>\n\n\nGrok 1.5 V\u2019s standout ability is its real-world understanding, as they explained in their official announcement<\/a>:<\/p>\n\n\n\n
\u201cIn order to develop useful real-world AI assistants, it is crucial to advance a model’s understanding of the physical world. Toward this goal, we are introducing a new benchmark, RealWorldQA. This benchmark is designed to evaluate basic real-world spatial understanding capabilities of multimodal models. While many of the examples in the current benchmark are relatively easy for humans, they often pose a challenge for frontier models.\u201d<\/p>\n\n\n\n
Conclusion<\/strong><\/h2>\n\n\n\n
The developers of Grok 1.5 vision anticipate significant growth in multimodal capabilities across all data types like audio, videos along with documents and images. The real-world understanding is a step in the right direction to achieve AGI<\/a> eventually.<\/p>\n","protected":false},"excerpt":{"rendered":"
Grok 1.5 Vision preview unveils better visual processing, real-world understanding, and coding use cases with its first multimodal model.<\/p>\n","protected":false},"author":20,"featured_media":3703,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jnews-multi-image_gallery":[],"jnews_single_post":null,"jnews_primary_category":{"id":"","hide":""},"footnotes":""},"categories":[57],"tags":[56,101,100,72,114],"class_list":["post-3694","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","tag-ai","tag-elon-musk","tag-grok","tag-llm","tag-xai"],"_links":{"self":[{"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/posts\/3694","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/users\/20"}],"replies":[{"embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/comments?post=3694"}],"version-history":[{"count":2,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/posts\/3694\/revisions"}],"predecessor-version":[{"id":3704,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/posts\/3694\/revisions\/3704"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/media\/3703"}],"wp:attachment":[{"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/media?parent=3694"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/categories?post=3694"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/tags?post=3694"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}