{"id":5870,"date":"2024-07-03T12:39:29","date_gmt":"2024-07-03T12:39:29","guid":{"rendered":"https:\/\/favtutor.com\/articles\/?p=5870"},"modified":"2024-07-03T12:40:01","modified_gmt":"2024-07-03T12:40:01","slug":"meta-3d-gen-generate-3d-assets","status":"publish","type":"post","link":"https:\/\/favtutor.com\/articles\/meta-3d-gen-generate-3d-assets\/","title":{"rendered":"Meta 3D Gen: This New AI Can Generate 3D Models in Seconds"},"content":{"rendered":"\n<p>Until now we have seen several sorts of Generative AI models, which can generate high-quality images and videos, and stunning audio in sound effects and music. Now we can get 3D Models generated in less than a minute, and all we have to do is just provide simple text prompts. Welcome Meta&#8217;s new AI: Meta 3D Gen!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Meta\u2019s Gen 3D Model<\/strong><\/h2>\n\n\n\n<p><strong>Meta 3D Gen (3DGen) model is a new text-to-3D asset generator that can create high-quality 3D assets in seconds.<\/strong><\/p>\n\n\n\n<div align=center><blockquote class=\"twitter-tweet\" data-media-max-width=\"560\"><p lang=\"en\" dir=\"ltr\">\ud83d\udce3 New research from GenAI at Meta, introducing Meta 3D Gen: A new system for end-to-end generation of 3D assets from text in &lt;1min.<br><br>Meta 3D Gen is a new combined AI system that can generate high-quality 3D assets, with both high-resolution textures and material maps end-to-end,\u2026 <a href=\"https:\/\/t.co\/rDD5GzNinY\" target=\"_blank\">pic.twitter.com\/rDD5GzNinY<\/a><\/p>&mdash; AI at Meta (@AIatMeta) <a href=\"https:\/\/twitter.com\/AIatMeta\/status\/1808157832497488201?ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener\">July 2, 2024<\/a><\/blockquote> <script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/div>\n\n\n\n<p><strong>It is compatible with physically-based rendering (PBR), which is required for real-world 3D asset relighting. <\/strong>Furthermore, 3DGen allows for the <a href=\"https:\/\/ai.meta.com\/research\/publications\/meta-3d-gen\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">generative retexturing<\/a> of previously generated (or artistically constructed) 3D shapes by utilizing user-supplied extra-textual inputs.<\/p>\n\n\n\n<p>This means not only this model will reflect your prompt as it is in the form of attractive 3D assets, but it will also enhance its texture and lighting to give you the perfect blend of a model you could have imagined.<\/p>\n\n\n\n<p>Not only this but there\u2019s one more catch here. After the object is created, it takes only 20 seconds to further modify and customize its texture, offering more quality at a significantly lower cost than other options. Without any changes, the same method may be used to texture 3D meshes made by artists.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Latest Text-to-3D Asset Technology<\/strong><\/h2>\n\n\n\n<p>It\u2019s quite interesting when you look at the blueprint, i.e. the powerful innovative technical approach behind this text to 3D generation model.<\/p>\n\n\n\n<p>3DGen, which builds upon AssetGen and TextureGen, efficiently integrates three highly complementary representations of the three-dimensional item: the volumetric space (three-dimensional shape and appearance), the UV space (texture), and the view spaces (pictures of the object).<\/p>\n\n\n\n<p>Two components are combined in Meta 3D Gen, a two-stage method: one for text-to-texture generation and the other for text-to-3D generation. Higher-quality 3D generation is the outcome of this integration for the production of immersive content. Let\u2019s look at the whole process through the stages in detail:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img decoding=\"async\" width=\"1025\" height=\"438\" src=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-826.png\" alt=\"\" class=\"wp-image-5871\" srcset=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-826.png 1025w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-826-768x328.png 768w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-826-750x320.png 750w\" sizes=\"(max-width: 1025px) 100vw, 1025px\" \/><\/figure>\n<\/div>\n\n\n<h3 class=\"wp-block-heading\"><strong>Stage 1) 3D Asset Generation<\/strong><\/h3>\n\n\n\n<p>In Stage 1, a 3D asset is created using the Meta 3D AssetGen model in response to a text prompt supplied by the user. In this step, a 3D mesh with texture and PBR material maps is generated. It takes about 30 seconds to make an inference.<\/p>\n\n\n\n<p>Using a multi-view and multi-channel variant of a text-to-image generator, AssetGen generates several fairly consistent views of the object to start this process.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Stage 2) Generative 3D texture refinement<\/strong><\/h3>\n\n\n\n<p>Stage 2 creates a better-quality texture and PBR maps for a 3D asset created in Stage 1 and the original text prompt used for generation. It makes use of the&nbsp;Meta 3D TextureGen text-to-texture generator.<\/p>\n\n\n\n<p>To elaborate, an initial version of the 3D object in volumetric space is extracted by an AssetGen reconstruction network. Mesh extraction is the next step, which determines the object&#8217;s 3D shape and initial texture.<\/p>\n\n\n\n<p>This step is important for adding more depth and texture quality to the 3D material shapes initially generated.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Step 3) Generative 3D Retexturing<\/strong><\/h3>\n\n\n\n<p>Ultimately, the TextureGen component regenerates the texture by combining UV-space and view-space generation to increase the material&#8217;s resolution and quality while maintaining the original prompt details.<\/p>\n\n\n\n<p>Stage 2 can also be used to produce a texture for this 3D asset from scratch, given an untextured 3D model and a prompt specifying its desired appearance (the mesh can be already developed or made by an artist). <\/p>\n\n\n\n<p>The inference takes about 20 seconds. All things considered, every phase of 3DGen expands upon Meta&#8217;s robust text-to-image models. Better textures are produced by fine-tuning the assets&nbsp;using synthetic 3D data rendering from an internal dataset&nbsp;to accomplish multi-view creation in both view space and UV space.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How Efficient is the Model?<\/strong><\/h2>\n\n\n\n<p><strong>The results obtained after testing Meta 3D Gen on various evaluations showed that it\u2019s quite efficient when it comes to producing 3D assets while maintaining the integrity of the user prompt and enhancing the texture quality of the volumetric mesh shapes.<\/strong><\/p>\n\n\n\n<p>In both phases, 3DGen surpasses all industry baselines in this parameter, with third-party text-to-3D (T23D) generators emerging as the most formidable rival.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img decoding=\"async\" width=\"1282\" height=\"260\" src=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-827.png\" alt=\"\" class=\"wp-image-5872\" srcset=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-827.png 1282w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-827-768x156.png 768w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-827-750x152.png 750w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-827-1140x231.png 1140w\" sizes=\"(max-width: 1282px) 100vw, 1282px\" \/><\/figure>\n<\/div>\n\n\n<p>It was found that annotators with less 3D experience are insensitive to the presence of even minor texture and geometry artifacts and prefer assets with sharper, more vibrant, realistic, and detailed textures. Across all categories, professional 3D artists indicated a greater preference for 3DGen generations.<\/p>\n\n\n\n<p>As a function of the scene complexity as indicated by the text prompt, the researchers additionally examine performance rates for visual quality, geometry, texture details, and the existence of texture artifacts. <\/p>\n\n\n\n<p>Plots demonstrate that, although certain baselines match up well for basic prompts, 3DGen begins to significantly outperform baselines as prompt complexity rises, moving from objects to characters and their compositions.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img decoding=\"async\" width=\"1293\" height=\"507\" src=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-828.png\" alt=\"\" class=\"wp-image-5873\" srcset=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-828.png 1293w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-828-768x301.png 768w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-828-750x294.png 750w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-828-1140x447.png 1140w\" sizes=\"(max-width: 1293px) 100vw, 1293px\" \/><\/figure>\n<\/div>\n\n\n<p><strong>They compare the 3DGen win rate to baselines and show the 50% threshold (dashed line) at which their approach outperforms the baselines.<\/strong><\/p>\n\n\n\n<p>Lastly, they also performed visual comparisons of Stage 1 and Stage 2. Stage 2&nbsp;has&nbsp;a propensity for greater visual aesthetics, realism, and higher frequency details. Stage 2 generations were preferred in 68% of the cases across different objects and compositions.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img decoding=\"async\" width=\"1291\" height=\"499\" src=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-829.png\" alt=\"\" class=\"wp-image-5874\" srcset=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-829.png 1291w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-829-768x297.png 768w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-829-750x290.png 750w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/07\/Screenshot-829-1140x441.png 1140w\" sizes=\"(max-width: 1291px) 100vw, 1291px\" \/><\/figure>\n<\/div>\n\n\n<p>Just a few months ago, Stability AI had a similar development in the same space with their <a href=\"https:\/\/favtutor.com\/articles\/triposr-stability-ai-image-to-3d-objects\/\">TripoSR to generate 3D objects<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Conclusion<\/strong><\/h2>\n\n\n\n<p>Meta 3D Gen lays a brand new foundation for Generative AI models in generating captivating 3D assets. Its innovative technology takes a step ahead the baseline models in not only just generating your desired shapes and meshes but also enhancing them across all compositions and texture qualities.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Learn more about Meta&#8217;s new 3D Gen AI which can create 3D assets in seconds. Learn more about how it works and how efficient it is.<\/p>\n","protected":false},"author":15,"featured_media":5883,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jnews-multi-image_gallery":[],"jnews_single_post":null,"jnews_primary_category":{"id":"","hide":""},"footnotes":""},"categories":[57],"tags":[56,81,310],"class_list":["post-5870","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","tag-ai","tag-meta","tag-meta-3d-gen"],"_links":{"self":[{"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/posts\/5870","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/users\/15"}],"replies":[{"embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/comments?post=5870"}],"version-history":[{"count":3,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/posts\/5870\/revisions"}],"predecessor-version":[{"id":5885,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/posts\/5870\/revisions\/5885"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/media\/5883"}],"wp:attachment":[{"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/media?parent=5870"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/categories?post=5870"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/tags?post=5870"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}