{"id":3222,"date":"2024-04-04T07:50:17","date_gmt":"2024-04-04T07:50:17","guid":{"rendered":"https:\/\/favtutor.com\/articles\/?p=3222"},"modified":"2024-04-04T07:50:18","modified_gmt":"2024-04-04T07:50:18","slug":"swe-agent-devin-alternative","status":"publish","type":"post","link":"https:\/\/favtutor.com\/articles\/swe-agent-devin-alternative\/","title":{"rendered":"Open-Source AI SWE-Agent Takes on Devin (Better Alternative?)"},"content":{"rendered":"\n<p>Devin got some new competition from an open-source alternative called SWE-Agent. It is an open-source agent that can turn any GitHub issue into a pull request.<\/p>\n\n\n\n<p><strong>Highlights:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Researchers from Princeton NLP Group announced SWE-agent, an open-source AI software development system.<\/li>\n\n\n\n<li>It can turn language models like GPT-4 into software engineering agents that can fix bugs in real GitHub repositories.<\/li>\n\n\n\n<li>It achieves an accuracy of 12.29% on the SWE-bench benchmarks, very close to Devin AI\u2019s 13.86%.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>SWE-Agent Explained<\/strong><\/h2>\n\n\n\n<p><strong>The SWE-Agent (Software Engineering Agent) turns LMs into software engineering agents to fix bugs in GitHub repos. <\/strong><\/p>\n\n\n\n<p>It has demonstrated near-parity with <a href=\"https:\/\/favtutor.com\/articles\/devin-ai-software-engineer\/\">Devin\u2019s performance<\/a> on the SWE-bench Benchmark. This remarkable performance showcases the potential for revolutionizing software engineers&#8217; approach to addressing complex issues and streamlining their workflows.<\/p>\n\n\n\n<p>The video below shows how an SWE agent resolves an issue in a GitHub repository by finding out what is causing the issue:<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"jeg_video_container jeg_video_content\"><iframe title=\"A First Look at SWE-agent\" width=\"500\" height=\"375\" src=\"https:\/\/www.youtube.com\/embed\/CeMtJ4XObAM?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/div>\n<\/div><\/figure>\n\n\n\n<p>The agent takes an average of 93 seconds to complete any task. The system interacts with a specialized terminal that lets you open and search files, edit specific lines, and write and run tests.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>How to access SWE-agent?<\/strong><\/h3>\n\n\n\n<p>With SWE-agent being open-source, developers can leverage its capabilities by easily setting it up on their local machines. The setup instructions for local deployment are available on the <a href=\"https:\/\/github.com\/princeton-nlp\/SWE-agent\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">agent\u2019s official GitHub repository<\/a>. <\/p>\n\n\n\n<p>Developers can access the official demo on the <a href=\"https:\/\/swe-agent.com\/demo\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">official website<\/a>. This free accessibility empowers software engineers to seamlessly integrate the agent into their current workflows, unlocking the advantages of AI-assisted development without requiring extensive technical know-how.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Working of SWE-agent<\/strong><\/h3>\n\n\n\n<p>SWE-agent follows a systematic problem-solving strategy, which consists of planning, execution, observation, and iterative adjustment. This helps the agent to break down complex issues into simpler steps, ensuring efficient resolution of a problem.<\/p>\n\n\n\n<p>This is accomplished by creating straightforward LM-centric commands and feedback structures to simplify the LM&#8217;s navigation of the repository, thus enabling it to view, edit, and execute code files.<\/p>\n\n\n\n<p>This is called an Agent-Computer Interface (ACI) which facilitates communication between the agent and terminals. By enabling the agent to engage directly with the development environment, the interface reduces reliance on human involvement and accelerates the problem-solving process.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img decoding=\"async\" width=\"887\" height=\"356\" src=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image2.png\" alt=\"Working of SWE-Agent\" class=\"wp-image-3224\" srcset=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image2.png 887w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image2-300x120.png 300w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image2-768x308.png 768w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image2-750x301.png 750w\" sizes=\"(max-width: 887px) 100vw, 887px\" \/><\/figure>\n<\/div>\n\n\n<p>SWE-agent contains features that the team discovered to be immensely helpful during the ACI design process:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>They added a linter that runs when an edit command is issued and does not let the edit command go through if the code isn&#8217;t syntactically correct.<\/li>\n\n\n\n<li>They provided the agent with a custom file viewer rather than solely utilizing the \u2018cat\u2019 command for file display. It was observed that this file viewer functions optimally when presenting a maximum of 100 lines per iteration. Additionally, the developed file editor includes functionalities such as scrolling and search commands within the file.<\/li>\n\n\n\n<li>The agent was supplied with a specially designed full-directory string searching command. It was important for this tool to concisely list the matches, presenting each file containing a minimum of one match. Providing the model with more context about each match proved to be overly confusing for the model.<\/li>\n\n\n\n<li>When commands had an empty output, they returned a message saying &#8220;Your command ran successfully and did not produce any output.&#8221;<\/li>\n<\/ol>\n\n\n\n<p>The image demonstrates the agent&#8217;s thought process to fix any issues that occur in a repository:<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img decoding=\"async\" width=\"442\" height=\"628\" src=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image4-1.png\" alt=\"SWE agent's thought process to fix any issues in a repository\" class=\"wp-image-3225\" srcset=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image4-1.png 442w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image4-1-211x300.png 211w\" sizes=\"(max-width: 442px) 100vw, 442px\" \/><\/figure>\n<\/div>\n\n\n<h3 class=\"wp-block-heading\"><strong>How does it compete with Devin?<\/strong><\/h3>\n\n\n\n<p><strong>SWE-agent achieves similar accuracy to Devin AI on the SWE-bench benchmark, solving 12.29% of problems autonomously, compared to Devin\u2019s 13.86%. <\/strong><\/p>\n\n\n\n<p>However, it is important to remember that Devin was trained on only 25% of the SWE Benchmark. The agent takes, on average, 93 seconds to complete a task as opposed to 5 minutes by Devin. <\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img decoding=\"async\" width=\"791\" height=\"616\" src=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image1-1.png\" alt=\"SWE-bench test performance for SWE-Agent vs Devin\" class=\"wp-image-3226\" srcset=\"https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image1-1.png 791w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image1-1-300x234.png 300w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image1-1-768x598.png 768w, https:\/\/favtutor.com\/articles\/wp-content\/uploads\/2024\/04\/image1-1-750x584.png 750w\" sizes=\"(max-width: 791px) 100vw, 791px\" \/><\/figure>\n<\/div>\n\n\n<p>Also, its open-source design lets developers access and contribute to it whenever needed. However, this is not the case with Devin which has not been officially released yet. This encourages developers to customize and expand their functionalities to tackle various software engineering hurdles. Still, there are many key things <a href=\"https:\/\/favtutor.com\/articles\/devin-ai-early-insights\/\">we found out about Devin AI<\/a> to know about.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Conclusion<\/strong><\/h2>\n\n\n\n<p>The potential influence of the SWE-Agent extends beyond simply improving GitHub issue management efficiency. Through leveraging the collective expertise of the developer community, the SWE-Agent could evolve into a tool capable of revolutionizing the software development and maintenance processes.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The open-source AI software agent called SWE-agent is challenging Devin AI on benchmarks. Also, how to access SWE-Agent?<\/p>\n","protected":false},"author":18,"featured_media":3252,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jnews-multi-image_gallery":[],"jnews_single_post":null,"jnews_primary_category":{"id":"","hide":""},"footnotes":""},"categories":[57],"tags":[56,102,123,80,146],"class_list":["post-3222","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","tag-ai","tag-devin","tag-github","tag-gpt-4","tag-swe-agent"],"_links":{"self":[{"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/posts\/3222","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/users\/18"}],"replies":[{"embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/comments?post=3222"}],"version-history":[{"count":4,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/posts\/3222\/revisions"}],"predecessor-version":[{"id":3248,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/posts\/3222\/revisions\/3248"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/media\/3252"}],"wp:attachment":[{"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/media?parent=3222"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/categories?post=3222"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/favtutor.com\/articles\/wp-json\/wp\/v2\/tags?post=3222"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}