After 2 Weeks of Testing, What Do Developers Really Think About Claude 3?

Highlights:

• Almost 2 weeks into Claude 3’s release, developers worldwide have explored a variety of its potential use cases.
• Comes with several functionalities starting from developing an entire multi-player app to even writing tweets that mimic your style.
• Can even perform search based and reasoning tasks from large documents and generate Midjourney prompts. We can expect much more in the days to come.

It’s been almost two weeks since Anthropic announced the world’s most powerful AI model, the Claude 3 family. Developers worldwide have tested it and explored its vast functionalities across diverse use cases.

Some have been really amazed by the performance capabilities and have put the chatbot on a pedestal, favoring it over ChatGPT and Gemini. Here in this article, we are going to explore the game-changing capabilities that come with Claude 3 and analyze them in-depth, stating how the developer community can benefit from it.

13 Game-Changing Features of Claude 3

1. Developing an entire Multi-player App

A user named Murat on X prompted Claude 3 Opus to develop a multiplayer drawing app that allows users to collaborate and see real-time strokes emerge on other people’s devices. The user also instructed Claude to implement an additional feature that allows users to select color and name. The user’s names should also be saved when they log in.

Not only did Claude 3 successfully develop the application but it also didn’t produce any bugs in the deployment. The most impressive aspect of this development was that it took Claude 3 only 2 minutes and 48 seconds to deploy the entire application.

Opus did an amazing job extracting and saving the database, index file, and Client- Side App. Another interesting aspect of this deployment was that Claude was constantly retrying to get API access while initially developing the application. In the video obtained from the user’s tweet, you can see how well the application has been developed, moreover, multi-user strokes are also reflected on the app interface.

This is truly the first time an AI-based Chatbot has perfectly encapsulated the development of a Multi-player Application.

2. Decoding IKEA Instructions

A user named Gabriel on X, gave quite an interesting prompt. He asked Claude 3 Opus, to generate instructions from a series of user manual images. The images were numbered in sequence.

The results from Claude 3 were great. It did an amazing job of explaining the entire manual instructions step by step. Surprisingly it even specifically mentioned the type of tools to be used along with their numbers! It perfectly analyzed all the images, even mentioned which images show the additional components, and also stated which image shows the final overall representation.

So now you can have IKEA instructions decoded from your user manual images and can be hassle-free about following the instructions step-by-step. Take a look at the images below obtained from the user’s tweet.

3. Writing a fuzzer for a GIF Generator

A user named Brendan on X gave Claude 3 Opus the entire source of a small C GIF decoding library that he found on GitHub. He asked Claude to write a Python function to generate random GIFs that could exercise the parser. Here’s the prompt that he gave:

“Given this C program, can you write a Python function that generates random gif files that fully exercise the parsing code? The function should have the signature:“

# Generates a random file into `out`
def generate_random_input(out: BinaryIO):

Claude perfectly generated the Python function where its GIF generator got 92% of line coverage in the decoder and found 4 memory safety bugs and one hang. It enabled the function to write random GIF data to the provided binary output stream.

It also continued to explain the code step by step stating how the GIF generator function operates. It elaborates on how it generates a Logical Screen Descriptor, a Global Color Table, and an Extension Introducer or Image Separator for adjusting the size, color, and sub-block of the GIF respectively. It also found 5 signed integer overflow issues with the code.

Here’s the Python function that Claude 3 Opus generated:

import random
import struct

def generate_random_input(out):
    # Generate Header
    out.write(b'GIF89a')

    # Generate Logical Screen Descriptor
    width = random.randint(1, 65535)
    height = random.randint(1, 65535)
    out.write(struct.pack('<HH', width, height))
    packed_field = random.randint(0, 255) | 0x80  # Set Global Color Table flag
    out.write(struct.pack('B', packed_field))
    out.write(struct.pack('B', random.randint(0, 255)))  # Background Color Index
    out.write(struct.pack('B', random.randint(0, 255)))  # Pixel Aspect Ratio

    # Generate Global Color Table
    gct_size = 1 << ((packed_field & 0x07) + 1)
    for _ in range(gct_size):
        r, g, b = random.randint(0, 255), random.randint(0, 255), random.randint(0, 255)
        out.write(struct.pack('BBB', r, g, b))

    # Generate Extensions and Image Blocks
    while True:
        block_type = random.choice([0x21, 0x2C, 0x3B])  # Extension Introducer, Image Separator, or Trailer
        out.write(struct.pack('B', block_type))

        if block_type == 0x21:  # Extension Introducer
            extension_label = random.choice([0x01, 0xF9, 0xFE, 0xFF])  # Plain Text, Graphic Control, Comment, or Application
            out.write(struct.pack('B', extension_label))

            if extension_label == 0x01:  # Plain Text Extension
                out.write(struct.pack('B', 12))  # Block Size
                out.write(struct.pack('<HHHH', random.randint(0, width), random.randint(0, height),
                                      random.randint(0, width), random.randint(0, height)))
                out.write(struct.pack('BB', random.randint(0, 255), random.randint(0, 255)))
                out.write(struct.pack('BB', random.randint(0, 255), random.randint(0, 255)))
                out.write(struct.pack('B', 0))  # Block Terminator

            elif extension_label == 0xF9:  # Graphic Control Extension
                out.write(struct.pack('B', 4))  # Block Size
                packed_gce_field = random.randint(0, 255)
                out.write(struct.pack('B', packed_gce_field))
                out.write(struct.pack('<H', random.randint(0, 65535)))  # Delay Time
                out.write(struct.pack('B', random.randint(0, 255)))  # Transparent Color Index
                out.write(struct.pack('B', 0))  # Block Terminator

            elif extension_label == 0xFE:  # Comment Extension
                out.write(struct.pack('B', random.randint(1, 255)))
                out.write(b'\x00')  # Block Terminator

            elif extension_label == 0xFF:  # Application Extension
                out.write(struct.pack('B', 11))  # Block Size
                out.write(b'NETSCAPE2.0')
                out.write(struct.pack('B', 3))  # Block Size
                out.write(b'\x01')
                out.write(struct.pack('<H', random.randint(0, 65535)))  # Loop Count
                out.write(b'\x00')  # Block Terminator

        elif block_type == 0x2C:  # Image Descriptor
            out.write(struct.pack('<HHHH', random.randint(0, width), random.randint(0, height),
                                  random.randint(1, width), random.randint(1, height)))
            packed_idesc_field = random.randint(0, 255)
            out.write(struct.pack('B', packed_idesc_field))

            if packed_idesc_field & 0x80:  # Local Color Table Flag
                lct_size = 1 << ((packed_idesc_field & 0x07) + 1)
                for _ in range(lct_size):
                    r, g, b = random.randint(0, 255), random.randint(0, 255), random.randint(0, 255)
                    out.write(struct.pack('BBB', r, g, b))

            # Image Data
            lzw_min_code_size = random.randint(2, 8)
            out.write(struct.pack('B', lzw_min_code_size))

            # Generate random sub-blocks
            while True:
                sub_block_size = random.randint(1, 255)
                out.write(struct.pack('B', sub_block_size))
                for _ in range(sub_block_size):
                    out.write(struct.pack('B', random.randint(0, 255)))
                if random.random() < 0.1:
                    break
            out.write(b'\x00')  # Block Terminator

        else:  # Trailer
            break

import sys
for f in sys.argv[1:]:
    with open(f,'wb') as of:
        generate_random_input(of)
    print(f)

You can visit the GitHub repository for more information about the fuzzer code.

This is really huge news for the developer community as Claude is taking coding and debugging to another level. Now it takes just several minutes to deploy Python functions which a few months before developers took several hours to fix and analyze.

4. Automated Prompt Engineering

A team of developers at LangChain AI devised a mechanism that teaches Claude 3 to prompt engineer itself. The mechanism workflow involves writing a prompt, running it on test cases, grading responses, letting Claude3 Opus use grades to improve the prompt, & repeat.

To make the entire workflow easier they used LangSmith, a unified DevOps platform from LangChain AI. They first created a dataset of all possible test cases for the prompts. An initial prompt was provided to Claude 3 Opus from the dataset. Next, they annotated example generations in the form of tweets and provided manual feedback based on the prompt quality and structure. This feedback was then passed to Claude 3 opus to re-write the prompt.

This whole process was repeated iteratively to improve prompt quality. Claude 3 executes the workflow perfectly, fine-tuning the prompts and getting better with every iteration. Here credit not only goes to Claude 3 for its mindblowing processing and iterating capabilities but also to LangChain AI for coming up with this technique.

Here’s the video taken from LangChain where they applied the approach of paper summarization on Twitter and asked Claude 3 to summarize papers in excellent communication styles with the main goal of prompt engineering in an iterative manner. Claude 3 adjusts its summary prompt based on feedback and generates more interesting document summaries.

5. Detection of Software Vulnerabilities and Security Threats

One of Claude 3’s most impressive features comes in the form of detecting software vulnerabilities and hidden security threats. Claude 3 can read entire source codes and identify several underlying complex security vulnerabilities that are used by Advanced Persistent Threats (APTs).

Jason D. Clinton, CISO at Anthropic, wanted to see this feature for himself. So he simply asked Claude 3 to role-play as a software detecting and vulnerability assistant and asked it to identify the vulnerabilities present in a Linux Kernel Code of 2145 lines. The user asked to specifically identify the vulnerability and also provide a solution to it.

Claude 3 excellently responds by first stating the location where the vulnerability is present and it also proceeds to provide the code blocks containing the threat.

It then continues to explain the entire vulnerability in detail even stating why it has arisen. It also explains how an attacker could potentially use this vulnerability to their benefit.

Lastly and most importantly it also provides a solution to address the concurrency vulnerability. It also provided the modified code with the fix.

You can see the entire Claude 3 conversation here: https://claude.ai/share/ddc7ff37-f97c-494c-b0a4-a9b3273fa23c

6. Solving a Chess Puzzle

Nat, a writer at The AI Observer, shared a screenshot with Claude 3 Opus consisting of a simple mate-in-2 puzzle. He asked Claude to solve the Chess puzzle and find a checkmate in 2 moves. He had also attached a solution to the puzzle as part of the JSON.

Claude 3 perfectly solved the puzzle with a fast response. However, it failed to do the same when the user deleted the JSON solution from the screenshot and prompted Claude again.

This shows Claude 3 is good at learning and solving tasks even including visual puzzles, however, it still needs an updated knowledge base in such matters.

7. Extracting Quotes from large books with provided reasoning

Claude 3 does an excellent job of extracting relevant quotes and key points from very large documents and books. It performs extremely well compared to Google’s Notebook LM.

Joel Gladd, Department Chair of Integrated Studies; Writing and Rhetoric, American Lit; Higher-Ed Pedagogy; OER advocate, asked Claude 3 to provide some relevant quotes from a book to support the points that the Chatbot had previously made in their discussion.

Claude amazingly gave 5 quotes as responses and even stated how they helped to illustrate the key points that Claude had made earlier. It even provided a short summary of the entire thesis. This just goes to show how well and advanced Claude 3’s thinking and processing capabilities are. For an AI Chatbot to support its points by extracting quotes from a book is a marvelous achievement.

8. Generating Midjourney Prompts

Apart from iteratively improving prompts in prompt engineering, Claude 3 even performs well in generating prompts itself. A user on X conducted a fun experiment with Claude 3 Opus. He gave a single text file of 1200 Midjourney prompts to the Chatbot and asked it to write 10 more.

Claude 3 did an amazing job in producing the prompts, keeping the right length, proper aspect ratio, and also appropriate prompt structure.

Later he also asked Claude to generate a prompt for a Total Recall-like movie, keeping the original prompts as basis. Claude responded well with a well-described prompt along with aspect ratios mentioned.

9. Decrypting Emails

Claude 3 does an amazing job in even decrypting emails that contain deliberately hidden texts. Lewis Owen, an AI enthusiast provided Claude 3 with an OpenAI email screenshot in which various parts of the email had been blacked out.

Claude did amazingly well in guessing the hidden text content and analyzing the entire email. This is highly crucial as OpenAI’s emails are edited word by word. The length of each original word is proportional to the newly done edit mark.

This groundbreaking technology from Claude has the potential to help us analyze and reveal information, paving the way toward the truth. This is all attributed to Claude 3’s excellent text understanding and analysis technology.

10. Creating custom animations to explain concepts

Claude 3 does amazingly well in creating custom video-like animations to explain basic educational concepts. It fully encapsulates every detail and also explains the concept algorithm step by step. In one of our recent articles, we already explored how users can create Math animations with Claude 3 and also provided tutorials on how to do so.

Here’s another instance obtained from Min Choi, an AI educator and entrepreneur, where he asked Claude 3 to generate a Manim animation explaining the Neural Network Architecture. The result was amazing where Claude provided a perfect video response explaining each Neural Network layer and how they are interconnected.

So, Claude 3 is making wonders when it comes to visually encapsulating concepts and portraying them to the audience. Who thought that one day we would have a Chatbot that perfectly explains concepts with complete video details?

11. Writing social media posts or tweets mimicking your style

Claude 3 is also designed to write social media captions just as you would on Twitter or any other platform. A well-known Twitter user chose to enter 800 of his tweets into Claude 3, and the outcomes were unexpected. Claude 3 can mimic the author’s writing style and, when necessary, make references to accounts such as @Replit and @everartai.

This is unimaginable and it’s all thanks to Claude 3’s intelligent processing based on the structured data provided. Now users can even have their post captions generated for them, that too in their writing style. This will be highly beneficial for those who run out of ideas and captions on what to post and how to post it.

12. Large Scale Text Search

For testing purposes, a user submitted a modified version of “The Great Gatsby” document to Claude 3. This test was created to assess Claude 3’s effectiveness and precision in rapidly locating certain information from vast quantities of text.

Claude 3 was asked to find out if there was anything wrong with the text’s context. The outcomes demonstrate that Claude 3 outperforms Claude 2.1, which was its predecessor and occasionally provided erroneous results (a behavior known as “hallucination”) when handling substantially identical tasks.

This shows that developers can use Claude 3 in tasks related to finding, editing, or sorting out specific information in large documents and save up a lot of time with the help of the Chatbot family.

13. A Potential Decompiler

A great decompiler for Python-compiled files (.pyc) is Claude 3. Furthermore, it can also function well in certain more complicated situations in addition to being effective in handling straightforward cases.

In the images below a user can be seen feeding a portion of a compiled Python bytecode to Claude 3. The chatbot decompiles it perfectly line by line and even mentions a decompiler tool named uncompyle6 for reference.

Conclusion

The diverse use cases and functionalities just goes to show how far Claude 3 has come in achieving brilliance in the field of Generative AI. Almost every developer’s aspect has been fulfilled by the Chatbot, and the list keeps on evolving. Who knows what else can we expect? This is just the beginning of our journey with Claude 3 as surely a lot more will unfold in the days to come. Stay tuned!