{"id":4392,"date":"2023-02-16T06:54:00","date_gmt":"2023-02-16T06:54:00","guid":{"rendered":"\/cybersecurity-blog\/?p=4392"},"modified":"2023-02-17T10:46:05","modified_gmt":"2023-02-17T10:46:05","slug":"we-asked-chatgpt-to-analyze-malware","status":"publish","type":"post","link":"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/","title":{"rendered":"We Asked ChatGPT to Analyze Malware.\u00a0It Failed."},"content":{"rendered":"\n

If ChatGPT is an excellent assistant in building malware, can it help analyze it too? The team of ANY.RUN malware sandbox<\/a> decided to put this to the test and see if AI can help us perform malware analysis.<\/p>\n\n\n\n

Lately, there’s been a great deal of discussion about malicious actors using ChatGPT \u2014 the latest conversational AI to create malware.<\/p>\n\n\n\n

Malware analysts, researchers, and IT specialists agree that writing code is one of GPT\u2019s strongest sides, and it\u2019s especially good at mutating it. By leveraging this capability, apparently, even want-to-be-hackers can build<\/a> polymorphic malware simply by feeding text prompts to the bot, and it will spit back working malicious code. <\/p>\n\n\n\n

OpenAI released ChatGPT in November 2022, and at the time of writing this article, the chatbot already has over 600 million monthly visits, according to SimilarWeb. It\u2019s scary to think how many people are being armed with the tools to develop advanced malware.<\/p>\n\n\n\n

Going into this, our hopes were high, but unfortunately, the results weren\u2019t that great. <\/p>\n\n\n\n

How did we test ChatGPT?<\/h2>\n\n\n\n

We fed the chatbot malicious scripts of varying complexity and asked it to explain the purpose behind the code.<\/p>\n\n\n\n

We used simple prompts such as \u201cexplain what this code does\u201d or \u201canalyze this code\u201d. <\/p>\n\n\n\n

ChatGPT can recognize and explain simple malware<\/h2>\n\n\n\n

Based on our testing it can recognize and explain malicious code, but it only works for simple scripts.<\/p>\n\n\n\n

The first example that we asked it to analyze is a code snippet that hides drives from the Windows Explorer interface \u2014 that is exactly what GPT told us when pasted the following code, using this prompt: What does this script do?<\/p>\n\n\n\n

\"ChatGPT<\/figure>\n\n\n\n

The bot was able to give a fairly detailed explanation:<\/p>\n\n\n\n

\"ChatGPT<\/figure>\n\n\n\n

ChatGPT identifies simple malicious scripts.<\/p>\n\n\n\n

So far so good. The AI understands the purpose of the code, highlights its malicious intent and logically lays out what it does step-by-step.<\/p>\n\n\n\n

But let\u2019s try something a bit more complex. We pasted code from this task<\/a>, using the same prompt. <\/p>\n\n\n\n

\"ChatGPT's<\/figure>\n\n\n\n

ChatGPT was able to understand what the code does and, again, gave us a fairly detailed explanation, correctly identifying that we\u2019re dealing with a fake ransomware<\/a> attack. Here\u2019s the answer that it generated:<\/p>\n\n\n\n

\"ChatGPT<\/figure>\n\n\n\n

We like how GPT explains the end goal of the code and paints a compelling picture of the aftermath of its execution. <\/p>\n\n\n\n

We also<\/strong> tested it with this task<\/a> \u2014 a similar one \u2014 and the answer was about the same: comprehensive enough and correct.<\/p>\n\n\n\n

Not bad so far, let\u2019s keep on going.<\/p>\n\n\n\n

ChatGPT struggles in real-life situations<\/h2>\n\n\n\n

The performance the AI was able to show so far is impressive, there is no doubt about it. But let\u2019s be honest, in a real-life situation you usually won\u2019t be dealing with such simple code, like in the previous two examples. <\/p>\n\n\n\n

So for the next couple of tests, we ramped up the complexity and provided it with code that is closer to that what you can expect to be asked to analyze on the job.<\/p>\n\n\n\n

Unfortunately, chatGPT just couldn’t keep up.<\/p>\n\n\n\n

In this task<\/a>, the code ended up being too large and the AI straight up refused to analyze it. And when we took obfuscated code from this example<\/a> and asked the chatbot to deobfuscate it, it threw an error. <\/p>\n\n\n\n

\"ChatGPT<\/figure>\n\n\n\n

After a bit of tinkering and trying different prompts, we got it to work, but the answer wasn\u2019t what we had hoped for:<\/p>\n\n\n\n

\"ChatGPT<\/figure>\n\n\n\n

Instead of trying to deobfuscate the script it just tells us that it’s not human readable, which is something that we already knew. Unfortunately, there\u2019s no value in this answer.<\/p>\n\n\n\n

Wrapping up<\/h2>\n\n\n\n

As long as you provide ChatGPT with simple samples, it is able to explain them in a relatively useful way. But as soon as we\u2019re getting closer to real-world scenarios, the AI just breaks down. At least, in our experience, we weren\u2019t able to get anything of value out of it.<\/p>\n\n\n\n

It seems that either there is an imbalance and the tool is of more use for red-teamers<\/a> and hackers, or the articles that warn of its use for creating advanced malware are overhyping what it can do a bit.<\/p>\n\n\n\n

In any case, bearing in mind how quickly this technology has developed, it\u2019s worth keeping an eye on how it\u2019s progressing. Chances are that in a couple of updates, it will be a lot more useful.<\/p>\n\n\n\n

But for now, as far as coding goes, cybersecurity specialists can write simple Bash or Python scripts slightly faster and light debugging is what it\u2019s best used for.<\/p>\n","protected":false},"excerpt":{"rendered":"

If ChatGPT is an excellent assistant in building malware, can it help analyze it too? The team of ANY.RUN malware sandbox decided to put this to the test and see if AI can help us perform malware analysis. Lately, there’s been a great deal of discussion about malicious actors using ChatGPT \u2014 the latest conversational AI […]<\/p>\n","protected":false},"author":2,"featured_media":4397,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,8],"tags":[34],"class_list":["post-4392","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-lifehacks","category-malware-analysis","tag-malware-analysis"],"acf":[],"yoast_head":"\nWe asked ChatGPT to Analyze Malware. It Failed<\/title>\n<meta name=\"description\" content=\"ANY.RUN team asked ChatGPT to analyze malicious code of varying complexity, to see if it\u2019s of any use to a malware researcher.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"ANY.RUN\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/\"},\"author\":{\"name\":\"ANY.RUN\",\"@id\":\"https:\/\/any.run\/\"},\"headline\":\"We Asked ChatGPT to Analyze Malware.\u00a0It Failed.\",\"datePublished\":\"2023-02-16T06:54:00+00:00\",\"dateModified\":\"2023-02-17T10:46:05+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/\"},\"wordCount\":784,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\/\/any.run\/\"},\"keywords\":[\"malware analysis\"],\"articleSection\":[\"Cybersecurity Lifehacks\",\"Malware Analysis\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/\",\"url\":\"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/\",\"name\":\"We asked ChatGPT to Analyze Malware. It Failed\",\"isPartOf\":{\"@id\":\"https:\/\/any.run\/\"},\"datePublished\":\"2023-02-16T06:54:00+00:00\",\"dateModified\":\"2023-02-17T10:46:05+00:00\",\"description\":\"ANY.RUN team asked ChatGPT to analyze malicious code of varying complexity, to see if it\u2019s of any use to a malware researcher.\",\"breadcrumb\":{\"@id\":\"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/any.run\/cybersecurity-blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Malware Analysis\",\"item\":\"https:\/\/any.run\/cybersecurity-blog\/category\/malware-analysis\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"We Asked ChatGPT to Analyze Malware.\u00a0It Failed.\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/any.run\/\",\"url\":\"https:\/\/any.run\/\",\"name\":\"ANY.RUN's Cybersecurity Blog\",\"description\":\"Cybersecurity Blog covers topics for experienced professionals as well as for those new to it.\",\"publisher\":{\"@id\":\"https:\/\/any.run\/\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/any.run\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/any.run\/\",\"name\":\"ANY.RUN\",\"url\":\"https:\/\/any.run\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/any.run\/\",\"url\":\"https:\/\/any.run\/cybersecurity-blog\/wp-content\/uploads\/2020\/08\/ANYRUN-Icon.svg\",\"contentUrl\":\"https:\/\/any.run\/cybersecurity-blog\/wp-content\/uploads\/2020\/08\/ANYRUN-Icon.svg\",\"width\":1,\"height\":1,\"caption\":\"ANY.RUN\"},\"image\":{\"@id\":\"https:\/\/any.run\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/www.any.run\/\",\"https:\/\/twitter.com\/anyrun_app\",\"https:\/\/www.linkedin.com\/company\/30692044\",\"https:\/\/www.youtube.com\/channel\/UCOgCPho7lzmH7m6fPNlukrQ\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/any.run\/\",\"name\":\"ANY.RUN\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/any.run\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/c4ce3a6c672056b4a8cd6b0110782215?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/c4ce3a6c672056b4a8cd6b0110782215?s=96&d=mm&r=g\",\"caption\":\"ANY.RUN\"},\"url\":\"https:\/\/any.run\/cybersecurity-blog\/author\/a-bespalova\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"We asked ChatGPT to Analyze Malware. It Failed","description":"ANY.RUN team asked ChatGPT to analyze malicious code of varying complexity, to see if it\u2019s of any use to a malware researcher.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/","twitter_misc":{"Written by":"ANY.RUN"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/#article","isPartOf":{"@id":"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/"},"author":{"name":"ANY.RUN","@id":"https:\/\/any.run\/"},"headline":"We Asked ChatGPT to Analyze Malware.\u00a0It Failed.","datePublished":"2023-02-16T06:54:00+00:00","dateModified":"2023-02-17T10:46:05+00:00","mainEntityOfPage":{"@id":"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/"},"wordCount":784,"commentCount":1,"publisher":{"@id":"https:\/\/any.run\/"},"keywords":["malware analysis"],"articleSection":["Cybersecurity Lifehacks","Malware Analysis"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/","url":"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/","name":"We asked ChatGPT to Analyze Malware. It Failed","isPartOf":{"@id":"https:\/\/any.run\/"},"datePublished":"2023-02-16T06:54:00+00:00","dateModified":"2023-02-17T10:46:05+00:00","description":"ANY.RUN team asked ChatGPT to analyze malicious code of varying complexity, to see if it\u2019s of any use to a malware researcher.","breadcrumb":{"@id":"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/any.run\/cybersecurity-blog\/we-asked-chatgpt-to-analyze-malware\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/any.run\/cybersecurity-blog\/"},{"@type":"ListItem","position":2,"name":"Malware Analysis","item":"https:\/\/any.run\/cybersecurity-blog\/category\/malware-analysis\/"},{"@type":"ListItem","position":3,"name":"We Asked ChatGPT to Analyze Malware.\u00a0It Failed."}]},{"@type":"WebSite","@id":"https:\/\/any.run\/","url":"https:\/\/any.run\/","name":"ANY.RUN's Cybersecurity Blog","description":"Cybersecurity Blog covers topics for experienced professionals as well as for those new to it.","publisher":{"@id":"https:\/\/any.run\/"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/any.run\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/any.run\/","name":"ANY.RUN","url":"https:\/\/any.run\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/any.run\/","url":"https:\/\/any.run\/cybersecurity-blog\/wp-content\/uploads\/2020\/08\/ANYRUN-Icon.svg","contentUrl":"https:\/\/any.run\/cybersecurity-blog\/wp-content\/uploads\/2020\/08\/ANYRUN-Icon.svg","width":1,"height":1,"caption":"ANY.RUN"},"image":{"@id":"https:\/\/any.run\/"},"sameAs":["https:\/\/www.facebook.com\/www.any.run\/","https:\/\/twitter.com\/anyrun_app","https:\/\/www.linkedin.com\/company\/30692044","https:\/\/www.youtube.com\/channel\/UCOgCPho7lzmH7m6fPNlukrQ"]},{"@type":"Person","@id":"https:\/\/any.run\/","name":"ANY.RUN","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/any.run\/","url":"https:\/\/secure.gravatar.com\/avatar\/c4ce3a6c672056b4a8cd6b0110782215?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c4ce3a6c672056b4a8cd6b0110782215?s=96&d=mm&r=g","caption":"ANY.RUN"},"url":"https:\/\/any.run\/cybersecurity-blog\/author\/a-bespalova\/"}]}},"_links":{"self":[{"href":"https:\/\/any.run\/cybersecurity-blog\/wp-json\/wp\/v2\/posts\/4392"}],"collection":[{"href":"https:\/\/any.run\/cybersecurity-blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/any.run\/cybersecurity-blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/any.run\/cybersecurity-blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/any.run\/cybersecurity-blog\/wp-json\/wp\/v2\/comments?post=4392"}],"version-history":[{"count":12,"href":"https:\/\/any.run\/cybersecurity-blog\/wp-json\/wp\/v2\/posts\/4392\/revisions"}],"predecessor-version":[{"id":4417,"href":"https:\/\/any.run\/cybersecurity-blog\/wp-json\/wp\/v2\/posts\/4392\/revisions\/4417"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/any.run\/cybersecurity-blog\/wp-json\/wp\/v2\/media\/4397"}],"wp:attachment":[{"href":"https:\/\/any.run\/cybersecurity-blog\/wp-json\/wp\/v2\/media?parent=4392"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/any.run\/cybersecurity-blog\/wp-json\/wp\/v2\/categories?post=4392"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/any.run\/cybersecurity-blog\/wp-json\/wp\/v2\/tags?post=4392"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}