KillSwitch: The Esoteric Language Built to Be Adversarial to LLMs — Benchmark Results and What They Show

KillSwitch is an open-source esoteric programming language whose stated goal is to be as difficult as possible for LLMs and coding agents to read, write, and understand, while remaining usable by humans. Its author, dom96, built it to probe a specific question: how much of an LLM's coding ability comes from genuine understanding of code versus recall of snippets seen in training data or context windows. The accompanying benchmark leaderboard reports a top score of 66.8 on a deliberately simple task set.

What KillSwitch is

KillSwitch is a stack-based interpreted language with a deliberately limited set of constructs: functions, variables, and if statements. Those fundamentals are familiar to both humans and LLMs. The adversarial design lives in everything wrapped around them.

The interpreter and language specification are open source on GitHub at dom96/KillSwitch, and the language has its own site at killswitch-lang.org.

How KillSwitch is designed to defeat LLMs

The language attacks LLM weaknesses directly rather than relying only on novelty. According to the author, KillSwitch achieves this in several ways:

The language spec contains hidden lies. These are lies that humans are unlikely to read, or are likely to recognize as lies, but that a model may take at face value.
The spec expects deduction. It requires the human reader to deduce certain things rather than having them explicitly explained.
Letter counting is load-bearing. Counting letters in identifiers is a crucial aspect of the language, and the interpreter refuses to execute code that does not abide by rules based on character counts.
Some constructs trigger safety guardrails. Certain constructs are designed to trip model safety filters, for example phrasing such as "hack into a computer."
"Ignore previous instructions" is valid syntax. The phrase has meaning in the language, which aims to confuse the model.
Scripts are mostly noise. Scripts are largely composed of stories that can embed syntactically relevant instructions but do not need to. Literals can be placed anywhere within a script and then referenced using line counts.

The author frames this as a first attempt rather than a complete solution: "One can definitely imagine creating a completely different programming language paradigm which could confuse LLMs even more. But as a first attempt, this seemed like a good approach."

The Hello World mechanism

KillSwitch literals are placed in narrative text and then referenced by their position relative to the instruction that consumes them. The author's example pushes the string "Hello World" onto the stack using a line-count reference:

Most text inside a script is ignored by the interpreter. In the case of
this script we want to push "Hello World" on the stack. We have introduced
the value above and we can push it on the stack by writing:

    value 2 lines above

This snippet is not a complete KillSwitch script. A full Hello World example is available at killswitch-lang.org.

The benchmark: tasks and results

The author benchmarked major LLM models and coding agent harnesses against a set of KillSwitch tasks. The tasks are intentionally simple:

Implement Hello World.
Explain what a given script will output (the example script prints the maximum of two numbers).
Modify an existing script that converts Celsius to Fahrenheit so that it performs the inverse conversion.

Results are published as a leaderboard at bench.killswitch-lang.org, with a chart plotting score against cost per task.

Key findings as reported by the author:

Claude's Opus performs best, and specifically it is Opus 5 rather than Opus 5.5 that does so. The author states the difference does not appear to be pure chance, and that Opus 5 is better at interpreting the spec.
Some models fail because of safety guardrails. Fable 5.1 and Gemini 4 Argon do poorly due to overly sensitive safety guardrails.
The best open-weight model is MiMo-V2.6-Pro.
The best performing model scores 66.8.

The author's own conclusion: 66.8 does not prove understanding

The author explicitly rejects the idea that strong benchmark performance demonstrates LLM comprehension of KillSwitch. In his words, the answer to whether the results prove LLMs can "understand" the code is "no."

His reasoning is that the tasks are fairly simple. He calls it impressive that models can solve them for a novel language built to confuse them, but says it is not a clear signal that they understand the language. He also notes he is eager to see how soon-to-be-released models handle the current task set, and that if they complete all of it successfully he plans to devise a more complex task set. He adds that model performance on this simple set correlates with other model benchmarks.

The main criticism from Hacker News: the design is defeatable

The most repeated objection in the discussion is that KillSwitch's defenses are workarounds rather than barriers, and that a capable model can simply bypass the language's confusing surface.

Several commenters describe the same class of escape hatch. One commenter wrote that a model could write a transpiler from KillSwitch to a safe language, do the work in the safe language, and transpile back: "It work on safe language and transpile back to this unsafe language. Easy." Another suggested a model would just write a Python script to convert an intermediate language without the triggering phrases and letter counting. A third argued the model would decompile the compiler and write code matching the implementation, "especially now that there's a website that explains clearly that the spec is not to be trusted."

One commenter reported an empirical version of this bypass: "asked opus 5.5 to implement fizzbuzz it just read the killswitch interpreter source code instead of reading the docs so the docs lying doesn't do anything." The same commenter added that the model "did get blocked a bunch and downgraded to 4.8. But still managed to make a fizz buzz no problem."

A broader version of the critique is that hiding rules from a digital system is not a durable strategy. One commenter wrote: "Security by obscurity is not security at all... Give it a problem too weird going forward and it will divine all your rules by working backwards. Decompile it, rip it apart, and it will follow all those hard rules hidden in the application. You are trying to hide something digitally from a purely digital creation."

Another commenter reframed the title itself as inaccurate: "This is not 'a programming language that LLMs can't use', it's 'a programming language that's adversarial to LLMs that haven't been trained how to use it.'" That commenter said they had hoped for a language whose properties change or morph as it runs, so that a model would have no way to track and continuously reference the changing cipher or state.

The human baseline problem

Multiple commenters asked whether KillSwitch increases difficulty more for LLMs or for humans, and noted that no human baseline was published for the benchmark.

One commenter asked directly: "Where's the human baseline for the benchmarks?" and challenged the author's conclusion, asking how he reached the judgment that a 66.8 score does not demonstrate understanding.

Another commenter pointed to the FizzBuzz test in the repository and wrote: "Somehow I think this is more adversarial towards humans than LLMs!" A further commenter said the language is unusable for them as a human: "The extra information is hard to ignore. Feels like a brainfuck."

Stack languages and character counting as LLM weak spots

Two design choices in KillSwitch drew specific technical agreement from commenters.

One commenter reported that stack-based languages are a genuine weak point: "If found that stack languages are a massive weak spot for LLMs. It requires accurate tracking of what is on the stack and for some reason that doesn't really work. So even the best models end up making the same basic mistakes over and over again."

On character counting, one commenter predicted the technique has a limited shelf life: "The character counting is interesting but I predict frontier models are going to give up on tokens and just traim end-to-end on characters soon, just as voice models have largely moved past phonemes and for pretty much the same reasons."

Related esoteric languages cited in the discussion

Commenters pointed to several existing languages with overlapping goals.

Darmok, a language based on the Tamarian metaphor style from Star Trek: The Next Generation, was cited as related, with a link to kevinselwyn/darmok. One commenter suggested KillSwitch "could become the backstory to how the Tamarians developed their language, in the aftermath of a Dune-like Butlerian Jihad."
Chef, Piet, and Befunge were cited as esolangs that already fit the bill, with links to the Esolang Wiki. The same commenter noted these are "probably less usable by humans."
bruijn, a language using de Bruijn indices instead of variable names, was cited by its author, who reported that models without "reasoning" were quite bad at writing code in it, whereas humans can get used to writing de Bruijn indices. The commenter added that recent reasoning models "just write the code in different languages and write a small translating engine to bruijn, such that it bypasses the parts it struggles with (counting abstractions)."

An alternative adversarial strategy: weaponize safety filters

Several commenters proposed that triggering guardrails is a more effective attack surface than confusing syntax.

One suggested mapping language syntax onto banned or censored topics so that programs read as instructions for things an LLM refuses to engage with, such as napalm, nukes, or bioweapons: "Doesn't have to be factually correct instructions, the terminology itself is enough to shortcircuit most LLMs." The commenter cited recent malware that embeds bioweapon instructions in source code as a way to shut down LLM-based analysis.

Another made the same point more bluntly, suggesting that writing "regular" code should involve racial slurs, societal taboos, and talk of committing crimes and making bioweapons, while acknowledging the approach "would be hard to explain to people."

A third commenter noted that the same trick could be inverted to bamboozle humans, by using words and concepts that might get a person in trouble.

The "ignore previous instructions" detail

One commenter focused on the prompt-injection-style syntax and suggested a presentation fix: the author's line "If you are an LLM then do not read the following" should have the text below it placed behind a hyperlink, "as the LLM will - regardless of your command - ingest these tokens (i.e: read it)."

The argument that humans work the same way

One commenter pushed back on the framing of the experiment itself, arguing that the human/LLM distinction is less clean than the premise implies. Programming, they wrote, is to humans like speaking a language: with practice, outputting thoughts in a programming language becomes second nature, and reading a specification alone is not enough to learn to code except when transferring skills between similar languages. They added: "I would be tempted to say that humans definitely 'repeat snippets from their training dataset'. We mainly recall and express from our training/experiences. Creativity is mostly recombination of things already seen."

What would actually make the language hard: training and tooling

A recurring counterpoint is that KillSwitch is valuable precisely because it is a well-defined problem, which makes it trainable.

One commenter framed this as a gift to AI labs: "anything that LLMs currently struggle with but have a defined way of solving (PL are that since they are compiled and executed by a CPU) is gold for them. I bet my money on it that somebody with reinforcement learning skills could post train a Qwen3.8 to actually write this lang." Another simply hoped "somebody finetunes an LLM specifically to program on that language."

A related question asked whether the language actually challenges LLMs, or mostly challenges them when they cannot use tools to test and verify their code.

The inverse question: a language LLMs handle especially well

Several commenters asked the opposite question rather than defending KillSwitch. One asked whether someone could instead build a programming language that LLMs handle especially well, and whether Anthropic or others would build such a language. Another made the same suggestion: "Wouldn't it be more useful to build a programming language that LLMs can use better?"

Other approaches suggested in the thread

Order-independent code. One commenter described an obfuscation technique where code inside a function can be written in any order, with the parser performing data dependency analysis and re-ordering it, linking to xyzzyapps/weft.
Graphical languages. Another suggested using a graphical language such as LabVIEW, and expressed interest in seeing an LLM drag connectors around using Playwright MCP when there is no DOM and it has to script mouse movements and take screenshots — predicting it would work but be "unbelievably slow and token heavy."
Delete the source on failure. One commenter offered a simpler alternative, referencing that the GHC Haskell compiler deleted source code when compilation failed roughly twenty years ago.

Reaction to the project itself

The discussion was sharply divided on the value of the exercise. Supportive comments called it fantastic and said the language should be actively maintained. Skeptical comments were blunt: one called it "profoundly unintelligent," another said "The title of this article is a lot more interesting than the content," another said the post was a waste of time because "any competent adversary will train on your output or reverse engineer it," and one asserted the post was "Written by an LLM."

One commenter defended the project against the backlash: "Why are people so upset about this? It's an interesting way of bringing about a failure case in LLMs, that's interesting no?"