Why Large Language Models Aren't Cracking Cryptography Yet

cryptography, the race to crack symmetric keys is relentless. Researchers have explored a many of methods, including machine learning models trained on plaintext and ciphertext pairs. Recently, the idea of using large language models (LLMs) for neural distinguishers has emerged, but let's apply some rigor here, this approach hasn't lived up to its hype.

LLMs vs. ResNet

The study focused on SPECK-32/64, a lightweight block cipher, comparing LLM-based neural distinguishers with those using ResNet. The results were underwhelming. LLMs didn't offer any observable improvement in distinguishing capability. This raises the question: are LLMs really the right tool for this kind of cryptanalysis?

The Role of Differences in High Rounds

Another key insight from the research was the diminishing effectiveness of selecting specific differences at higher rounds for both LLMs and ResNet. As the rounds increase, the noise overwhelms the signal, rendering the choice of differences practically moot. This isn't entirely surprising, I've seen this pattern before in other cryptographic analyses.

Glimmer of Promise

All isn't lost for LLM enthusiasts, though. The study found that by tweaking prompt designs to include the XOR operation results, the performance of LLM-based distinguishers could be significantly enhanced. This suggests a potential niche where LLMs might excel. But let's be clear: without tangible, reproducible results improving upon existing methods, this remains more of an academic curiosity than a practical breakthrough.

Color me skeptical, but until these models demonstrate a consistent edge over established methodologies, the cryptography community will likely stick with tried-and-true methods. After all, trust in cryptographic systems isn't something earned easily, nor should it be.