Oh wow, LLMs are having issues with using an esoteric language they haven't been trained on. Might as well point them at a picture of Indus script and be baffled how it cannot parse it.

As long as its a language, an LLM will be proficient in parsing it given enough training and data.

This is a very interesting experiment though, surprised so see how some of the models faired.