Large Language Models like GPT-4 can exhibit unsafe behaviors when communicating via ciphers, highlighting the need for safety alignment in non-natural languages.