Large language models can be effectively used for zero-shot reranking to significantly improve audio topic retrieval performance without requiring any task-specific in-domain training data.