The paper introduces a novel approach, VQCT, to transfer a well-trained codebook from language models to enhance Vector-Quantized Image Modeling (VQIM). By utilizing part-of-speech knowledge and semantic relationships from pretrained language models, the proposed framework aims to alleviate codebook collapse issues. Experimental results demonstrate superior performance over existing methods on four datasets. The method involves constructing vision-related codebooks, designing a codebook transfer network, and achieving cooperative optimization between codes.
다른 언어로
소스 콘텐츠 기반
arxiv.org
더 깊은 질문