The paper introduces a novel approach, VQCT, to transfer a well-trained codebook from language models to enhance Vector-Quantized Image Modeling (VQIM). By utilizing part-of-speech knowledge and semantic relationships from pretrained language models, the proposed framework aims to alleviate codebook collapse issues. Experimental results demonstrate superior performance over existing methods on four datasets. The method involves constructing vision-related codebooks, designing a codebook transfer network, and achieving cooperative optimization between codes.
Na inny język
z treści źródłowej
arxiv.org
Głębsze pytania