Integrating lexical and syntactic knowledge can consistently improve the performance of unsupervised cross-lingual transfer across various NLP tasks.