Conceitos essenciais
Efficiently improving Language Model sensitivity by weighting domain-specific terms during fine-tuning.
Estatísticas
Fine-tuning introduces new knowledge into an LM.
MSLM improves LM sensitivity and detection of DS-terms.
Optimal masking rate depends on the LM, dataset, and sequence length.
Citações
"The awareness of or sensitivity of PLMs towards DS-terms can be appropriately elevated without hurting their downstream performance." - Abstract