Concepts de base
Developing a domain-specialized generative legal language model from scratch is feasible with limited data, showcasing strong legal reasoning capabilities.
Résumé
The PARAMANU-AYN model is a novel generative legal language model exclusively trained on Indian legal documents. It demonstrates the ability to draft legal contracts and clauses with limited instruction tuning. The model's evaluation on various metrics shows promising results despite not being pretrained on extensive legal data.
Introduction:
Introducing PARAMANU-AYN, a generative legal language model based on Indian case documents.
Abstract:
Presents the model pretrained exclusively on Supreme Court cases, Constitution of India, and Indian Penal Code.
Related Work:
Discusses existing large language models in the legal domain and their limitations.
Background:
Explains language modeling objectives and model performance evaluation metrics.
Data:
Details the pretraining data sources and instruction tuning dataset for the model.
Training:
Describes the training process for PARAMANU-AYN at an 8192 context size.
Results and Analyses:
Reports validation perplexity, MFU metrics, CPU inference speed, and GPT-3.5-Turbo evaluation results.
Conclusions:
Summarizes the contributions of PARAMANU-AYN as a dedicated generative legal language model for Indian jurisdiction.
Stats
Our model can be run on CPU and achieved 42.46 tokens/sec CPU inference speed.
We found that our models were able to learn the domain knowledge required for drafting various legal contracts and clauses.
Despite not being pretrained on extensive legal data, our models showcased strong domain-specific generative capabilities.