A company is using a pre-trained large language model (LLM) that must handle several tasks needing domain-specific technical knowledge. The LLM lacks information on some domain topics, and the company has unlabeled domain data available for tuning. Which fine-tuning approach should the company use?

Continued pre-training is the most suitable approach because the company has unlabeled domain-specific data and the model lacks knowledge in those areas. This method involves further training the pre-trained LLM on the new domain text without requiring human-labeled examples, allowing the model to learn the specific vocabulary, concepts, and patterns of the domain. Full training is impractical and resource-intensive, as it discards the existing knowledge of the pre-trained model. Supervised fine-tuning requires labeled data, which the company does not have. RAG is an inference-time technique that retrieves information from external sources but does not inherently update the model's foundational knowledge or address its lack of understanding of the domain itself.