From Manual Training to Domain-Specific Adaptation through Constrained Prompt Engineering and Self-Training

Current fine-tuning of large language models typically relies on manually curated datasets to enhance model performance in specialized domains. However, with the rise of prompt engineering, is it possible for models to utilize constrained prompts like “You are an expert in semiconductor materials” or “You are an expert in fluid mechanics” to trigger domain-specific self-training? Could these prompts enable the model to autonomously retrieve and analyze information from open databases, thereby achieving a refined level of self-tuning without human intervention? This Perspective explores the feasibility of this idea and its potential to transform the conventional fine tuning paradigm. We present conceptual models and experimental comparisons that illustrate the differences in model responses with and without constrained prompts. Finally, we discuss how enabling self-training in large models could greatly enhance their utility in solving targeted domain specific challenges.