Large Language Models (LLMs) have achieved tremendous success in natural language processing tasks. However, in vertical domains such as underground engineering, the lack of large-scale structured corpora and the requirement for high-accuracy responses significantly increase the difficulty of constructing domain knowledge LLMs. Simultaneously, these challenges amplify the inherent flaws of models, such as machine hallucination and outdated content. To address the aforementioned challenges, firstly, we proposed a framework for constructing underground engineering domain knowledge LLMs, which includes human-machine collaborative knowledge extraction, parameter efficient fine-tuning, and retrieval-augmented generation. The framework aims to provide a universal construction paradigm for underground engineering domain knowledge LLMs. Secondly, within the proposed framework, we established an underground engineering domain knowledge database (UndergrData), introduced a domain knowledge extraction approach based on a few-shot prompt learning data augmentation strategy, resulting in the formation of the training dataset; we applied an underground engineering domain knowledge LLMs fine-tuning method under low-resource and constructed the first underground engineering domain knowledge LLM base (UndergrGPT); we adopted the approach of underground engineering domain knowledge retrieval-augmented generation, enabling high-quality response generation. Finally, compared with general LLMs, UndergrGPT effectively reduced hallucinations, demonstrating a significant advantage in response accuracy. It’s worth noting that the proposed framework can be extended to other vertical domains.