Exploratory Studies LAIA@UIB : Pre-trained Natural Language Processing models to respond to domain-specific inputs

Abstract

With the rapid development of conversational agents, there is a need to create models that can understand human requests and respond effectively to them. This thesis explores the process of fine-tuning a pre-trained BLOOM language model with 3 billion parameters for use in a chatbot question-answering system. The study was conducted using both quantitative and qualitative assessment methods, including analysis of keyword relevance, text readability, and sentiment analysis of model responses. Methods for optimizing memory use for loading and configuring the model were also discussed and were found to be effective in saving resources. The results of fine-tuning the BLOOM model are promising in terms of relevance, readability and tone of responses. In summary, this study demonstrates the potential of fine-tuning large pretrained models such as BLOOM to create effective question answering systems in chatbots, but more research is needed to further adapt to different contexts and refine aspects of model behavior.