Explaining Text-to-Command Conversational Models

Abstract

Large Language Models (LLMs) have revolutionized machine learning and natural language processing, demonstrating remarkable versatility across various tasks. Despite their advancements, their application in critical fields is hindered by a lack of effective interpretability and explainability. In our company, we have fine-tuned a text-to-command conversational AI model that translates natural language inputs into executable network commands. This paper presents our findings on explaining the model’s reasoning processes, aiming to enhance understanding, identify biases, and improve performance. We explore techniques such as token attributions, hidden state visualizations, neuron activation, and attention mechanisms to elucidate model behavior. Our work contributes to the development of more interpretable and trustworthy AI systems, pushing the boundaries of conversational AI.