Chatbots in Software Development

With the rise of Large Language Models (LLMs) in performing software engineering activities, software organizations have shown an increased interest in integrating LLM-based chatbots in their internal processes with a goal to assist software engineers and thus improve their productivity.

ICET-lab has been investigating three main aspects in connection to chatbots in software engineering:

• The usage of chatbots and LLMs in the software engineering industry.
• The design of chatbots for different tasks in the Software Development Lifecycle (SDLC).
• The evaluation of chatbots for different activities in software engineering.

Our research present a framework that descibes how software engineers of different roles use chatbots (Khojah et al., 2024). The framework is based on a categorization of engineer-to-chatbot dialogues that illustrated three main purposes of a conversation. Software engineers initate a dialogue with a chatbot when they want to

• manipulate software artifacts,
• get guidance on how to solve a problem, and
• get training on a certain skill or concept.

The different purposes along with internal factors (concern the prompting) and external factors (concern the organization and chatbot policy) impact the personal experience of the software engineer in terms of productivity and trust.

Framework of ChatGPT usage in software engineering industry.

We are currently studying the impact of context (one of the internal factors) on the quality of the generation of software artifacts. This will be the first step to understand the aspects to consider when designing chatbots to assist in software projects.

Contacts:

Ranim Khojah

Dr. Francisco Gomes de Oliveira Neto

Dr. Philipp Leitner