Infrastructure for the deployment of Large Language Models: challenges and solutions

Authors

  • Tomasz Walkowiak Faculty of Information and Communication Technology, Wroclaw University of Science and Technology, Wroclaw, Poland
  • Bartosz Walkowiak Faculty of Information and Communication Technology, Wroclaw University of Science and Technology, Wroclaw, Poland

DOI:

https://doi.org/10.24425/ijet.2025.153620

Abstract

Large Language Models are increasingly prevalent, and their capabilities are advancing rapidly due to extensive research in this field. A growing number of models are being developed, with sizes significantly surpassing 70 billion parameters. As a result, the ability to perform efficient and scalable inferences on these models is becoming crucial to maximize the utilization of valuable resources such as GPUs and CPUs. This thesis outlines a process for selecting the most effective tools for efficient inference, supported by the results of experiments. Additionally, it provides a comprehensive description of an end-to-end system for the inference process, encompassing all components from model inference and communication to user management and a userfriendly web interface. Furthermore, we detail the development of an LLM chatbot that leverages the function-calling capabilities of LLMs and integrates various external tools, including weather prediction, Wikipedia information, symbolic math, and image generation.

Published

2025-07-11

How to Cite

Walkowiak, Tomasz, and Bartosz Walkowiak. “Infrastructure for the Deployment of Large Language Models: Challenges and Solutions”. International Journal of Electronics and Telecommunications, vol. 71, no. 3, July 2025, doi:10.24425/ijet.2025.153620.

Issue

Section

Artykuły