All publications

Report

Implementing Large Language Models on Low-Power Embedded Systems

This paper presents a methodology for integrating LLMs into constrained environments, with a focus on the NVIDIA Jetson Orin Nano. We examine optimization techniques such as 4-bit quantization and knowledge distillation, which significantly reduce memory requirements with minimal accuracy loss.