NVIDIA/TensorRT-LLM
Star 趋势
项目介绍与用途
「NVIDIA/TensorRT-LLM」是由 NVIDIA 维护、主要使用 Python 开发的开源项目。TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way。目前该项目在 GitHub 上累计获得 15.0k 个 Star、2.9k 次 Fork（今日新增 9 Star）。如果你关注 Python 方向的开源生态，它值得收藏、学习与参考。
项目简介
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
相关项目
