Repositories list

• lightllm

  Public
  LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

  nlpdeep-learningllama+ 4gptmodel-servingllmopenai-triton

  Python

  •

  Apache License 2.0

  •207•2.6k•61•7•Updated Nov 26, 2024Nov 26, 2024

• general-sam-py

  Public
  Python bindings for general-sam and some utilities

  Python

  •

  Apache License 2.0

  •0•3•0•2•Updated Nov 25, 2024Nov 25, 2024

• llmc

  Public
  [EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".

  benchmarkdeploymenttool+ 17evaluationpruningquantizationpost-training-quantizationawqlarge-language-modelsllm+ 10

  Python

  •

  Apache License 2.0

  •34•326•2•0•Updated Nov 25, 2024Nov 25, 2024

• mtc-token-healing

  Public
  Token healing implementation in Rust

  Rust

  •

  Apache License 2.0

  •0•3•0•3•Updated Nov 18, 2024Nov 18, 2024

• general-sam

  Public
  A general suffix automaton implementation in Rust with Python bindings

  Rust

  •

  Apache License 2.0

  •0•4•0•1•Updated Oct 18, 2024Oct 18, 2024

• EasyLLM

  Public
  Built upon Megatron-Deepspeed and HuggingFace Trainer, EasyLLM has reorganized the code logic with a focus on usability. While enhancing usability, it also ensures training efficiency.

  Python

  •

  Apache License 2.0

  •7•41•0•0•Updated Sep 18, 2024Sep 18, 2024

• DeepSpeed

  Public
  DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

  Python

  •

  Apache License 2.0

  •4.1k•0•0•0•Updated Sep 13, 2024Sep 13, 2024

• opencompass

  Public
  OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

  Python

  •

  Apache License 2.0

  •446•1•0•0•Updated Sep 6, 2024Sep 6, 2024

• xtuner

  Public
  An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

  Python

  •

  Apache License 2.0

  •314•0•0•0•Updated Aug 22, 2024Aug 22, 2024

• InternVL

  Public
  [CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的可商用开源多模态对话模型

  Python

  •

  MIT License

  •475•0•0•0•Updated Aug 16, 2024Aug 16, 2024

• OmniBal

  Public
  Python

  •0•16•2•0•Updated Aug 9, 2024Aug 9, 2024

• TFMQ-DM

  Public
  [CVPR 2024 Highlight] This is the official PyTorch implementation of "TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models".

  highlightquantizationcvpr+ 6ldmdiffusion-modelspost-training-quantizationddimstable-diffusioncvpr2024

  Jupyter Notebook

  •

  Apache License 2.0

  •4•55•0•0•Updated Aug 1, 2024Aug 1, 2024

• L2_Compression

  Public
  Python

  •

  Apache License 2.0

  •0•11•1•0•Updated Jun 16, 2024Jun 16, 2024

• msbench

  Public
  A tool for model sparse based on torch.fx

  Python

  •

  Apache License 2.0

  •1•7•0•0•Updated Jun 3, 2024Jun 3, 2024

• MQBench

  Public
  Model Quantization Benchmark

  Shell

  •

  Apache License 2.0

  •140•766•5•5•Updated Jun 3, 2024Jun 3, 2024

• FCPTS

  Public template
  Python

  •0•2•0•0•Updated May 14, 2024May 14, 2024

• statecs

  Public
  Rust

  •

  Apache License 2.0

  •1•1•0•0•Updated May 10, 2024May 10, 2024

• greedy-tokenizer

  Public
  Greedily tokenize strings with the longest tokens iteratively.

  Python

  •

  Apache License 2.0

  •0•0•0•0•Updated Mar 27, 2024Mar 27, 2024

• QLLM

  Public
  [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models"

  transformerspytorchllama+ 4quantizationpost-training-quantizationllmllama2

  Python

  •

  Apache License 2.0

  •3•36•1•0•Updated Mar 11, 2024Mar 11, 2024

• Dipoorlet

  Public
  Offline Quantization Tools for Deploy.

  Python

  •

  Apache License 2.0

  •16•116•11•2•Updated Dec 28, 2023Dec 28, 2023

• awesome-lm-system

  Public
  Summary of system papers/frameworks/codes/tools on training or serving large model

  Apache License 2.0

  •5•56•0•0•Updated Dec 17, 2023Dec 17, 2023

• LPCV_2023_solution

  Public
  Python

  •2•18•0•0•Updated Nov 29, 2023Nov 29, 2023

• Outlier_Suppression_Plus

  Public
  Official implementation of the EMNLP23 paper: Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling

  Python

  •

  MIT License

  •3•42•0•0•Updated Oct 21, 2023Oct 21, 2023

• UP_LPCV2023_Plugin

  Public
  Python

  •1•0•0•0•Updated Aug 11, 2023Aug 11, 2023

• ChatGLM-6B

  Public
  ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型

  Python

  •

  Apache License 2.0

  •5.2k•2•0•0•Updated Jun 20, 2023Jun 20, 2023

• pyvlova

  Public
  Yet another Polyhedra Compiler for DeepLearning

  Python

  •

  Apache License 2.0

  •4•19•0•0•Updated Apr 14, 2023Apr 14, 2023

• systemnoise_web

  Public
  HTML

  •0•0•0•0•Updated Apr 3, 2023Apr 3, 2023

• NART

  Public
  NART = NART is not A RunTime, a deep learning inference framework.

  Python

  •

  Apache License 2.0

  •14•38•1•0•Updated Mar 2, 2023Mar 2, 2023

• United-Perception

  Public
  United Perception

  Python

  •

  Apache License 2.0

  •65•429•27•1•Updated Dec 5, 2022Dec 5, 2022

• AAAI2023_EAMPD

  Public
  AAAI2023 Efficient and Accurate Models towards Practical Deep Learning Baseline

  0•13•2•0•Updated Nov 29, 2022Nov 29, 2022