LLaMA-Factory
Easy-to-use LLM fine-tuning framework (LLaMA, BLOOM, Mistral, Baichuan, Qwen, ChatGLM)
Fork A fork of hiyouga/LlamaFactory; the README may describe the upstream project.
Overview
👋 Join our WeChat or NPU user group.
Fine-tuning a large language model can be easy as...
https://github.com/user-attachments/assets/7c96b465-9df7-45f4-8053-bf03e58386d3
[!NOTE] Except for the above links, all other websites are unauthorized third-party websites. Please carefully use them.
Compared to ChatGLM's P-Tuning, LLaMA Factory's LoRA tuning offers up to 3.7 times faster training speed with a better Rouge score on the advertising text generation task. By leveraging 4-bit quantization technique, LLaMA Factory's QLoRA further improves the efficiency regarding the GPU memory.
[24/12/21] We supported SwanLab experiment tracking and visualization. See this section for details.
[24/11/27] We supported fine-tuning the Skywork-o1 model and the OpenO1 dataset.
[24/10/09] We supported downloading pre-trained models and datasets from the Modelers Hub. See this tutorial for usage.
From the project’s README on GitHub.
At a glance
| Repository | Kentucky-Open-Science/LLaMA-Factory |
|---|---|
| Research area | Forks of other projects |
| Primary language | Python |
| Languages | Python 97.1%, Shell 2.3%, Dockerfile 0.5% |
| License | Apache-2.0 |
| Stars / forks | 1 / 1 |
| Open issues and pull requests | 7 |
| Created | 2023-12-19 |
| Last push | 2026-01-14 |
| Default branch | main |
| Forked from | hiyouga/LlamaFactory — Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024) |
What the README covers
- Table of Contents
- Features
- Benchmark
- Changelog
- Supported Models
- Supported Training Approaches
- Provided Datasets
- Requirement
- Hardware Requirement
- Getting Started
- Installation
- Data Preparation
- Quickstart
- Fine-Tuning with LLaMA Board GUI (powered by Gradio)
- Build Docker
- Deploy with OpenAI-style API and vLLM
- Download from ModelScope Hub
- Download from Modelers Hub
- Use W&B Logger
- Use SwanLab Logger
- Projects using LLaMA Factory
- License
- Citation
- Acknowledgement
- Star History
Top contributors
- @hiyouga (1986 commits)
- @codybum (582 commits)
- @BUAADreamer (96 commits)
- @codemayq (58 commits)
- @marko1616 (23 commits)
- @Kuangdd01 (20 commits)
- @wangxingjun778 (12 commits)
- @MengqingCao (11 commits)
- @hzhaoy (11 commits)
- @johannhartmann (10 commits)
Get the code
git clone https://github.com/Kentucky-Open-Science/LLaMA-Factory.git