Swift DOCUMENTATION

Get Started

• SWIFT Installation
  • Wheel Packages Installation
  • Source Code Installation
  • Mirror
  • Supported Hardware
  • Running Environment
  • Notebook Environment
• Quick Start
  • Installation
  • Usage Example
  • Learn More
• Web-UI

Instruction

• Command Line Parameters
  • Base Arguments
  • Atomic Arguments
  • Integration Arguments
  • Specific Model Arguments
  • Other Environment Variables
• Pre-training and Fine-tuning
  • Environment Preparation
  • Pre-training
  • Fine-tuning
  • Merge LoRA
  • Inference (Fine-Tuned Model)
  • Deployment (Fine-Tuned Model)
• GRPO
  • Get Started
  • Developer Guide
  • Advanced Research
• GKD
  • Loss Function
  • Divergence Metrics
  • Three Training Modes
  • Parameter Settings
  • Sampling Acceleration
  • On-Policy Distillation
  • OPSD (On-Policy Self-Distillation)
• RLHF
  • Dataset
  • GRPO
  • DPO
  • RM
  • PPO
  • KTO
  • CPO
  • ORPO
  • SimPO
• Inference and Deployment
  • Inference
  • Deployment
• Sampling
  • Capability Introduction
  • Environment Setup
  • Using PRM and ORM for Result Filtering
  • Customizing PRM or ORM
  • Memory Control
  • Practical Example
  • Sampling From Large Model
• Evaluation
  • Capability Introduction
  • Environment Preparation
  • Evaluation
  • Evaluation During Training
  • Custom Evaluation Datasets
  • Question-and-Answer Format (QA)
• Export and Push
  • Merge LoRA
  • Quantization
  • Push Model
• Ray Support
  • Technical Details
  • Parameter Settings
• Reinforced Fine-Tuning
  • Concept of Reinforced Fine-Tuning
  • When to Use Reinforced Fine-Tuning
  • SWIFT Implementation
  • Experimental Results
• Agent Support
  • Dataset Format
  • Tools Format
  • Usage of loss_scale
  • Training
  • Inference
  • Deployment
• Supported Models and Datasets
  • Models
  • Datasets
• Using Tuners
  • Interface List
• Frequently-asked-questions
  • Training
  • Inference
  • Export
  • Deployment
  • Evaluation

Megatron-SWIFT

• Quick Start
  • Environment Setup
  • Quick Start Example
  • Training Tips
  • Benchmark
  • Megatron-SWIFT Wechat Group
• Command Line Arguments
  • Megatron Parameters
  • Training Parameters
  • RLHF Parameters
  • Export Parameters
• LoRA Training
  • Traditional Method
  • Mcore-Bridge [Recommended]
• Multimodal Models
  • Dense Model
  • MoE Model
• Mcore Bridge
  • Seamless Training
  • megatron export and Conversion Accuracy Testing
  • Using Code
• Megatron GRPO
  • Parameter Comparison
• GKD
  • Feature Support
  • Parameters
  • Three Training Modes
  • Reference
• Ascend NPU
  • NPU Performance Data Collection
• NPU Accuracy Data Collection
  • Installing msprobe
  • Code Modification
  • Enable
• Custom Megatron Model
  • Download the Model
  • Register the Model
  • Test Accuracy
  • Submit a PR

Customization

• Architecture Introduction
  • Agent Template
  • Callbacks
  • Loss
  • Loss Scale
  • Metrics
  • Optimizers
  • Tuner Plugin
  • ORM
  • PRM
  • Introduction to Other Directory Structures
• Custom Model
  • Model Registration
• Custom Dataset
  • Standard Dataset Format
  • dataset_info.json
  • Dataset Registration

Best Practices

• Complete GRPO Experiment Process
  • Task and Dataset Definition
  • Reward Function Definition
  • GRPO Training Experiment Record
• Complete Multimodal GRPO Experiment Workflow
  • ClevrCount Task
  • Reward Function Definition
  • Geometric QA Task
  • Multimodal Open R1 Dataset Experiment
• Code Training with GRPO
  • Reward Functions
  • Training Script
• Qwen3 Best Practices
  • Inference
  • Training
  • Supervised Fine-Tuning (SFT)
  • Reinforcement Learning (RL)
  • Megatron-SWIFT
• Qwen3-VL Best Practices
  • Environment Setup
  • Inference
  • Training
• Qwen3.5 Best Practices
  • Environment Setup
  • Inference
  • Fine-tuning
  • Reinforcement Learning (RL)
• DeepSeek-V4 Training Support
  • Precision Alignment
  • LoRA Training
• Best Practices for Registering Multimodal Models
  • Environment Setup
  • Model Registration
  • Template Registration
  • Inference Alignment
  • Start Training
  • Submitting a PR
• Embedding Training
  • Loss
  • Dataset Format
  • Training
  • Inference
  • Advanced
• Reranker Training
  • Implementation Methods
  • Loss Function Types
  • Dataset Format
  • Training Scripts
  • Advanced
• Best Practices for Rapidly Training Vision-Language (VL) Models
  • Model Modification
  • Training
  • Inference / Deployment / Evaluation
• NPU Support
  • Support Scope at a Glance
  • Choose Your Usage Path
  • Environment Preparation
  • Quick Start: ModelScope Model + Dataset
  • Training
  • Model Saving, Merge LoRA, and Resume Training
  • Inference
  • Deployment
  • Evaluation
  • Release
  • FAQ
  • NPU WeChat Group
• Metax Support
  • 1. use swift with Metax
  • 2. Environment check
  • 3. run example
• AMD GPU Support
  • 1. Environment setup
  • 2. Run examples
  • Known issues
• More Best Practices

Indices and tables

• Index

• Module Index

• Search Page