accelerate 命令

常用示例

说明

accelerate is a Hugging Face library that enables PyTorch code to run on any distributed configuration with minimal code changes. It handles the complexity of distributed training across multiple GPUs, TPUs, and machines while keeping your training code simple. The tool abstracts away the boilerplate needed for mixed precision training, gradient accumulation, and multi-device parallelism. It automatically detects available hardware and configures the training environment appropriately.

参数

config

Run the configuration wizard to set up your environment

launch

Launch a training script with the configured settings

--num_processes _n_

Total number of processes to launch

--gpu_ids _ids_

Comma-separated GPU IDs to use

--mixed_precision _type_

Enable mixed precision: no, fp16, bf16

--num_machines _n_

Number of machines for distributed training

--machine_rank _n_

Rank of the current machine (0-indexed)

--main_process_ip _ip_

IP address of the main machine

--main_process_port _port_

Port for the main machine (default: 29500)

--use_deepspeed

Enable DeepSpeed for training

--use_fsdp

Enable Fully Sharded Data Parallel

test

Test your accelerate configuration

env

Print environment information