OpenBMB - The “engine” for big model training

BMTrain

The “engine” for big model training. BMTrain performs efficient pre-training and tuning for big models.
Compared with toolkit such as DeepSpeed, BMTrain can save 90% on cost in the training process.

GitHub

Doc

Powerful Performance

BMTrain performs amazingly compared to popular frameworks.

Easy-to-use

In line with the usage habits of PyTorch, the threshold for getting started is lower, and the training speed can be completed with simple replacement..

Original Code Code after Replacement

Supported Models

Encoder

bert-base-cased bert-base-uncased bert-large-cased bert-large-uncased bert-base-chinese bert-base-multilingual-cased

Decoder

CPM-1(large) GPT-2(base) GPT-2(medium) GPT-2(large) GPT-2(XL) GPT-J(6B)

Encoder-Decoder

CPM-2(large) T5-small T5-base T5-large T5(3B) T5(11B)

Toolkits

BMTrain

BMCook

BMInf

OpenPrompt

OpenDelta

ModelCenter

Resources

General Model License

Community

Blogs

BM Course

GitHub

About OpenBMB

About

Paper