The Mengzi GPT Model is a large-scale language model developed with Langboat self-developed technology, which undergoes pre training, SFT, and alignment steps. It can handle multilingual and multimodal data, while supporting multiple text comprehension and generation tasks, meeting the needs of different fields and application scenarios. The Mengzi models, based on the Transformer architecture, contains parameters from 1B，10B to 100B of parameters. They were trained with tens of trilions of tokens of high-quality corpus covering numourous internet web pages, communities, news, books, e-commerce websites, finance websites and other sources . Mengzi is a well-known domestic model brand that has achieved excellent results in benchmark evaluations of Chinese LLMs, such as C-EVAL and SUPERCLUE. The Mengzi model has been registered with the China Cyberspace Administration for Generative Artificial Intelligence by the end of 2023, and it has officially opened to the public for general service.

In addtion to GPT, we also developed LLMs based on BERT、T5 architectures，which have been widely applied in our information extraction and macine translation products.