Baidu Wang Haifeng: ERNIE Bot will gradually open the plug-in ecology

On July 6, the 2023 World Artificial Intelligence Conference (WAIC) opened at the Shanghai World Expo Center. During the conference, Wang Haifeng, chief technology officer of Baidu and director of the National Engineering Research Center for Deep Learning Technology and Application, interpreted the core technology of Wenxin Model Version 3

On July 6, the 2023 World Artificial Intelligence Conference (WAIC) opened at the Shanghai World Expo Center. During the conference, Wang Haifeng, chief technology officer of Baidu and director of the National Engineering Research Center for Deep Learning Technology and Application, interpreted the core technology of Wenxin Model Version 3.5, released the latest development of fly propeller ecology, expounded the AI industry model, and spoke for the latest AI technology and industry.

Flying Paddle has gathered 7.5 million developers, with a 50% increase in Wenxin 3.5 effect and a 30 times increase in reasoning speed

Currently, artificial intelligence technology represented by large language models has sparked a wave of technological and industrial innovation worldwide, accelerating industrial upgrading and economic growth, and causing significant changes in various industries. The IT technology stack has undergone a fundamental change, shifting from a three tier architecture of chips, operating systems, and applications to a four tier architecture of chips, frameworks, models, and applications. The deep learning framework and large model form the foundation of industrial intelligence, which will support the intelligent reconstruction of applications in various industries and promote high-quality economic development.

It is understood that Baidu has a layout and leading self-developed technology in the four layer artificial intelligence technology stack, especially in the framework layer and model layer located at the core of the four layer architecture. The latest achievements of the Wenxin Big Model also benefit from the joint optimization of the Flying Paddle Platform and Wenxin. Fly Propeller is the first industrial level deep learning open source Open platform independently developed in China. Wang Haifeng revealed on site that as of now, Feifan has attracted 7.5 million developers, which is also the first time Baidu has disclosed the latest data on Feifan ecology to the public since 2023.

Since the release of version 1.0 of the Wenxin Big Model in March 2019, Baidu has undergone four years of deep technical cultivation and research and development iterations, and has now upgraded to Wenxin Big Model 3.5. Wang Haifeng stated that the Wenxin Big Model 3.5 has comprehensively improved its effectiveness, functionality, and performance, achieving basic model upgrades, fine tuning technology innovation, knowledge point enhancement, and logical reasoning enhancement. The model effect has been improved by 50%, the training speed has been increased by 2 times, and the reasoning speed has been increased by 30 times.

Continuous breakthroughs in core technologies, resulting in a leap in effectiveness and efficiency

In March this year, Baidu was the first major technology company in the world to release the big language model ERNIE Bot. ERNIE Bot is a big language model for knowledge enhancement. First, it fuses and learns from trillions of data and hundreds of billions of knowledge to obtain a large model for pre training. On this basis, it uses Reinforcement learning and tips with supervision and fine tuning, human feedback and other technologies, and has technical advantages such as knowledge enhancement, retrieval enhancement and dialogue enhancement.

Wang Haifeng interpreted the core technological innovation of Wenxin Big Model 3.5. In terms of basic model training, he adopted the most advanced adaptive hybrid parallel training technology and hybrid accuracy calculation strategy of the flying propeller, and used multiple strategies to optimize data sources and distribution, accelerating model iteration speed, significantly improving model effectiveness and security. At the same time, it has innovated technologies such as multi type and multi-stage supervised fine tuning, multi-level and multi granularity reward model, multi Loss function hybrid optimization strategy, model optimization combined with two flying wheels, and further improved the model effect and scene adaptation capability.

On the basis of knowledge enhancement and retrieval enhancement, Wenxin Big Model 3.5 proposes "knowledge point enhancement technology" to analyze and understand the queries and questions entered by users, analyze the relevant knowledge points needed to generate answers, and then use the Knowledge graph and search engine to find the corresponding answers for these knowledge points. Finally, use these knowledge points to construct input tips to the big model to inject more specific, detailed More professional knowledge points significantly enhance the mastery and application of world knowledge in large models.

In terms of reasoning, through large-scale logical data construction, logical knowledge modeling, multi granularity semantic knowledge combination, and symbolic neural network technology, the performance of Wenxin Big Model 3.5 in tasks such as logical reasoning, mathematical calculation, and code generation is improved.

Add plugin mechanism to expand the capability boundary of large models

ERNIE Bot has released the official plug-ins Baidu Search and ChatFile on June 17. Baidu Search is the default built-in plug-in, which enables ERNIE Bot to generate real-time and accurate information. ChatFile is a long text summary and Q&A plugin that supports ultra long text input.

Wang Haifeng said that ERNIE Bot will release more high-quality Baidu official and third-party plug-ins, so that users can better apply Wenxin's big model. At the same time, it will gradually open the plug-in ecosystem to help developers build their own applications based on Wenxin's big model.

Widely used in various scenarios to accelerate industrial intelligence upgrading

Wang Haifeng showed the application of ERNIE Bot in office, meeting, coding and other scenarios on the spot. ERNIE Bot became a "super assistant" in the work, helping to summarize work communication points, record meeting content in real time, form key information such as meeting topics, summaries and summaries, and complete instruction tasks through various plug-ins, including querying the agenda, creating meetings, setting to-do lists, and applying for leave, It can also automatically recommend and generate code during the engineer's coding process. It is reported that these functions have been applied to Baidu's workflow through the intelligent work platform "Ruliu", helping to improve work efficiency and decision-making quality.

Wang Haifeng said that any application scenarios that need to deal with language or program code may have the potential of ERNIE Bot. There have been many scenarios where ERNIE Bot has been actively applied, such as energy, finance, education, office, media, etc. In the process of implementing large model industries such as ERNIE Bot, the mode of "intensive production and platform application" can be adopted, that is, enterprises with comprehensive advantages in algorithm, computing power and data can encapsulate the complex process of model production and provide large model services for thousands of industries through a low threshold and efficient production platform.

Disclaimer: The content of this article is sourced from the internet. The copyright of the text, images, and other materials belongs to the original author. The platform reprints the materials for the purpose of conveying more information. The content of the article is for reference and learning only, and should not be used for commercial purposes. If it infringes on your legitimate rights and interests, please contact us promptly and we will handle it as soon as possible! We respect copyright and are committed to protecting it. Thank you for sharing.(Email:[email protected])