Zhiyuan Research Institute Huang Tiejun: The Visual Model is on the eve of the "Explosion"

Global Network Technology Reporter Zheng XiangqiCompared to the language model, the visual model is still on the eve of its explosion and requires a 'killer' application to emerge. Recently, during the 2023 Beijing Zhiyuan Conference (hereinafter referred to as the "Conference"), Huang Tiejun, the director of Zhiyuan Research Institute, said in an interview with reporters

Global Network Technology Reporter Zheng XiangqiCompared to the language model, the visual model is still on the eve of its explosion and requires a 'killer' application to emerge. Recently, during the 2023 Beijing Zhiyuan Conference (hereinafter referred to as the "Conference"), Huang Tiejun, the director of Zhiyuan Research Institute, said in an interview with reporters.

Currently, new research and products related to large models are emerging in a competitive manner. According to the "Research Report on China's Artificial Intelligence Large Model Map" (hereinafter referred to as the "Report"), according to incomplete statistics, as of now, 79 large models with parameters of over 1 billion have been released nationwide.

In Huang Tiejun's view, a large model needs to have three conditions: firstly, it must be large in scale, with parameters that can even reach a scale of over ten billion yuan; The second is emergence, which can generate unexpected new abilities; The third is universality, which is not limited to specialized problems or fields, and can handle various different tasks.

As early as October 2020, the Zhiyuan Research Institute began exploring the path of the large-scale pre training model "Wudao" project. Subsequently, in March 2021, as China's first large-scale pre training model, "Wudao 1.0" was released; In June of the same year, the Zhiyuan Research Institute once again released "Wudao 2.0".

Taking Wudao Tianying as an example, Huang Tiejun introduced that as the first open-source language big model with bilingual knowledge in Chinese and English, supporting commercial license agreements, and domestic data compliance requirements, it started training from "0" on the basis of high-quality Chinese and English corpus. Through data quality control and multiple training optimization methods, it achieved better performance in smaller datasets and shorter training times.

Large model development entering the "fast lane"

Recalling the iterative process of "Wudao", Huang Tiejun once said, "The development of artificial intelligence has gradually moved from 'refining models' to the stage of' refining big models'. The industry has designed advanced algorithms to integrate as much data as possible, gather a large amount of computing power, and intensively train big models for use by a large number of enterprises, which is an inevitable trend

Obviously, the prediction of the large model by the Zhiyuan Research Institute has been incorporated into reality. The report shows that the current large-scale model in China is showing a vigorous development trend. A group of universal large-scale models are rapidly developing, and the application industry is accelerating its expansion from office, life, entertainment and other fields to medical, industrial, education and other fields.

Huang Tiejun told the reporter that the reason why so many big models are coming out is the emergence of new learning methods, the most important of which is self Supervised learning. It is reported that the advantage of self Supervised learning is that training can be completed on unlabeled data, while Supervised learning requires labeled data, and the labeling of data is inseparable from human costs.

"Small data can't be trained into a big model, but based on self Supervised learning, data is no longer limited to cost, manpower and other factors. As long as the data has results, the model can intelligently extract hidden rules from it through self Supervised learning, and then solve the corresponding problems," said Huang Tiejun.

In terms of categories, Huang Tiejun believes that compared to the language big models that have already sparked a wave around the world, the visual big models are still on the eve of the outbreak, and there are still many problems to be solved. When a "killer" application appears, it can stimulate the ability behind visual big models and everyone's enthusiasm for visual big models.

The industry should "concentrate on doing big things"

It is worth noting that the large model that has already entered the "fast lane" still faces development constraints. Huang Tiejun admitted, "The 'big' of big models now is far from reaching the level of ceiling and comprehensiveness. From the perspective of language alone, it may take about three years to achieve comprehensiveness. In the next three years, the scale of big models will continue to increase and their capabilities will become stronger, which should be the basic trend

Regarding this, Huang Tiejun also suggests that the industry should form a collaborative force in the field of large model research and expand the ecosystem. I think repetitive efforts can actually dissipate resources. We always talk about concentrating our efforts on big things, and whether the industry can leverage their respective strengths in big models, and do their best in their areas of expertise. Then, we can connect these strongest links together to organically form an ecosystem, which is the direction we should strive for

In terms of open source ecology, Zhiyuan Research Institute has also made a series of efforts. For example, the FlagOpen open source system for large model technology released earlier this year has solidified the underlying technology stack for the development of large models. Based on FlagOpen, Zhiyuan Research Institute hopes to create an open-source algorithm system and a one-stop basic software platform that fully supports the development of large model technology, and jointly build a "new Linux" open-source open ecosystem in the era of shared large models with the industry.

In terms of datasets, Zhiyuan has opened up the first large-scale and commercially available Chinese instruction dataset, COIG. It is reported that COIG Phase I has opened a total of 191000 instruction data, and COIG Phase II is currently constructing the largest and continuously updated Chinese multitasking instruction dataset. It integrates over 1800 massive open source datasets, manually rewrites 390 million instruction data, and provides comprehensive data filtering and version control tools.

When it comes to the impact of big models on people's lives, Huang Tiejun said that on the one hand, as a technological tool, artificial intelligence has replaced many tasks that only humans could complete, bringing efficiency improvement and cost reduction to enterprises. On the other hand, some professions may face challenges as a result, such as repetitive tasks that can be achieved at lower costs through AI. However, new opportunities may also arise, and those who are impacted can find new jobs that better utilize their abilities.

I think this is a normal state of technological development. On the one hand, it will bring about a 'honeymoon period', and on the other hand, it will also cause some pain. However, I believe that human-machine integration will have good development in the next decade or two, "Huang Tiejun said.

Disclaimer: The content of this article is sourced from the internet. The copyright of the text, images, and other materials belongs to the original author. The platform reprints the materials for the purpose of conveying more information. The content of the article is for reference and learning only, and should not be used for commercial purposes. If it infringes on your legitimate rights and interests, please contact us promptly and we will handle it as soon as possible! We respect copyright and are committed to protecting it. Thank you for sharing.(Email:[email protected])