China Releases First National Standard for Batch-Stream Fusion in Big Data, Alibaba Cloud Leads Industry Development

China Releases First National Standard for Batch-Stream Fusion in Big Data, Alibaba Cloud Leads Industry DevelopmentRecently, the State Administration for Market Regulation and the State Administration for Standardization of China officially released the first national standard for batch-stream fusion in big data, GB/T44216-2024 "Information Technology Big Data Batch-Stream Fusion Computing Technical Requirements". The standard was drafted by Alibaba Cloud Computing Co

China Releases First National Standard for Batch-Stream Fusion in Big Data, Alibaba Cloud Leads Industry Development

Recently, the State Administration for Market Regulation and the State Administration for Standardization of China officially released the first national standard for batch-stream fusion in big data, GB/T44216-2024 "Information Technology Big Data Batch-Stream Fusion Computing Technical Requirements". The standard was drafted by Alibaba Cloud Computing Co., Ltd. and will take effect on February 1, 2025.

The standard defines batch-stream fusion technology clearly and specifies the key capabilities that batch-stream fusion systems need to possess. These capabilities include:

• Simultaneous support for real-time and offline computing models: Batch-stream fusion systems can flexibly handle various data processing needs, meeting the real-time and offline requirements of different scenarios.

• Unified resource management: Unifies the management of computing resources, improves resource utilization, and reduces operating costs.

• Unified computing framework: Provides a unified computing framework, simplifying development and maintenance, and improving efficiency.

• Unified API: Provides a unified application programming interface for easy user invocation, enhancing development efficiency.

• Unified SQL interface: Offers a unified SQL language interface for user convenience and reduced learning costs.

• Unified job management: Provides unified job management tools for easy user management and monitoring of jobs, improving efficiency.

• Unified permission management: Offers a unified permission management mechanism to ensure data security and system stability.

• Elastic expansion: Supports flexible system expansion to adapt to changes in data volume and computing needs.

Batch-stream fusion is a significant emerging technological trend in the big data field. With the development of the internet, enterprises' demand for real-time data computation is constantly increasing. Traditional batch computing paradigms struggle to meet the requirements in terms of latency, while stream computing, which processes data in real-time, can respond quickly to needs. Integrating stream computing with batch computing under a unified technical framework has become a necessary path for the industry.

Apache Flink, an open-source computing engine with batch-compatible stream computing, is a rising star in the field of big data computation. Flink allows users to perform stream processing or batch processing with a single codebase, saving significant development costs. Alibaba acquired the parent company of Apache Flink in 2019 and, combined with its own e-commerce scenario practices, invested heavily in research and development to optimize Apache Flink products, achieving enterprise-level application standards of high throughput, low latency, high availability, and high accuracy. Alibaba Cloud continues to contribute to the global open-source community, becoming a leading force in driving community and technological innovation.

Currently, Apache Flink has become the de facto standard for batch-stream fusion computing in the industry. Over a hundred companies, including Alibaba, Ant Group, ByteDance, Xiaomi, Kuaishou, Xiaohongshu, Bilibili, Didi, China Unicom, and others, are using Apache Flink for real-time computation.

In 2019, Alibaba Cloud initiated the national standard proposal and drafting process for batch-stream fusion computing technology, working collaboratively with various organizations in the industry to draft and refine it. The standard incorporates Alibaba Cloud's experience in building real-time computing Flink versions and rich industry practices to better promote the development of batch-stream fusion computing technology, guide the design, development, and deployment of domestic batch-stream fusion computing systems, and further facilitate the implementation of batch-stream fusion technology in various industries.

With the official implementation of the national standard, batch-stream fusion technology will enter a period of rapid development and be applied in more areas, bringing greater value to enterprises. As the lead drafting unit of this standard, Alibaba Cloud will continue to be committed to promoting the innovation and development of batch-stream fusion technology and providing the industry with more reliable and advanced big data solutions.

Disclaimer: The content of this article is sourced from the internet. The copyright of the text, images, and other materials belongs to the original author. The platform reprints the materials for the purpose of conveying more information. The content of the article is for reference and learning only, and should not be used for commercial purposes. If it infringes on your legitimate rights and interests, please contact us promptly and we will handle it as soon as possible! We respect copyright and are committed to protecting it. Thank you for sharing.(Email:[email protected])