On August 6th, it was reported that video generation has become one of the most vibrant sectors in the AI industry this year. Following the global attention garnered by Seedance 2.0, just last week, MiniMax H3 and Seedance 2.5 were launched in succession. Chinese players are leading the world in text-to-video and image-to-video generation scenarios, while the application of video generation technology in film, short dramas, e-commerce, and other scenarios is accelerating.
But behind the excitement, the costs are becoming increasingly difficult to calculate: the inference costs of video generation remain high, the real-time demand for multi-channel video understanding is still expanding, and general-purpose GPUs, as the main computing power, have long suffered from objective problems such as wasted computing power, high power consumption, and high latency in video scenarios.
The supply-demand gap is opening a window for specialized chips.Domestic and international internet giants are investing in the development of their own video chips, and new startups are emerging one after another. Among them, dataflow architecture chips have become a common focus for many manufacturers, and many chip companies have begun to adopt this approach in their new generation of products.
In the first half of this year, Zhongke Tongliang, one of the earliest domestic companies to industrialize data flow architecture,King Kong GC3, a new generation data stream architecture video AIGC chip, was officially released.This chip boasts 12 RISC-V cores, 200 TOPS INT8 computing power, and 128GB LPDDR5 ECC unified memory, and is marketed as "born for video," catering to both video encoding/decoding and video generation needs. Recently, ChipThings interviewed...Wu Dongdong, CTO of Zhongke TongliangWe discussed the technical decisions and business considerations behind this chip.
01.
The computational power dilemma of video intelligence:
Why are there not enough general-purpose GPUs?
What kind of journey does a video go through in a GPU server? Wu Dongdong explained two typical scenarios to ChipThings.
First, watch videos to understand (such as security, quality inspection, and multi-channel content review).After a massive influx of video streams into the server, the CPU or a dedicated decoding unit first decodes the video, restoring the compressed bitstream to pixel frames. Then the GPU takes over, running AI inference tasks such as object detection and behavior recognition to extract structured information from the image. "At this point, the main load on the GPU is to 'understand' the image," Wu Dongdong said. "But the bottleneck is often not computing power, but the latency of data transfer when multiple streams are running concurrently."
Next, let's look at video generation (such as text-based videos and image-based videos).The process is exactly the reverse—the GPU first runs a diffusion model or Transformer to generate pixel content frame by frame, and then the encoder compresses it into a video file that can be transmitted or stored. "Generation is computationally intensive, and the more computing power the better, but the problem of data transfer is still unavoidable, especially since the intermediate parameters of large video models are often hundreds of gigabytes, and repeated data transfer is just repeated burning money."
He discovered that, whether in understanding or generation, video computation is stuck on the same problem:The cost of data migration is too high.The understanding is that in multi-path concurrency, each frame has to travel back and forth between memory and computing units, and the latency increases as the number of paths increases; the generation is that the scale of a single calculation is too large, and the model parameters and intermediate results are repeatedly read and written, causing bandwidth and power consumption to soar.
Wu Dongdong stated that when the workload is small, this waste is difficult to detect; however, once it enters an industrial-level, high-intensity, high-concurrency scenario,The cost of transportation will be amplified into delays and increased power consumption.
Latency and power consumption are merely symptoms; the underlying cause lies at the architectural level. Currently, mainstream CPUs and GPUs in the industry are based on the von Neumann architecture: instructions and data are stored in the same memory block, and execution relies on a pointer called a "program counter" to fetch the next instruction one by one—this is the so-called...Control Flow.
This design was created for general-purpose computing and it is indeed capable of doing so: program logic varies greatly and branch jumps are unpredictable. A central controller schedules instructions one by one, which is a relatively flexible solution. The success of CPUs and GPUs over the past few decades has been built on this paradigm.
However, video computing has the opposite characteristics: the computation pattern is highly regular, the data volume is extremely large, and parallelism is inherent, thus amplifying the three problems of the control flow architecture simultaneously:
First, the degree of parallelism is limited.Optimization techniques in control flow architectures, such as out-of-order execution and multiple issuance, can only select executable instructions within a limited instruction window. However, video generation involves massive parallel computation at the pixel and matrix levels, meaning that many parallel operations outside the window must wait, reducing computational efficiency.
Second, the overhead of accessing the memory is large.In a control flow architecture, instructions and data are crammed into the same memory block, and intermediate results need to be repeatedly accessed and retrieved between different levels of storage. For video generation, the model parameters and intermediate results are enormous, and repeated data transfer means wasting computing power on data. For encoding and decoding, the video stream is continuous, with each channel constantly moving in and out of memory, and the waste increases exponentially with the number of channels.
Third, the synchronization overhead is large.Multiple computing cores need to rely on barriers to align their progress. Video encoding and decoding have intra-frame and inter-frame dependencies, and when multiple streams are running concurrently, a lot of inter-core coordination is required. The more cores there are, the more time is spent waiting, which will affect real-time performance.
These bottlenecks translate into real costs in specific business operations. For video generation companies, pricing is based on tokens, and if inference costs cannot be reduced, the business will be difficult to sustain. In video understanding scenarios such as security, smart cities, and industrial quality inspection, tens of thousands of video streams need to be analyzed simultaneously, making it difficult for general-purpose architectures to achieve real-time performance. At the edge, parks and factories wanting to deploy AI computing power locally cannot avoid the hurdles of GPU server power consumption and heat dissipation.
The conclusion is clear:Video intelligence does not require stronger general-purpose computing power, but rather a dedicated architecture tailored to the way video data flows.
In Wu Dongdong's observation, the industry's solutions are converging. One approach is to integrate the CPU, VPU, GPU, and NPU onto a single chip, using unified memory to eliminate the need for data transfer between different components, thus saving time. Another, more radical approach is: "Data flow replacing control flow or instruction flow may also be a future trend."
02.
Dataflow + RISC-V
An architectural solution designed for video
The King Kong GC3, released this year by Zhongke Tongliang, is precisely such a chip. It uses a dataflow architecture as its computing engine, a RISC-V instruction set as its control core, and a unified memory design.The two main industry trends that Wu Dongdong observed have converged on this chip.
First lookDataflow ArchitectureCompared to the von Neumann architecture, its biggest change is...The program counter was cancelled.In control flow architecture, a program is a sequence of instructions arranged in order, with the execution order determined by their position. In dataflow architecture, however, the program is unfolded into a "dataflow graph": each computational operation is a node on the graph, and the dependencies between data are the connections between nodes. When an instruction is executed is no longer determined by its position in the graph, but only by whether all its input data is ready.
This can be understood using a factory assembly line: each workstation (computing unit) starts automatically as soon as all the parts (data) arrive, without waiting for orders from the central command. Once completed, the data is directly passed to the next workstation. The workstation whose parts arrive first starts operating first. The entire assembly line has no overall scheduling, yet it almost never idles.
"Our core philosophy is that once the data is ready, I will perform the calculations."Wu Dongdong said.
It's worth noting that this is also the core idea adopted by the LPU (Language Processing Unit), which has recently garnered much attention in the AI chip field. The rise of the LPU precisely confirms the natural advantage of dataflow architecture in dealing with high-density, low-latency AI inference scenarios—as the industry's demand for token generation efficiency approaches its limit, dataflow is transforming from an academic approach into an industry consensus.
In video generation scenarios, this architecture offers many direct benefits. All input-ready operations can be executed simultaneously, eliminating the massive pixel-level and matrix-level computations required to generate a video from being blocked by the command window.The utilization rate of computing power will increase accordingly.Furthermore, intermediate computation results can flow directly between computation nodes without repeatedly loading and unloading memory. Video generation, on the other hand, involves a massive workload of model parameters and intermediate results; reducing memory movement significantly reduces this burden.Bandwidth pressure and power consumption will also decrease.
In terms of memory design, the GC3 is equipped with 128GB of LPDDR5 ECC unified memory, enabling video generation to run on a single chip. Large model parameters and intermediate activations that previously required cross-card communication can now be entirely contained within the unified address space of a single chip. Large model parameters commonly found in video generation can easily reach hundreds of gigabytes in size. If video memory is insufficient, frequent data transfer or time-sharing between multiple cards is necessary, each transfer resulting in latency and power consumption. The GC3, however, stores the entire model on a single chip, eliminating the need for copying data between different computing units; addresses are directly passed.In Wu Dongdong's words, "The less you move, the cheaper the tokens become."
Furthermore, since the execution order of instructions is naturally determined by the dependencies between data,When running large model inference with multi-core collaboration, there is no need to align the progress with global barriers.The time spent waiting is minimized. At the same time, complex control flow architecture designs such as branch prediction and register renaming are eliminated.The logic on the film is simplerHigher performance can be achieved with less area and power consumption.
Another major feature of GC3 isRISC-V instruction set usedThis open, free, and customizable instruction set allows chips to achieve independent control at the instruction set level, which also means that software and hardware co-optimization can be performed for video intelligence scenarios.
An example of this hardware and software collaboration is...ConfigurableAlthough both video understanding and video generation involve video, their underlying workloads differ significantly: understanding is computationally intensive (decoding heavy, computationally light, high concurrency), requiring a single chip to simultaneously process multiple high-definition streams; generation, on the other hand, is computationally intensive (fewer streams, high bandwidth), with massive AI computational demands for a single 4K or 8K stream. GC3's solution involves allowing components to deform according to the scene.With 128-channel 1080P decoding capability, it can be reassembled into 8K, 120fps video encoding in generated scenarios.
This capability is supported by 12 RISC-V cores. They act as...control planeIt is responsible for configuring and orchestrating the various computing components on the chip. "Actually, each component is configurable; the only thing that can be changed is how we control it," Wu Dongdong explained. "These cores act as the glue, scheduling to ensure that all components operate in a way that best matches the business flow."The efficiency can reach 80% to 90%..
This design also addresses a common concern in the industry. Currently, models are iterated every three months, and chips with fixed operators may become obsolete as soon as they are designed. GC3's approach is a combination of hardware and software: the algorithm changes, but 70-80% of the underlying operators remain the same. Heavy workloads such as matrix multiplication are fixed to efficient NPU units, while the variable parts are handled by compilation tools and CPU scheduling. This architecture's flexibility allows for adaptability to model iterations.
03.
Open source, adaptation, and implementation:
GC3 Commercialization Roadmap
For a chip like the GC3, which employs a novel computing architecture, ecosystem development is also a crucial aspect. The dataflow architecture faces the dilemma of high theoretical performance but an underdeveloped ecosystem.
Zhongke Tongliang's solution is long-term investment and multi-pronged approach. In terms of toolchain, it has already provided the necessary development tools such as compilers, and is still continuously improving them; in terms of open source, Zhongke Tongliang has open-sourced its software stack and is also building its own developer community.
Furthermore, Zhongke Tongliang, as a platinum donor to the OpenAtom Open Source Foundation, contributed money, effort, and code. This investment has already yielded direct returns: domestically developed operating systems such as Kylin, Euler, and HarmonyOS can now be directly installed on new chips."We won't face the predicament of having chips but no operating system available.".
In terms of commercialization, GC3 adopts a full-scale deployment across cloud, edge, and device. In the cloud, it builds high-density intelligent computing centers, providing cost-effective video token services to operators and platform companies, and also constructing private token service platforms for large platforms and industrial parks. At the edge, it utilizes AI PCs and the Xiaojingang intelligent computing box. Xiaojingang prioritizes research scenarios, as these users are more open to new products and willing to try them; their feedback drives product improvement.
04.
Conclusion: Video intelligence hits the ceiling of general-purpose architecture
Data stream chips are betting on the next path.
Video represents the largest data workload in the AI era and is also one of the first to hit the limitations of general-purpose architectures. From dedicated codec chips to video AIGC chips, the cost-effectiveness of specialized chips is continuously improving. Industry research also points in the same direction; McKinsey predicts that by 2030, AI accelerators equipped with dedicated ASIC chips will handle the majority of AI workloads.
Of course, the replacement won't happen overnight. Wu Dongdong's assessment of the pace is not aggressive: "GPUs won't be replaced immediately either. They have a large user base, high computing power, and a good ecosystem. For individual users or when there isn't much concurrency, they are a relatively easy way to get started." However, for scenarios like the industrial internet and telecom operators that face a large number of high-concurrency task requests, the replacement of general-purpose GPUs is already underway.
Speaking of the future, Wu Dongdong said, "Video is still our core business." But his vision extends beyond video: with the improvement of chip capabilities, language models and multimodal applications have also entered the chip's capability radius. "Images and language together are the way humans understand everything, and we can reach all of these."