TrajeQ: Trajectory-Preserving Flow Matching Quantization for Low-Bit Video Generation

1Anonymous

TL;DR

TrajeQ preserves trajectory stability in flow-matching video generation under low-bit quantization through direction-aware precision allocation and selective layer preservation.

Abstract

While flow matching (FM) has emerged as a prominent paradigm for high-quality video generation, its significant computational overhead presents a major obstacle to practical implementation. Although post-training quantization (PTQ) is an established approach for efficient generation, existing PTQ methods primarily minimize reconstruction error in a magnitude-centric manner. This objective is misaligned with deterministic FM trajectories, where low-bit quantization introduces velocity-field orientation errors that can compound across integration steps and cause deviations from the full-precision trajectory. To address this issue, we introduce TrajeQ to preserve the geometric integrity of the velocity field and reduce trajectory drift under low-bit quantization. Our approach introduces Curvature-Adaptive Quantization (CAQ) to detect angular shifts across timesteps and mitigate geometric distortion via adaptive precision. We also integrate Directional Bottleneck Preservation (DBP), a sensitivity-guided precision assignment scheme that retains only the most vulnerable layers in full precision. Experiments demonstrate that TrajeQ achieves state-of-the-art VBench performance and maintains stable video generation under extreme W4A4 and W3A6 quantization regimes where existing methods degrade severely. We provide the code in the supplementary material and additional video results at https://anonymous-orange.github.io/.

Figure 1: TrajeQ Overview

Generation Quality on VBench

FP16

TrajeQ W6A6

Extreme Compression Performance (W6A6 vs W3A6)

FP16

TrajeQ W6A6

TrajeQ W3A6

Method

TrajeQ is the first quantization framework specifically engineered to preserve the geometric trajectory of Flow Matching models. In contrast to conventional approaches that minimize magnitude error and often lead to significant trajectory drift, TrajeQ prioritizes the integrity of vector field directionality. Our framework integrates two complementary mechanisms: Curvature-Adaptive Quantization (CAQ), which dynamically adapts precision using inter-step changes in activation direction, and Directional Bottleneck Preservation (DBP), which identifies and safeguards structural bottlenecks. This synergistic approach enables TrajeQ to achieve high-fidelity video generation without structural collapse.

Figure 1: TrajeQ Overview

Curvature-Adaptive Quantization
(CAQ)

Motivation

Activation directions can change sharply during flow-matching sampling. Since full-precision reference velocities are unavailable during quantized inference, CAQ uses inter-step activation changes as a proxy to guide precision allocation and reduce trajectory drift.

Key Mechanism
  • Timestep Selection: At each quantized layer, CAQ computes the cosine distance between the current flattened activation and its cached value from the preceding timestep. This directional-shift score identifies timesteps for additional precision.
  • Adaptive Precision: When the score exceeds the threshold of 0.25, CAQ increases the activation bit-width by one. Precision is selected independently for each layer and timestep and applies uniformly to the entire activation tensor.
Impact

CAQ directs additional precision toward sharp activation-direction changes. In our ablations, this allocation improves generation quality over random boosting with the same number of boosted timesteps, supporting directional shifts as a useful selection signal.

Directional Bottleneck Preservation
(DBP)

Motivation

MSE can underemphasize angular distortion in layers with small activation norms. Our analysis shows that the same small orthogonal perturbation can cause larger angular deviations at smaller norms, motivating sensitivity-guided layer preservation.

Key Mechanism
  • Layer Sensitivity Analysis: Before inference, DBP profiles full-precision and quantized activations across prompts and timesteps. Directional Sensitivity (DS) scores each layer using the ratio of mean angular error to mean squared error, with a stabilized denominator.
  • Strategic Preservation: DBP ranks quantizable layers by DS and preserves the highest-sensitivity subset in FP16 under a fixed parameter budget. Selected layers concentrate in embedding and normalization modules, rather than being chosen by manually specified types.
Impact

In our analysis, preserved layers account for less than 1% of total parameters. This targeted allocation suppresses quantization-induced distortion in vulnerable components while enabling low-bit quantization elsewhere with minimal computational overhead.

CAQ Visualization
DBP Visualization

Experimental Results

Table 1: VBench Performance Comparison

Table 1: VBench Performance Comparison

Quantitative evaluation on VBench demonstrates TrajeQ's superior performance across various quantization schemes. At W6A6, TrajeQ achieves near-lossless quality comparable to FP16, while maintaining robust structural integrity even under the extreme W3A6 regime.

Table 2: OpenSora Performance Analysis

Table 2: OpenSora Performance Analysis

Performance comparison on OpenSora prompts. TrajeQ achieves SOTA results across all bit-widths, delivering superior visual fidelity and motion stability compared to existing quantization methods.

Figure 6: Visual Ablation of Structural Bottlenecks

Figure 6: W4A6 visual comparison of quantized and FP16 embedding and normalization layers

Visual ablation under W4A6 quantization compares quantizing both embedding and normalization layers, quantizing either group alone, and preserving both in FP16. Quantizing these components introduces color artifacts or texture degradation in this example. Preserving both in FP16 yields higher visual fidelity, illustrating the benefit of protecting structural bottlenecks.

Quantization Quality Comparison (VBench)

MinMax W6A6
Volumetric Artifacts

ViDiT-Q W6A6
Physics Inconsistency

TrajeQ W6A6

MinMax W6A6
Object Missing

ViDiT-Q W6A6
Structural Distortion

TrajeQ W6A6

MinMax W6A6
Temporal Warping

ViDiT-Q W6A6
Scale Inconsistency

TrajeQ W6A6

MinMax W4A6
Perspective Distortion

ViDiT-Q W4A6
Structural Deformation

TrajeQ W4A6

MinMax W4A6
contrast collapse

ViDiT-Q W4A6
chromatic deviation

TrajeQ W4A6

Extreme Quantization: W4A4 Comparison

TrajeQ maintains visual coherence and structural integrity even at the extreme W4A4 quantization regime, where standard methods completely fail to preserve meaningful content.

"Iron Man flying in the sky"

"River"

MinMax

"Iron Man flying in the sky"

"River"

SmoothQuant

"Iron Man flying in the sky"

"River"

QuaRot

"Iron Man flying in the sky"

"River"

Q-DiT

"Iron Man flying in the sky"

"River"

ViDiT-Q

"Iron Man flying in the sky"

"River"

TrajeQ (Ours)

Future Work

While TrajeQ analyzes the geometric trajectory of velocity vectors, future work will investigate how quantization alters internal feature representations, especially whether it disrupts the geometry of superposed features.