Motivation
Activation directions can change sharply during flow-matching sampling. Since full-precision reference velocities are unavailable during quantized inference, CAQ uses inter-step activation changes as a proxy to guide precision allocation and reduce trajectory drift.
Key Mechanism
- Timestep Selection: At each quantized layer, CAQ computes the cosine distance between the current flattened activation and its cached value from the preceding timestep. This directional-shift score identifies timesteps for additional precision.
- Adaptive Precision: When the score exceeds the threshold of 0.25, CAQ increases the activation bit-width by one. Precision is selected independently for each layer and timestep and applies uniformly to the entire activation tensor.
Impact
CAQ directs additional precision toward sharp activation-direction changes. In our ablations, this allocation improves generation quality over random boosting with the same number of boosted timesteps, supporting directional shifts as a useful selection signal.