SORT-AI: Accelerator Runtime Coherence in Heterogeneous AI Inference Infrastructure A Structural Analysis of Cross-Layer Instability in Large-Scale Systems

Large scale AI inference is shifting from relatively homogeneous accelerator fleets toward structurally heterogeneous infrastructure composed of mixed accelerator generations, disaggregated serving paths, virtualized execution layers, and cross environment placement. This transition introduces a class of instability phenomena that does not arise from component failure in isolation, but from cross layer incoherence between accelerator runtimes, network topology, virtualization boundaries, and migration induced reconfiguration. Classical performance metrics such as utilization, average latency, and tokens per second remain necessary, but they do not capture these structural effects because they observe individual layers rather than the coupled system formed across them. This paper develops a vendor agnostic structural analysis of heterogeneous inference infrastructure and argues that runtime coherence functions as a hidden performance variable that conditions effective capacity, tail latency, and cost efficiency before conventional optimization can succeed. The paper introduces a taxonomy of five instability modes, latency asymmetry drift, memory path incoherence, interconnect induced capacity inaccessibility, virtualization induced control distortion, and migration induced runtime reconfiguration risk. It then maps these modes to four diagnostic domains, accelerator runtime control, structural network scalability, virtualization overhead stability, and structural cloud migration risk. The contribution is analytical rather than prescriptive. It does not propose implementation mechanisms or vendor specific remedies, but provides a research framework for identifying where and why heterogeneous inference systems produce emergent instability under scale.