Sparse matrix-vector multiplication (SpMV) is a crucial operation in scientific computing, graph analytics, and machine/deep learning. Its performance is highly sensitive to matrix sparsity patterns, necessitating tailored program designs. This paper introduces \textsc{SparseZETA}, an intelligent auto-tuner that generates high-performance, machine-designed SpMV programs by directly mimicking and composing human-expert actions. To efficiently navigate the vast design space, \textsc{SparseZETA} reformulates auto-tuning as a behavior-cloning problem: rather than costly exploration, it directly synthesizes programs by sequentially predicting actions in a one-pass decision-making process, guided by the real-time state of the evolving, partially constructed program designs.
A novel self-training mechanism further accelerates the collection of training data for the prediction models.
On NVIDIA A100 (and RTX 2080 Ti) GPUs, \textsc{SparseZETA} achieves average speedups of $1.27\times$–$15.66\times$ ($1.44\times$–$19.07\times$) over existing auto-tuners, human-designed programs, and a sparse compiler.
\textsc{SparseZETA} substantially reduces the human effort required to design SpMV programs, including sparse format creation and kernel implementation, cutting the design time from days or even months to an average of \SI{82.52}{ms} per matrix via lightweight inference on only one CPU.