KELM: Knowledge-Enhanced Learning Methodology for Cardinality Estimation

Recent years, the database community has attempted to develop learned cardinality estimator for improving the estimation accuracy. Although some researches show that the applying deep learning to cardinality estimation is a significant and promising direction, there still exists many problems in implementing these techniques to real applications (long preparation time, unstable performance and poor model transition). We comprehensively analyze the characteristics of statistical estimators which have the potential ability to enhance the learned estimators for solving these problems. After carefully design, we propose a knowledge-enhanced learning methodology for cardinality estimation, called KELM. To our knowledge, KELM is the first work to regard the statistical estimators as knowledge for enhancing the learned estimators. Specifically, we propose a knowledge enhanced preparation phase which could accelerate the model pretraining by reducing queries execution. For online phase, we design stable model inference algorithm and a two-stage transition method for efficiently adapting to dynamic workloads. Experimental results in four open source datasets demonstrate that KELM outperforms existing works in model preparation, model inference and model transition. Our work brings a new development direction for the application of learned estimators in real database environments.