Can AI reproduce observed chemical diversity?

Generating diverse molecules with desired chemical properties is important for drug discovery. The use of generative neural networks is promising for this task. To facilitate evaluation of generative models, this paper introduces a metric of internal chemical diversity, and raises the following challenge: can a nontrivial AI model reproduce observed internal diversity for desired molecules? To illustrate this metric, a mini-benchmark is performed with two generative models: a Reinforcement Learning model and the recently introduced ORGAN. The aim of this paper is to encourage research about internal diversity metrics.