An insect-inspired, spike-based, in-sensor, and night-time collision detector based on atomically thin and light-sensitive memtransistors

Abstract Collision detection under poor illumination and nightly conditions pose significant challenge for manned and unmanned vehicles, flying drones, and robots navigating complex terrestrial and extraterrestrial geographies. Existing night vision cameras offer solution based on sophisticated enhancement algorithms or additional thermal sensors necessitating extensive and expensive hardware, which make them power-hungry and untenable for deployment in remote and resource constrained locations. In contrast, nocturnal flying insects can avoid collision using very limited neural resources. Insect-inspired collision detectors based on silicon complementary metal oxide semiconductor technology and field programmable gate arrays also exist, however, the physical separation between sensing and compute and absence of spike-based information processing capability increases their area and energy overhead. Here, we introduce an insect-inspired, spike-based, and in-sensor collision detector using a reconfigurable optoelectronic integrated circuit constructed based on atomically-thin and light-sensitive memtransistors. We imitate the escape response of lobula giant movement detector (LGMD) neuron found in many insect species and demonstrate timely collision detection for various real-life scenarios at night involving cars on collision course. Our collision detector has a small effective footprint of 40 µm2 and consumes miniscule energy of few hundreds pico-Joules.