Principles of Local and Global Grouping that Underlie Segmentation of Natural Texture Images

Abstract The human visual system segments images using both high-level recognition mechanisms and low-level mechanisms that are largely independent of specific prior experience. The low-level mechanisms are essential for initiating recognition processes, and for learning to recognize new materials, objects, and contexts. Here we describe a hierarchical Bayesian observer (HBO) model of texture segmentation that is biologically plausible, takes into account the statistics of natural scenes, and does not depend on prior experience. The HBO model consists of five steps: local similarity grouping with local normalization, mutual similarity grouping (local grouping is strengthened if the neighboring regions are similar to the same set of other regions), transitive grouping (good continuation), confidence grouping (neighboring regions far from the same-different decision boundary guide grouping of regions near the decision boundary), and region grouping (similarity grouping of the regions from the initial segmentation). We find that a local similarity grouping process, trained to maximize accuracy, predicts human texture discrimination accuracy. We then find that the four additional steps accurately segment images with randomly shaped regions containing arbitrary natural textures. The success of the model depends on all the steps, but especially on local-similarity and transitive grouping. We also find that the transitive grouping allows correct segmentation of non-stationary texture regions (e.g., textures slanted in depth). Further, we find that when illumination varies across the image, local normalization enables both correct texture segmentation and estimation of illumination change. Finally, we find that unlike our model large state-of-the-art deep networks often fail on these stimuli. Competing Interest Statement The authors have declared no competing interest.