Apple detection in nighttime tree images using the geometry of light patches around highlights

Detection of fruit in tree images has been the focus of numerous studies. Although most studies considered approaches based primarily on color analysis, the major drawback of such approaches is that the fruit apparent color depends not only on variety or physiological stage but also on illumination, which is inherently non-uniform within the canopy, even if artificial lighting is used. In the present work we developed a novel approach to detect apples in nighttime images by analyzing the spatial distribution of the light around highlights ("bright spots"). The approach is based on the observation that, under the artificial illumination used, apples exhibit strong specular reflection so that a small, but very bright, spot is visible on almost all apples. Each of these highlights serves as the center of a region of interest and is the seed of the investigated light patch. This patch is initially very small but its size is increased iteratively by annexing pixels with predefined decreasing gray level intensities. The evolution of the patch geometry is used to determine whether it corresponds to an apple. The approach was tested with two datasets containing over 360 images (close to 13,000 apples) acquired in the same 'Golden Delicious' orchard in July 2012 and August 2013. Twenty images from the 2012 dataset were randomly selected to develop and calibrate the procedure. The results of these 20 images were used to establish a linear relationship between the number of detected objects and the actual number of apples visible in the images (R²~0.75). Applying the calibrated procedure to the remaining images of this dataset led to an estimate of 6739 apples compared to a visual count of 6195 apples (~9% overestimate). Analysis of the 2013 dataset, in which the apparent size of the apples was smaller, required only adjustment of the two parameters related to apple size. Following this adjustment, 12 images were randomly selected to determine the relationship between the number of detected objects and the actual number of apples (R²~0.74). Using this relationship, the estimated number of apples was 6687, compared to the visual count of 6713 fruits.