论文部分内容阅读
Abstract Aiming at the problems of single classification method and high classification cost of kiwifruit in China， we proposed a grading method based on kiwifruit surface defects. A set of kiwifruit image acquisition system was built. The K-means clustering segmentation algorithm was used to segment the surface defects， and then color contrast was performed to determine whether it was a piece of defective fruit. Then， the shape features of normal fruit were extracted and an SVM classifier was designed to further determine its grade. This method has the advantages of low cost， simple algorithm and high efficiency， which opens a new way for fruit classification， and is of great significance to promoting the development of fruit classification industry in China and enhancing international competitiveness.
Key words Kiwifruit; Surface defect identification; Fruit classification
Received： May 10， 2021 Accepted： July 1 2021
Supported by the Chinese Society of Logistics （2021CSLKT3-286）.
Jingjing MA （1992-）， female， P. R. China， master， research field big data/deep learning.
*Corresponding author. Tao YANG （1991-）， male， P. R. China， master， devoted to research about Agricultural Engineering and Informatization/Machine Vision Technology. E-mail： egstao@163.com.
Kiwifruit planting bases in China are mainly distributed in the Dabie Mountains， the northern foot of the Qinling Mountains in Shaanxi， the Guizhou Plateau and the western part of Hunan Province， Guangdong， Sichuan and other places. Its output reaches 2.37×106 t， far surpassing Italy and New Zealand， and China has become the world’s number one producer[1]. However， the export volume of kiwifruit （approximately 2 000 t） is less than one-thousandth of the total output of China. We investigated the main producing areas of kiwifruit in Sichuan Province （Cangxi， Dujiangyan， Pujiang） and found that the proportion and level of post-production processing of kiwifruit in China is low， and the quality of the products is uneven， which makes it difficult for kiwifruit to enter the high-end market. It can be seen that the quality of kiwifruit is one of the main factors affecting its sales and price. However， the size， color， shape， surface defects， maturity， pesticide residues and other traits of kiwifruit are the main factors affecting the quality of kiwifruit. Grading kiwifruit according to certain standards is an important means to increase the added value of the fruit and enhance international competitiveness. However， the conventional method of distinguishing fruit grades by size or weight has been difficult to meet people’s needs for high-quality kiwifruit in modern society. Therefore， exploring a fast， inexpensive and accurate grading method for kiwifruit is an important part of promoting the development of the kiwifruit industry. Based on this， scholars at home and abroad have joined the research on online detection and automatic grading of kiwifruit. Among them， Northwest A&F University has carried out a systematic study on kiwifruit grading methods. Huo et al.[2] proposed a detection method of establishing a recognition model by hyperspectral technology combined with machine learning， which can perform fast and non-destructive testing of kiwifruit with excessive use of 1-MCP chemical preservative， with an average recognition rate of 100%. Yan et al.[3] judged whether the kiwifruit had been treated with a swelling agent by comparing the length-to-width ratio of the smallest circumscribed rectangle of the calyx area of kiwifruit， and its recognition rate reached 91.55%. Liu et al.[4] designed an area-based kiwifruit size grading control system， with an average grading rate up to 2.5 s/piece. Foreign fruit automatic grading production equipment has been quite perfect， and the US OSCARTM and MERLIN high-speed fruit grading production lines have been widely used in the grading of fruits such as apples， pears and citrus[5]. The research conducted by scholars at home and abroad on kiwifruit classification has not yet involved the classification method of surface defect recognition. Therefore， a kiwifruit grading system was designed based on the kiwifruit surface defect recognition technology. The system has the advantages of low cost and fast recognition speed. It has positive significance for eveloping small kiwifruit grading devices， increasing fruit farmers’ income， and helping rural development. Kiwifruit Grade
According to the Hayward kiwifruit grading standard of Chengdu Kiwifruit Association and NY/T 1794-2009 Kiwifruit Grading Standard， fresh kiwifruit that meets the standard was manually selected （Fig. 1）. In short， the fresh kiwifruit had a long oval shape and uniform color distribution. The larger the volume， the higher the quality.
Kiwifruit will inevitably have some surface defects such as spots， sunburns， scars， rolled pits during growth， picking， transport， sorting， and packaging （Fig. 2）. Such fresh kiwifruit with surface defects is collectively referred to as defective fruit.
Kiwifruit Grading Method
The kiwifruit grading method based on surface defect recognition proposed in the study is shown in Fig. 3. First， kiwifruit images are collected by an image acquisition system; then they are preprocessed to adjust the image effect; and next， the K-means clustering algorithm is applied to cluster and segment the preprocessed kiwifruit images， obtaining the kiwifruit surface defect
images and extract their color moment information， which is compared with the color moment information of the kiwifruit without surface defects to determine whether the kiwifruit is defective. If it is judged to be a normal fruit， then its shape feature information will be extracted to form a feature vector; and finally， an SVM classifier is designed to output the kiwifruit grade information.
Image acquisition and preprocessing
The kiwifruit image acquisition system is shown in Fig. 4. It is composed of PC， CCD camera， lens， camera bracket， LED ring light source and other components. The system was used to collect 60 images of kiwifruit of grade I， grade II， grade III， and defective fruit. Then， the Otsu segmentation algorithm was applied to remove the background and perform Hough transform to obtain the smallest circumscribed rectangle of the target area， and extract the kiwifruit contour curve， thereby realizing kiwifruit image segmentation for further analysis and processing of the images[6].
Surface defect recognition
Defect segmentation
To extract the surface defect characteristics of kiwifruit， it is necessary to segment the defects， and the K-means clustering algorithm was applied to process the preprocessed kiwifruit images. The K-means clustering algorithm includes following steps： ① randomly selecting K initial cluster centers as the starting point; ② calculating the distance between each sample and each cluster center， and classifying each sample into the nearest cluster center; ③ determining the mean value of the samples in the cluster as the new cluster center of the cluster; and ④ repeating steps ② and ③ until the cluster center no longer changes or the set number of iterations is reached. Experiments have shown that the algorithm can segment surface defects in kiwifruit images well， and meanwhile， the clustering effect of normal fruit images is also good. Color discrimination
People can visually identify whether there are defects on the surface of kiwifruit and what grade the kiwifruit belongs to， while computers can only judge by specific data comparison. Therefore， each color component of segmented kiwifruit images was extracted and their third-order color moments were calculated respectively. The color moments were extracted from the images of fresh kiwifruit of different grades to obtain the color distribution in the images of fresh kiwifruit of different grades （Fig. 5）. It can be seen from Fig. 5 that the surface defects of kiwifruit would first show up as color changes， that is， surface defects such as scars and lesions would be darker than normal kiwifruit， especially the third-order moments in the HSV color space were quite different. However， the differences in the color feature information of normal kiwifruit were not obvious after segmentation. Therefore， the kiwifruit with large differences in color characteristics from the normal fruit can be judged as defective fruit.
Agricultural Biotechnology2021
Feature extraction
The combination of kiwifruit shape characteristics and size measurement is an important basis for distinguishing different grades of kiwifruit. From the appearance point of view， kiwifruit can be further distinguished by its shape and size. Therefore， extracting kiwifruit shape features and combining them with size measurement can be an important basis for distinguishing kiwifruit grades. The preprocessed kiwifruit images were converted into binary images， which was subjected to connected component analysis， and then five shape feature parameters， area A， perimeter L， circularity ρc， rectangularity ρR， and minimum circumscribed rectangle length-width ratio ρWL， were calculated， respectively， and the kiwifruit shape feature information was obtained， as shown in Fig. 6 and Fig. 7.
Area A refers to the total number of pixels in the connected domain. This feature is greatly affected by factors such as image size and shooting conditions. Therefore， it is necessary to obtain kiwifruit images to be recognized in a specific imaging system to achieve a certain comparative significance.
A=∑（x， y）∈Sf（x， y）（1）
Circumference L refers to the length of the contour line surrounding the boundary of a connected domain.
L=N1+2·N2（2）
In the formula， N1 represents the number of pixels with an even-numbered direction code; and N2 represents the number of pixels with an odd-numbered direction code. The degree of circularity ρc is also called quasi-circularity， which indicates the similarity of a target image to a circle. It is defined as the ratio of 4π times the area A of the target image to the square of its circumference L.
ρc=4πAL2（3）
Rectangularity ρR describes the fullness of a target image in its smallest enclosing rectangular area， that is， the ratio of the target image area A to its smallest enclosing rectangular area A′. Obviously， the rectangularity of a rectangle is the rectangularity of a circle is π/4， the rectangularity of an equilateral triangle is 0.5， and the rectangularity of other irregular shapes is between 0 and 1. And this feature is less affected by the imaging conditions， and has a strong ability to characterize the shape of the target image.
ρR=AA′（4）
The length-width ratio ρWL refers to the length-width ratio of the minimum enclosing rectangle of a target image. This value is not affected by the size and direction of the image， and is an ideal geometric shape feature.
ρWL=LRWR（5）
Discrimination
It can be seen from the shape characteristics of kiwifruit that there are different degrees of difference in the shape of kiwifruit of different grades. Simple logical discrimination will cause larger recognition errors. In order to improve the recognition rate of the system， an SVM multi-level classifier was designed， and the discriminant function is shown in Equation 6.
f（x）=∑ni=0a*iyiK（x， xi）+b*（6）
In the formula， a*i is the optimal solution of Lagrange coefficient; K（x， xi） is the kernel function， here the RBF kernel function was selected， that is， K（x， xi）=e-‖x-xi‖22σ2; σ is the width of the radial basis kernel function， σ> 0; xi is the support vector; yi is the class label corresponding to xi; n is the number of support vectors; and b* is the optimal vector of the optimal solution object.
Kiwifruit Grading System
In order to facilitate users to grade kiwifruit， a kiwifruit grading system was developed （Fig. 8）. The top is the menu bar， the left side is divided into two upper and lower areas to display the grading process images， and the right side is the button operation and result display area. It can meet the needs of small-scale farmers for kiwifruit grading. In the future， the system can be further expanded to accommodate more fruit grading needs.
Conclusions
The study was conducted based on the surface defect images of fresh kiwifruit. A kiwifruit surface defect image grading method combining K-means clustering algorithm and SVM was proposed and a kiwi fruit grading system was developed to meet the needs of small-scale farmers. It opens up new ideas for fruit grading， which is of great significance for increasing fruit farmers’ income. References
[1] MA JJ， YANG T， YE AS， et al. Study on kiwifruit combination sales and pricing strategy based on profit maximization[J]. Journal of Anhui Agricultural Sciences， 2019， 47（8）： 219-221. （in Chinese）
[2] HUO YQ， ZHANG C， LI YH， et al. Nondestructive detection for kiwifruit based on the hyperspectral technology and machine learning[J]. Journal of Chinese Agricultural Mechanization， 2019， 40（4）： 71-77. （in Chinese）
[3] YAN B， GUO WC. Identifying expanded ‘Hayward’ kiwifruits based on K-means clustering algorithm and calyx shape[J]. Journal of Northwest A & F University： Natural Science Edition， 2020（5）： 1-8. （in Chinese）
[4] LIU ZC， GAI XH. Design of kiwifruit grading control system based on machine vision and PLC[J]. Journal of Chinese Agricultural Mechanization， 2020， 41（1）： 131-135. （in Chinese）
[5] QU T， QI KK， LIU YD， et al. Design and experiment on automatic grading system of appearance size for kiwi[J] . Journal of Agricultural Mechanization Research， 2017， 39（10）： 98-103. （in Chinese）
[6] ZHANG M， LI P， DENG L， et al. Segmentation of navel orange surface defects based on mask and brightness correction algorithm[J]. Scientia Agricultura Sinica， 2019， 52（2）： 327-338. （in Chinese）
Key words Kiwifruit; Surface defect identification; Fruit classification
Received： May 10， 2021 Accepted： July 1 2021
Supported by the Chinese Society of Logistics （2021CSLKT3-286）.
Jingjing MA （1992-）， female， P. R. China， master， research field big data/deep learning.
*Corresponding author. Tao YANG （1991-）， male， P. R. China， master， devoted to research about Agricultural Engineering and Informatization/Machine Vision Technology. E-mail： egstao@163.com.
Kiwifruit planting bases in China are mainly distributed in the Dabie Mountains， the northern foot of the Qinling Mountains in Shaanxi， the Guizhou Plateau and the western part of Hunan Province， Guangdong， Sichuan and other places. Its output reaches 2.37×106 t， far surpassing Italy and New Zealand， and China has become the world’s number one producer[1]. However， the export volume of kiwifruit （approximately 2 000 t） is less than one-thousandth of the total output of China. We investigated the main producing areas of kiwifruit in Sichuan Province （Cangxi， Dujiangyan， Pujiang） and found that the proportion and level of post-production processing of kiwifruit in China is low， and the quality of the products is uneven， which makes it difficult for kiwifruit to enter the high-end market. It can be seen that the quality of kiwifruit is one of the main factors affecting its sales and price. However， the size， color， shape， surface defects， maturity， pesticide residues and other traits of kiwifruit are the main factors affecting the quality of kiwifruit. Grading kiwifruit according to certain standards is an important means to increase the added value of the fruit and enhance international competitiveness. However， the conventional method of distinguishing fruit grades by size or weight has been difficult to meet people’s needs for high-quality kiwifruit in modern society. Therefore， exploring a fast， inexpensive and accurate grading method for kiwifruit is an important part of promoting the development of the kiwifruit industry. Based on this， scholars at home and abroad have joined the research on online detection and automatic grading of kiwifruit. Among them， Northwest A&F University has carried out a systematic study on kiwifruit grading methods. Huo et al.[2] proposed a detection method of establishing a recognition model by hyperspectral technology combined with machine learning， which can perform fast and non-destructive testing of kiwifruit with excessive use of 1-MCP chemical preservative， with an average recognition rate of 100%. Yan et al.[3] judged whether the kiwifruit had been treated with a swelling agent by comparing the length-to-width ratio of the smallest circumscribed rectangle of the calyx area of kiwifruit， and its recognition rate reached 91.55%. Liu et al.[4] designed an area-based kiwifruit size grading control system， with an average grading rate up to 2.5 s/piece. Foreign fruit automatic grading production equipment has been quite perfect， and the US OSCARTM and MERLIN high-speed fruit grading production lines have been widely used in the grading of fruits such as apples， pears and citrus[5]. The research conducted by scholars at home and abroad on kiwifruit classification has not yet involved the classification method of surface defect recognition. Therefore， a kiwifruit grading system was designed based on the kiwifruit surface defect recognition technology. The system has the advantages of low cost and fast recognition speed. It has positive significance for eveloping small kiwifruit grading devices， increasing fruit farmers’ income， and helping rural development. Kiwifruit Grade
According to the Hayward kiwifruit grading standard of Chengdu Kiwifruit Association and NY/T 1794-2009 Kiwifruit Grading Standard， fresh kiwifruit that meets the standard was manually selected （Fig. 1）. In short， the fresh kiwifruit had a long oval shape and uniform color distribution. The larger the volume， the higher the quality.
Kiwifruit will inevitably have some surface defects such as spots， sunburns， scars， rolled pits during growth， picking， transport， sorting， and packaging （Fig. 2）. Such fresh kiwifruit with surface defects is collectively referred to as defective fruit.
Kiwifruit Grading Method
The kiwifruit grading method based on surface defect recognition proposed in the study is shown in Fig. 3. First， kiwifruit images are collected by an image acquisition system; then they are preprocessed to adjust the image effect; and next， the K-means clustering algorithm is applied to cluster and segment the preprocessed kiwifruit images， obtaining the kiwifruit surface defect
images and extract their color moment information， which is compared with the color moment information of the kiwifruit without surface defects to determine whether the kiwifruit is defective. If it is judged to be a normal fruit， then its shape feature information will be extracted to form a feature vector; and finally， an SVM classifier is designed to output the kiwifruit grade information.
Image acquisition and preprocessing
The kiwifruit image acquisition system is shown in Fig. 4. It is composed of PC， CCD camera， lens， camera bracket， LED ring light source and other components. The system was used to collect 60 images of kiwifruit of grade I， grade II， grade III， and defective fruit. Then， the Otsu segmentation algorithm was applied to remove the background and perform Hough transform to obtain the smallest circumscribed rectangle of the target area， and extract the kiwifruit contour curve， thereby realizing kiwifruit image segmentation for further analysis and processing of the images[6].
Surface defect recognition
Defect segmentation
To extract the surface defect characteristics of kiwifruit， it is necessary to segment the defects， and the K-means clustering algorithm was applied to process the preprocessed kiwifruit images. The K-means clustering algorithm includes following steps： ① randomly selecting K initial cluster centers as the starting point; ② calculating the distance between each sample and each cluster center， and classifying each sample into the nearest cluster center; ③ determining the mean value of the samples in the cluster as the new cluster center of the cluster; and ④ repeating steps ② and ③ until the cluster center no longer changes or the set number of iterations is reached. Experiments have shown that the algorithm can segment surface defects in kiwifruit images well， and meanwhile， the clustering effect of normal fruit images is also good. Color discrimination
People can visually identify whether there are defects on the surface of kiwifruit and what grade the kiwifruit belongs to， while computers can only judge by specific data comparison. Therefore， each color component of segmented kiwifruit images was extracted and their third-order color moments were calculated respectively. The color moments were extracted from the images of fresh kiwifruit of different grades to obtain the color distribution in the images of fresh kiwifruit of different grades （Fig. 5）. It can be seen from Fig. 5 that the surface defects of kiwifruit would first show up as color changes， that is， surface defects such as scars and lesions would be darker than normal kiwifruit， especially the third-order moments in the HSV color space were quite different. However， the differences in the color feature information of normal kiwifruit were not obvious after segmentation. Therefore， the kiwifruit with large differences in color characteristics from the normal fruit can be judged as defective fruit.
Agricultural Biotechnology2021
Feature extraction
The combination of kiwifruit shape characteristics and size measurement is an important basis for distinguishing different grades of kiwifruit. From the appearance point of view， kiwifruit can be further distinguished by its shape and size. Therefore， extracting kiwifruit shape features and combining them with size measurement can be an important basis for distinguishing kiwifruit grades. The preprocessed kiwifruit images were converted into binary images， which was subjected to connected component analysis， and then five shape feature parameters， area A， perimeter L， circularity ρc， rectangularity ρR， and minimum circumscribed rectangle length-width ratio ρWL， were calculated， respectively， and the kiwifruit shape feature information was obtained， as shown in Fig. 6 and Fig. 7.
Area A refers to the total number of pixels in the connected domain. This feature is greatly affected by factors such as image size and shooting conditions. Therefore， it is necessary to obtain kiwifruit images to be recognized in a specific imaging system to achieve a certain comparative significance.
A=∑（x， y）∈Sf（x， y）（1）
Circumference L refers to the length of the contour line surrounding the boundary of a connected domain.
L=N1+2·N2（2）
In the formula， N1 represents the number of pixels with an even-numbered direction code; and N2 represents the number of pixels with an odd-numbered direction code. The degree of circularity ρc is also called quasi-circularity， which indicates the similarity of a target image to a circle. It is defined as the ratio of 4π times the area A of the target image to the square of its circumference L.
ρc=4πAL2（3）
Rectangularity ρR describes the fullness of a target image in its smallest enclosing rectangular area， that is， the ratio of the target image area A to its smallest enclosing rectangular area A′. Obviously， the rectangularity of a rectangle is the rectangularity of a circle is π/4， the rectangularity of an equilateral triangle is 0.5， and the rectangularity of other irregular shapes is between 0 and 1. And this feature is less affected by the imaging conditions， and has a strong ability to characterize the shape of the target image.
ρR=AA′（4）
The length-width ratio ρWL refers to the length-width ratio of the minimum enclosing rectangle of a target image. This value is not affected by the size and direction of the image， and is an ideal geometric shape feature.
ρWL=LRWR（5）
Discrimination
It can be seen from the shape characteristics of kiwifruit that there are different degrees of difference in the shape of kiwifruit of different grades. Simple logical discrimination will cause larger recognition errors. In order to improve the recognition rate of the system， an SVM multi-level classifier was designed， and the discriminant function is shown in Equation 6.
f（x）=∑ni=0a*iyiK（x， xi）+b*（6）
In the formula， a*i is the optimal solution of Lagrange coefficient; K（x， xi） is the kernel function， here the RBF kernel function was selected， that is， K（x， xi）=e-‖x-xi‖22σ2; σ is the width of the radial basis kernel function， σ> 0; xi is the support vector; yi is the class label corresponding to xi; n is the number of support vectors; and b* is the optimal vector of the optimal solution object.
Kiwifruit Grading System
In order to facilitate users to grade kiwifruit， a kiwifruit grading system was developed （Fig. 8）. The top is the menu bar， the left side is divided into two upper and lower areas to display the grading process images， and the right side is the button operation and result display area. It can meet the needs of small-scale farmers for kiwifruit grading. In the future， the system can be further expanded to accommodate more fruit grading needs.
Conclusions
The study was conducted based on the surface defect images of fresh kiwifruit. A kiwifruit surface defect image grading method combining K-means clustering algorithm and SVM was proposed and a kiwi fruit grading system was developed to meet the needs of small-scale farmers. It opens up new ideas for fruit grading， which is of great significance for increasing fruit farmers’ income. References
[1] MA JJ， YANG T， YE AS， et al. Study on kiwifruit combination sales and pricing strategy based on profit maximization[J]. Journal of Anhui Agricultural Sciences， 2019， 47（8）： 219-221. （in Chinese）
[2] HUO YQ， ZHANG C， LI YH， et al. Nondestructive detection for kiwifruit based on the hyperspectral technology and machine learning[J]. Journal of Chinese Agricultural Mechanization， 2019， 40（4）： 71-77. （in Chinese）
[3] YAN B， GUO WC. Identifying expanded ‘Hayward’ kiwifruits based on K-means clustering algorithm and calyx shape[J]. Journal of Northwest A & F University： Natural Science Edition， 2020（5）： 1-8. （in Chinese）
[4] LIU ZC， GAI XH. Design of kiwifruit grading control system based on machine vision and PLC[J]. Journal of Chinese Agricultural Mechanization， 2020， 41（1）： 131-135. （in Chinese）
[5] QU T， QI KK， LIU YD， et al. Design and experiment on automatic grading system of appearance size for kiwi[J] . Journal of Agricultural Mechanization Research， 2017， 39（10）： 98-103. （in Chinese）
[6] ZHANG M， LI P， DENG L， et al. Segmentation of navel orange surface defects based on mask and brightness correction algorithm[J]. Scientia Agricultura Sinica， 2019， 52（2）： 327-338. （in Chinese）