1. Introduction
Medicinal plants include the different types used in herbology. The word herb was originally used to refer to non-woody plants but has since evolved to include stems, roots, bark, leaves, and flowers. Medicinal plants cure disease and herbal medicine is deemed safe, with minimal or no side effects. Herbs can be used to treat people across all age groups.
Medicinal plants offer the ingredients necessary to prepare Ayurvedic, Unani and Siddha medication as well as an array of synthetic drugs. Medicinal plants are widely used in diverse cultures around the world. Certain herbs are used as nutrient supplements owing to their therapeutic values, and in perfume, natural dyes and pesticide control. They heal wounds and boils and are used as blood purifiers, antibiotics and antacids that aid in digestion. In ancient India, herbs were used to treat snake bites in particular and animal bites in general. Certain herbs are used as nutrient supplements owing to their therapeutic values, and in perfume, natural dyes and pesticide control. They heal wounds and boils and are used as blood purifiers, antibiotics and antacids that aid in digestion. In ancient India, herbs were used to treat snake bites in particular and animal bites in general. Medicinal plants are used in agriculture and the preparation of essential oils, and give the economy a huge boost as well. Several species of herbs are used as preservatives, while others have replaced artificial color in the food industry. Medicinal and aromatic plants are used as food supplements and in the preparation of candy and cosmetics. The classification of medicinal herbs is, however, a challenge.
The classification of medicinal plants is necessary, from a pharmacological point of view, because medicines are derived from plants and microbes. The earth contains more than 5, 00, 000 botanical species like algae, fungi, lichens, liverworts, mosses, ferns, conifers and flowering plants. The International Code of Botanical Nomenclature [1] has provided various taxonomic ranks in botanical classification, including genus, family, order, class, division, subdivision and kingdom. Plant taxonomy comprises three types of plant classification systems - artificial or mechanical, natural and phylogenetic. The artificial or mechanical plant classification system comprises one or a few characters to group taxa, while the natural system contains numerous characters to classify plants and the phylogenetic system recognizes the evolutionary relationship among taxa. Botanists identify medicinal plants through their anatomical or morphological structural characteristics. A complete plant specimen is needed for a morphological analysis to be carried out before herbal drugs can be identified and extracted. However, of the two, an anatomical structural analysis works best at plant identification and authentication, though it is a time-consuming process. An anatomical structure analysis examines a plant’s internal structure, particularly cell structure, through light microscopy. However, such an analysis calls for superior knowledge of plant cellular structure [2]. To resolve the problems above faced by pharmacologists and practitioners of Ayurveda, Unani and Siddha with regard to herbal plants, an efficient plant species classification tool using computer vision techniques is the need of the hour.
Arun et al. (2017) [3] proposed an optimized color channel and texture models to classify medicinal leaves native to Kanyakumari district in the state of Tamil Nadu in India. Only five species of medicinal leaves were considered, totalling 250 images in all. Their findings showed that leaf color varies as a result of climate change, bacteria and fungus. Naresh et al. (2016) [4] introduced a modified local binary pattern (MLBP) technique to extract medicinal leaf features from the UoM Medicinal Plants Dataset, using the nearest neighbor classifier. Shitole et al. (2019) [5] used morphological shape features to classify leaves and tested their system with only five different species of plants. Xue et al. (2019) [6] used the geometric features of leaves to classify 20 different Chinese medicinal plants.
Hamid et al. (2017) [7] used the leaf quartile feature to handle high interclass similarity between different plant species. Leaf images were aligned, with their model supporting rotational invariance. Aakif et al. (2015) [8] utilized morphological leaf shape features to classify plants and applied different normalization techniques for rotation and translation invariances. Chaki et al. (2018) [9] exploited shape, color and texture features for plant classification, utilizing a significant number of features to classify plants in different hierarchical structures. Clark et al. (2017) [10] and Wang et al. (2019) [11] employed leaf tooth features to classify herbarium data, a process complicated by damage to the leaf teeth. Souza et al. (2016) [12] used multiscale bending energy (MBE) descriptors to classify plants, applying optimization techniques for feature selection, given that the MBE provides numerous features at different scales. Kolivand et al. (2019) [13] employed venation-based morphological features, though the venation patterns are undefined in some leaves and the computational time of the venation feature is high. Turkoglu et al. (2019) [14] exploited texture features using a local binary pattern of the red and green pixels of the leaves. Zhang et al. (2020) [15] used global and local leaf margin features to classify plant species. Thyagharajan et al. [16] reviewed an array of visual descriptors and classifiers used in leaf species classification.
Plants are a major source of ingredients in the preparation of herbal medicine. Every part of certain plants, including the root, stem, leaf, seed and flowers, is used to this end. Leaves are two-dimensional objects with indubitable therapeutic properties and are non-seasonal. A key issue with the digitization of herbal plants is the selection of sources from every part of a plant. The leaf [7-9] is a critical component of the medicinal plant species classification system. Leaves are used to extract domain-specific features such as shape, venation details, base and apex shapes and, finally, leaf types.
Computer vision techniques are applied to design a medicinal plant species classification system by extracting geometric, morphological and shape features from medicinal leaves. Morphological features alone, however, are not enough to classify leaves. Rather, all leaf features are to be extracted. This is because numerous inter-class species look the same in terms of leaf and base shapes as well as vein arrangement. Traditionally, attempts have been made to resolve the problem by analyzing the internal cell structure of leaves using the light intensity microscopic technique. An efficient computer vision-based medicinal leaf classification system is needed to differentiate between inter-species leaf similarity. Such a system aligns leaf images to remove image invariances. In this paper, we propose the LCFN model, which utilizes the biological characteristics of the PCNN [17] . The LCFN has distinctive characteristics. It is a simpler version of the PCNN, contains fewer parameters and is computationally fast. The number of neurons in the LCFN is equal to the size of the image and the network size changes dynamically, depending on image size. It detects distinct interspecies quickly, using pulse signals by grouping different neurons in each iteration. The LCFN model increases the speed at which neurons are fired by reducing the input with each iteration. It operates on images that contain no straight lines and on those with weak edges. The model supports translation, rotation and scaling invariance. Further, separate normalization and dimension reduction techniques are not called for. Image features are easily extracted using the LCFN, without their being aligned [2-5].
Distinguishing between herbal leaves requires an in-depth analysis of their shape, vein patterns and color, along with a combination of machine learning methods to achieve the best results.Our chief contributions towards resolving this problem are
i. to propose an LCFN model that combines different groups of pixels to extract different types of features in each iteration.
ii. to extract rotational-, scale- and transform-invariant features that demand no augmented images.
iii. to improve feature quality using the Chan-Vese active contour segmentation algorithm.
iv. to evaluate the extracted features using the 5-fold Bayesian regularization neural network (BRNN), K-nearest neighbors (KNN), support vector machine (SVM) and an ensemble classifier, and compare the recognition accuracy produced by each classification method.
v. to create an Indian Medicinal Leaf Species Dataset containing a total of 635 images.
2. Materials and Methods
2.1 The Herbal Dataset and its Classification
Medicinal plants are categorized on the basis of their use, active constituents life cycle and botanical classification. Based on their particular uses, herbs are divided into four classes - medicinal, culinary, aromatic and ornamental. Medicinal herbs are used in the preparation of medicine, culinary herbs in cooking, aromatic herbs for their pleasant smell and ornamental herbs for decor. They are further divided into aromatic, astringent, bitter, mucilaginous and nutritive, based on their active constituents. Astringent herbs are used as antiseptics; bitter herbs as antibiotics; mucilaginous herbs as digestives, producing a sweet taste when added to water, owing to their polysaccharide content, and nutritive herbs add restorative value to the diet. They are further divided into aromatic, astringent, bitter, mucilaginous and nutritive, based on their active constituents. Astringent herbs are used as antiseptics; bitter herbs as antibiotics; mucilaginous herbs as digestives, producing a sweet taste when added to water, owing to their polysaccharide content, and nutritive herbs add restorative value to the diet. Based on their life cycle, herbs are classified as annuals, biennials and perennials. Our dataset comprises 635 images of herbal leaves such as aloe, betel, Indian borage, bittergourd, grape, insulin herb, guava, mango, nilavembu, nithiyakalyani, sweet basil and pomegranate, collected from southern Tamil Nadu. These images are divided, depending on their use, into medicinal, culinary, aromatic and ornamental herbs.
The medicinal plants in Fig. 1 are identified through the morphological properties of leaves and their uses. However, certain herbal plants are similar in their morphological, physiological and biochemical characteristics. The morphological and chemical compounds of medicinal plants share an intrinsic relationship. Traditional laboratory-based plant classification methods are expensive and laborious. Medicinal plants are to be digitized to conserve them, and herbs are to be identified quickly by pharmacologists and doctors through an examination of their leaves. Researchers today use image processing and computer vision techniques to protect herbs.
Fig. 1. Sample medicinal species
3. The Proposed System
Fig. 2 shows the flow diagram of the proposed system, which consists of the three stages of segmentation, feature extraction and classification. Each processing stage is discussed below.
Fig. 2. Medicinal plant species classification system
3.1 The Chan-Vese Segmentation Algorithm
Image quality may vary, depending on image acquisition devices and how the image is captured. The segmentation process distinguishes the leaf image from its background. All the images captured in the Medicinal Leaf Database depict a transparent background. Background removal is indispensable to the process because the LCFN of the PCNN works, based on the intensity of the image pixels. Interference is a problem in the LCFN since it includes background pixels in its 1D time sequence. Consequently, the region-growing active contour model [18], [19], [20] is applied to separate the target medicinal leaves from their background by moving the initial contour of the leaf image towards the boundary of the object.The region growing active contour algorithm ([21], [22]) is defined in (1)
\(\begin{aligned} F\left(c_{1}, c_{2}, C\right)=& \mu \text {.length }(C)+v \text {. Area }(\text { inside }(c)) \\ &+\lambda_{1} \int_{\text {inside }(C)}\left|\varphi_{0}(x, y)-c_{1}(x, y)\right|^{2} d x d y \\ &+\lambda_{2} \int_{\text {outside }(C)}\left|\varphi_{0}(x, y)-c_{2}(x, y)\right|^{2} d x d y \end{aligned}\) (1)
In this algorithm, µ (smoothing factor) is a scaling factor that should be small because the herbal database contains clear leaf images with simple backgrounds. Here, 𝜑0 is a given image, C the evolving contour of the image, c1 the average value of 𝜑0 where 𝜑 ≥ 0, and c2 the average value of 𝜑0 outside the given image where 𝜑 ≤ 0. 𝜇 is the smoothing factor and v the contraction bias. The variable ‘Area’ is the area of curve C and λ is the fit weight. The constants λ1 and λ2 are fit weights, and the values of λ1 and λ2 are assigned 1. The Chan-Vese parameter setting is explained below.
3.1.1 Parameters used in the Chan-Vese Algorithm
1. Initial contour: An initial circular, square or rectangular-shaped mask is assigned close to the image boundary. In our work, the initial mask is square-shaped because of the differently shaped leaves. The initial contour mask should be adjacent to the image boundary.
2. Contraction bias (v): The contraction bias decides the orientation of the initial contour, and the direction of movement depends on the initial contour, that is, the contour shifts either inside or outside the square. If the initial contour mask is close to the image boundary, the negative values of the contraction bias expand the initial contour until it reaches the image boundary. If the initial contour is larger than the boundary, we set the contraction bias as positive. The contraction bias shrinks the contour inside the image boundary. The value of the contraction bias is between -1 and +1.
3. Smoothing factor (𝜇): The smoothing factor provides exceptional details of the leaf boundary. A large smoothing factor produces fine details of the leaf image boundary, while a small one results in irregular boundaries. In leaf image segmentation, we need an irregular leaf boundary; otherwise, all the leaf images look identical.
4. Iteration number: This parameter produces boundary convergence details. If the initial contour is close to the image boundary, the number of iterations is equal to the image width or height. If the initial contour is far from the boundary, the number of iterations is high.
\(\text { Evolving }(C)=\text { Inside }(C)+\text { Outside }(C)\) (2)
\(F(C)=F_{1}\left(c_{1}\right)+F_{2}\left(c_{2}\right)\) (3)
\(\text { where } F_{1}\left(c_{1}\right)=\lambda_{1} \int_{\text {inside }(C)}\left|\varphi_{0}(x, y)-c_{1}(x, y)\right|^{2} d x d y\) (4)
\(F_{2}\left(c_{2}\right)=\lambda_{2} \int_{\text {outside }(C)}\left|\varphi_{0}(x, y)-c_{2}(x, y)\right|^{2} d x d y\) (5)
Fig. 3(a) shows the input image with a pixel size of 800 × 600 × 3 and Fig. 3(b) the grayscale image with a pixel size of 800 ×600. The contour evolution is executed according to Equations (2) and (3). The initial contour, sized [200, 200, 200, 200], is close to the image boundary (Fig. 3(c)). The contraction bias, which is set at -0.1, expands the initial contour close to the herbal leaf image boundary, as in Fig. 3(d).-3(f). The smoothing factor is 0.5, because irregular leaf margin details are needed to differentiate between the herbal leaves (Fig. 3(d)- 3(f)). The number of iterations is 600, which is equal to the width (Fig. 3(f)), because the initial contour is close to the image boundary. Fig. 3(d) - 3(f) shows the movement of the initial contour, according to Equations (4) and (5). Though the contour is inside the image, the position of the contour should satisfy the condition (F1(c1)=0 and F2(c2)>0), that is, the area of the outside contour of the image boundary is greater than zero while that of the inside contour of the same is equal to zero. This process is repeated until the curve "C" reaches the image boundary at equilibrium, which is F1(c1)=0 and F2(c2)=0, and continues until it reaches the image boundary. Fig. 3(g) shows the segmented image.
Fig. 3. Evolving contour ‘C’ using the Chan-Vese segmentation algorithm
3.2. Feature Extraction
Images are collections of pixels. Each image has multiple pixels, depending on the width and height of the image. The Chan-Vese region growing active contour segmentation algorithm segments the leaf image from the background. The segmented image cannot be used directly because it contains a significant number of pixels, and all the pixel information needed to classify the herbs does not need to be stored. The distinct properties of the herbal leaf image, such as shape and texture, as well as geometrical and morphological features, are used. We use the LCFN model of the PCNN to extract various features of the herbs, not perceived by humans, in each iteration.
3.2.1. Leaky Capacitor- Fired Neuron
We propose a new LCFN with the pulse-coupled neural network (PCNN). The PCNN was first proposed by [23] and derived from a pig's visual cortex model. The original model used a leaky integrator synapse, but we use a leaky capacitance model that simplifies the design to improve firing speed. The feeding input of the synapse is given in (6). Here, the input is divided with the initial ‘n’ at a constant rate to speed up the neuron's firing.
\(S_{i j}(n)=S_{i j}(n-1) / n\) (6)
𝑆ij(n) denotes a change in the leaky capacitor at the nth instant of the time involved. The initial value, 𝑆ij(0), is the value of the pixel at the 0th location, and represents the capacitor’s initial charge, that is, the value at the 0th iteration, which is Sij(max). The neuron receives an input signal through the feeding synapse. The linking synapse connects a neuron to its 8 neighbors and modifies the output signal based on the values of the neighboring pixels. Thus, a neuron's internal activation unit contains a leaky capacitor that discharges through a constant circuit and a linking input. The internal activation unit is shown in (7).
\(U_{i j}(n)=f U i j(n-1)+\sum_{i j} M_{k l} Y_{i j}(n-1)+S_{i j}(n-1)\) (7)
𝑈ij(𝑛) denotes the output of the internal activation unit, of which the constant, ‘f’, is the linear decay.
The ‘k’ and ‘l’ values are indices of the neighboring neurons and 𝑀kl is the weight value assigned to the neuron’s output, 𝑌ij.
The dynamic threshold value, 𝑇ij, of the neuron positioned at the i, j indices decreases with the decay time constant, ‘g’, in each iteration. The dynamic threshold equation is shown in (8).
\(T_{i j}(n)=g T_{i j}(n-1)+h Y_{i j}(n-1)\) (8)
The constant, ‘h’, is a threshold that decides the neuron’s output for setting the current threshold.
When a neuron’s internal activation unit, 𝑈ij(𝑛), exceeds the dynamic threshold, 𝑇ij(𝑛), the neuron fires and an output pulse, 𝑌ij(𝑛), is produced. The generation of the pulse by each neuron is given by (9)
\(Y_{i j}(n)= \begin{cases}1 & \text { if } U_{i j}(n)>T_{i j}(n) \\ 0 & \text { otherwise }\end{cases}\) (9)
Equation (10) gives the time sequence of the image obtained with the optimum value network parameters.
\(G(n)=\sum_{i j} Y_{i j}(n)\) (10)
Here G(n) represents the time sequence of the pulse image signature. The LCFN generates pulse images based on the time (iterations), and the pulse images display segments, edges and textures from the input image. This network produces better results than others, though computation time is extended on account of its sequential nature. The LCFN model supports translation, scale and rotation-invariant features. Each neuron group with different neurons depends on the image patches, and since the time sequence contains the amplitude and frequency, the need to apply normalization techniques is dispensed with.The geometrical content of the image and its intensity is encoded in the time signal [24]. The time sequence is object-specific, and two-dimensional images can be represented as a 1-dimensional signature, thereby reducing the memory of the image data and facilitating its processing as far as possible.
3.2.2 Analysis of LCFN Parameters and their Features
Initially, we set the membrane potential at (Uij(0))=0 and the threshold at θij(0)=1. When Uij > θij, the resultant action potential is Yij=1 and, thus, f>g. That is, to ensure a larger Uij decay, the dynamic threshold is required to drop to θij and the decay constants (f and g) should be less than 1 to produce the step input. The neuron firing rate of the proposed system is linked to pixel brightness. A neuron is triggered, based on its unique input and the state of the neighboring neurons. If a single neuron fires, it triggers the entire group and determines the different input image regions. With the increasing value of iteration "n, " the value of the dynamic threshold (θij) decreases, and the value of h must be larger for all the neurons in the herbal leaf to be fired at once. Hence, 'g' and 'f' should be set between 0<' f', ' g' <1. The action potential, Yij, rises with the increasing value of the threshold, (h). However, for this experiment, the values of ‘f, ’ ‘g, ’ and ‘h’ are set at 0.7, 0.5 and 18, respectively. These settings produce good feature extraction results, as shown in Fig. 4(a) and Fig. 4(b). Each iteration fires different groups of neurons and hence extracts different features.
Fig. 4. (a). Original Image
Fig. 4. (b). Pulse Images Obtained by LCFN Model when f=0.7, g=0.6, and h=18
The binary images extracted contain feature information about the leaf. The features extracted by the NLIF at different iterations are also shown in Fig. 4(b). The binary image in iteration (n)=3 extracts the contours of the leaves. However, the iteration number that extracts contours for all the leaves need not be 3. For some leaf images, veins may appear in certain iterations. The vein details of the leaf are extracted in iteration 4, as shown in Fig. 4(b) (ii), when f =0.7, g=0.6 and h=18. Similarly, other iterations extract features that cannot be easily interpreted by humans. We select time sequence features based on the firing mechanism and the grouping of white pixels. The previously used algorithm [1-4] in leaf species classification extracts the shape, vein, tip, base and margin features separately.
It is an extremely tedious process to establish, approximately, the venation pattern or leaf base. However, time sequence features automatically generate images of the next pulse, based on the neighboring pixel predictions of the neuron. Therefore, it automatically generates the next feature, which makes it easy to identify the finer details of the leaf. Time sequencing facilitates the continuous processing of input stimuli and requires little time for feature computation.
A smaller leaf of the same species produces a binary image with a smaller number of pixels. In every iteration, we compute the total number of white pixels as the feature and normalize it to the leaf area, as in (11).
\(G(n)=\sum_{i j} \frac{\left(Y_{i j}(n)\right)}{\text { area of leaf in pixels }}\) (11)
Here, G(n) represents the normalized value of the feature obtained in iteration “n.” A sequence of such values obtained in consecutive iterations forms the time sequence of the specific leaf and is the feature vector produced by the LCFN for the particular leaf object.
3.2.3 Features Extracted by the LCFN
Symmetries in the receptive field of any two-portion weights are the same, and the exchange of the receptive field weight values does not affect input image changes. The neuron's output depends on the internal activity (Uij) of the LCFN, and internal activity is a function of linking inputs. If the image change depends on the translation, rotation and scale values, the neuron’s internal activity does not change and the output of the neuron is invariant against the changes. In translational invariance, every neuron utilizes the same weight pattern. In rotational invariance, the weight pattern changes to the circularly symmetric. In scale invariance, the weights are arranged according to the inverse square rule. The intensity of the image pixel is constant, affecting only scale changes in the original image. Accordingly, we align the weights as circularly symmetric, with the radius r=2. The intensity of the image pixel is constant, affecting only scale changes in the original image. Accordingly, we align the weights as circularly symmetric, with the radius r=2. Fig. 5 shows the same time sequence in translation, rotation and variance in scaling.
Fig. 5. Translation, rotation and scale invariance
Fig. 6 shows, graphically, the time sequences obtained by our method for the leaf images of the three different species studied, as well as the differences in the features extracted between inter-species leaves.The model can distinguish between leaves of different species even though their shapes or characteristics are almost similar. Fig. 7 shows the time sequence obtained by our method for leaf images of three betel species. Though the areas of the leaves are different, their shapes are similar and they belong to the same species. The graph makes it clear that the values are the same and there is little difference over large parts of the graph, indicative of the similarity between intra-species leaves.
Fig. 6. Interspecies features of mango, insulin herb and nilavembu leaves
Fig. 7. Intraspecies similarity of betel leaves
Species recognition time includes the time taken to segment the leaf, extract its features and classify it. Our system combines the three processes. The time taken for a leaf to be recognized is 0.34788 seconds with 10 LCFN features in use, 0.61669 seconds with 20 and 0.92401 seconds with 30. The resultant increased time is because of the time taken for each LCFN iteration. When more features are used, more iterations are required, increasing recognition accuracy. Section 4.2 discusses the recognition accuracy obtained with various combinations of features.
3.2.4. Geometrical Features
We fuse LCFN features with geometrical shape features ([25], [26], [27]) such as area, diameter, perimeter, major axis length, minor axis length, Feret diameter, shape factor, aspect ratio, compactness, rectangularity and eccentricity. The definitions and computational methods are shown in Table 2. The geometrical features are extracted and evaluated with the K-NN, SVM, ensemble and BCNN classifiers.
Table 2. Geometrical features
4. Results and Discussions
The Indian herb classification model was evaluated by means of various segmentation quality and classification metrics. The quality metrics are explained in this section.
4.1 Segmenting Herbs using the Chan-Vese Segmentation Method
The Chan-Vese segmentation results on the herbal leaves studied were analyzed through the overlapping measure between two segments, the pair counting-based metric and the information theoretics-based metric. The quality metrics used were the global consistency error (GCE), Rand index (RI) and the variation of information (VOI).
GCE: The GCE is an error measure between the region of the original image segment (R(S original)) and the region of the ground truth segment (R(S ground truth)). The error should be minimum in the particular pixel region (p). Equation (12) measures the error between the two segments.
\(G C E\left(S_{\text {original }}, S_{\text {groundtruth }}, p\right)=\frac{\left|R\left(S_{\text {original }}, p\right) \backslash R\left(S_{\text {ground truth }}, p\right)\right|}{\left|R\left(S_{\text {groundtruth }}, p\right)\right|}\) (12)
RI: The RI is a measure of similarity between two groups of clusters, that is, between the original segmented image (S original) and the ground truth image (S ground truth). Equation (13) measures the similarity between the two segments. The RI should be at a maximum.
\(R I\left(S_{\text {original }}, S_{\text {ground truth }}\right)=\frac{a+b}{a+b+c+d}\) (13)
where a is the measure of the cardinality of the pixels, pi and pj, placed in the same subset of the two segments, S original and S ground truth; b is the measure of the cardinality of the pixels, pi and pj, placed in the same subset in the partition S ground truth, and the pixels (pi, pj) are located in different subsets in the partition (S original); c is the measure of the cardinality of the pixels, pi and pj, located in the same subset in the partition (S original) and the pixels (pi, pj) are laid in different subsets in the partition S ground truth, and d is the measure of the cardinality of the pixels (pi, pj) positioned in different subsets of the two partitions, S original, and S ground truth.
VOI: The VOI is a measure of the amount of information lost or gained between the two segments, S original and S ground truth. Equation (13) measures the VOI between the two segments.
\(\begin{aligned} &\operatorname{VOI}\left(S_{\text {groundtruth }}, S_{\text {original }}\right) \\ &\qquad \begin{aligned} &=\text { Entropy }\left(S_{\text {ground truth }}\right)+\text { Entropy }\left(S_{\text {original }}\right) \\ &-2 \text { Mutual Information }\left(S_{\text {ground truth }}, S_{\text {original }}\right) \end{aligned} \end{aligned}\) (13)
Table 3 shows the quality metrics used in the segmentation of the herbal leaves. The Chan- Vese segmentation algorithm produces minimum error, maximum similarity between clusters and minimal information gain or loss between the two segments. This analysis makes it clear that the region growing active contour segmentation algorithm segments medicinal leaves perfectly.
Table 3. Segmentation quality metrics of the different herbs
4.2 A Classification Performance Analysis of Indian Medicinal Leaves
The Indian Medicinal Leaf Species Dataset we created comprises a total of 635 images. Given the limited number of images in each species, we applied data augmentation techniques to increase the number from 635 to 2550. In all, 70% of the images were used for training, 10% for validation and the remaining 20% for testing. The LCFN features are rotation-, scale- and transform-invariant. Data augmentation, however, helps recognize leaves that have portions missing. Shape features were concatenated with 30 LCFN features and fed to the BRNN classifier to obtain 91.19% maximum classification accuracy for testing, as shown in Table 4. Some preprocessed images were not segmented perfectly, owing to light and shadows. Such images were not classified correctly and produced false negatives that reduced the average accuracy.
Table 4. Percentage of classification accuracy of Indian herbs
The Indian herbs were classified using the Bayesian regularization neural network (BRNN). We applied 5-fold BRNN cross-validation and tested a range of input feature vectors with a number of hidden neurons. We obtained the highest recognition accuracy for a combination of 30 hidden neurons and 41 input feature vectors (30 LCFN+11 shape features), as shown in Fig. 7. The performance of the proposed system is evaluated through the metrics of accuracy, precision, true-positive rate (TPR), false-positive rate (FPR), Kappa (K) and Matthew’s correlation coefficient with its confusion matrix (Fig. 8).
Fig. 7. Classification accuracy of Indian herbs with different hidden neurons
Fig. 8. Confusion matrix for Indian herbs
Table 5 shows a statistical analysis of the 5-fold neural network classifier. We conclude that our proposed model has produced the highest recognition accuracy of more than 80% for aloe, betel, bittergourd, grape, guava, mango, nithyakalyani, Indian borage, China rose, sweet basil and pomegranate. Our model has produced a high true positive rate, that is, it correctly classifies all the samples. It classified herbs such as aloe, bittergourd, guava, nilavembu, Indian borage, China rose, sweet basil and pomegranate perfectly. Cohen’s kappa is equal to 1 compared to other herbs, and Mattthew’s correlation coefficient proves that our model incorrectly predicts the insulin herb and nilavembu leaves.
Table 5. Statistical analysis of the 5-fold neural network classifier
4.3. A Classification Performance Analysis of Herbs from the Flavia Database
The Flavia database contains 32 Chinese species, each comprising 50-77 images captured with a highly constrained white background. The images were downloaded from https://sourceforge.net/projects/flavia. We selected 32 species of medicinal leaves and tested 1907 images with no data augmentation techniques. In all, 70% of the images were used for training, 10% for validation and 20% for testing and obtained the highest recognition accuracy of 82% during testing. Our test results were compared with the previous leaf recognition model and found to be not promising. This is because all the images are sample single leaf images of the same size and orientation. Their similarity is, consequently, very high and there is no geometric invariance in the images. Given that our proposed model supports geometrically invariant data, we have changed the image size and orientation to obtain the highest classification accuracy of 97.34%. Table 6(a) shows classification accuracy on the Flavia database with the various k values of the nearest neighbor classifier which, when k=10, produces the highest classification accuracy of 90.36%. Table 6(b) depicts the classification accuracy of the different classifiers and Table 7 their accuracy on the Flavia database. Classification accuracy [26] for shape and color features was 96%, though the leaf color was found unsuitable for classification.
Table 6(a). Classification accuracy of the different K values of the KNN
Table 6(b). Classification accuracy of the different classifiers
Table 7. Classification accuracy on the Flavia database
5. Conclusion
This paper has reviewed several plant species classification systems. All the methods used the leaf as the base to classify plant species owing to its two-dimensional nature and availability. All the methods used the leaf as the base to classify plant species owing to its two-dimensional nature and availability. The proposed model handles angular and different-sized images. The model contains only a linking field, and the linking inputs connect faster than the time needed for the decay constant (h) parameter. We tested our system with freshly collected leaves from herbal plants. The results showed that our method avoids normalization and dimension reduction and solves the problem of translation, rotation and scale invariance. Compared to the basic pulse-coupled neural network, the LFCN implemented in this paper uses only three parameters. Though the features extracted depend on the parameter valuethrough the final classification accuracy is influenced by the classifier. Consequently, we tested our method with different classifiers to find that the BRNN provides maximum accuracy. In our future work, the proposed system will be analyzed with an adaptive parameter setting method for the LCFN, depending on image characteristics, through the application of intelligent optimization techniques.
References
- A. N. M. Alamgir, "Pharmacognostical Botany: Classification of medicinal and aromatic plants (MAPs), botanical taxonomy, morphology, and anatomy of drug plants," Therapeutic Use of Medicinal Plants and Their Extracts: Volume 1, vol. 73, pp. 177-293, 2017. https://doi.org/10.1007/978-3-319-63862-1_6
- A. A. Jayeola, "Micromorphological study of plant fragments in some powdered medicinal plants," J. Med. Plants Res., vol. 3, no. 5, pp. 438-442, 2009.
- D. C. D. C.H.Arun, "Identifying Medicinal Plant Leaves Using Textures and Optimal Colour spaces Channel," J. Comput. Sci. Inf., vol. 10, no. 1, pp. 19-28, 2017.
- Y. G. Naresh and H. S. Nagendraswamy, "Classification of medicinal plants: An approach using modified LBP with symbolic representation," Neurocomputing, vol. 173, pp. 1789-1797, 2016. https://doi.org/10.1016/j.neucom.2015.08.090
- D. Shitole, F. Tamboli, and K. M., R. K. Raj, "Ayurvedic Herb Detection using Image Processing," Int. J. Trend Sci. Res. Dev., vol. Vol. 3, no. 4, pp. 491-494, 2019.
- J. R. Xue et al., "Automated Chinese medicinal plants classification based on machine learning using leaf morpho-colorimetry, fractal dimension and visible/near infrared spectroscopy," Int. J. Agric. Biol. Eng., vol. 12, no. 2, pp. 123-131, 2019. https://doi.org/10.25165/j.ijabe.20191202.4637
- L. E. Hamid and S. A. R. Al-Haddad, "Automated leaf alignment and partial shape feature extraction for plant leaf classification," Electron. Lett. Comput. Vis. Image Anal., vol. 18, no. 1, pp. 37-51, 2019.
- A. Aakif and M. F. Khan, "Automatic classification of plants based on their leaves," Biosyst. Eng., vol. 139, pp. 66-75, 2015. https://doi.org/10.1016/j.biosystemseng.2015.08.003
- J. Chaki, R. Parekh, and S. Bhattacharya, "Plant leaf classification using multiple descriptors: A hierarchical approach," J. King Saud Univ. - Comput. Inf. Sci., vol. 32, no. 10, pp. 1158-1172, 2020.
- J. Y. CLARK, D. P. A. CORNEY, and P. WILKIN, "Leaf-based Automated Species Classification Using Image Processing and Neural Networks," Biological Shape Analysis, pp. 29-56, 2017.
- H. Wang, D. Tian, C. Li, Y. Tian, and H. Zhou, "Plant leaf tooth feature extraction," PLoS One, vol. 14, no. 2, Feb. 2019.
- M. M. S. de Souza, F. N. S. Medeiros, G. L. B. Ramalho, I. C. de Paula, and I. N. S. Oliveira, "Evolutionary optimization of a multiscale descriptor for leaf shape analysis," Expert Syst. Appl., vol. 63, pp. 375-385, Nov. 2016. https://doi.org/10.1016/j.eswa.2016.07.016
- H. Kolivand, B. M. Fern, T. Saba, M. S. M. Rahim, and A. Rehman, "A New Leaf Venation Detection Technique for Plant Species Classification," Arab. J. Sci. Eng., vol. 44, no. 4, pp. 3315-3327, 2019. https://doi.org/10.1007/s13369-018-3504-8
- M. Turkoglu and D. Hanbay, "Leaf-based plant species recognition based on improved local binary pattern and extreme learning machine," Phys. A Stat. Mech. its Appl., vol. 527, 2019.
- S. Zhang, C. Zhang, and X. Wang, "Plant species recognition based on global-local maximum margin discriminant projection," Knowledge-Based Syst., vol. 200, p. 105998, 2020,
- X. Deng, C. Yan, and Y. Ma, "PCNN Mechanism and its Parameter Settings," IEEE Trans. Neural Networks Learn. Syst., vol. 31, no. 2, pp. 488-501, 2020. https://doi.org/10.1109/tnnls.2019.2905113
- K.K.Thyagharajan, I.Kiruba Raji, "A Review of Visual Descriptors and Classification Techniques used in Leaf Spieces Identification," Archieves of Computational Methods in Engineering, vol. 26, no. 4, pp. 933-960, 2019. https://doi.org/10.1007/s11831-018-9266-3
- T. F. Chan and L. A. Vese, "Active Contours Without Edges," IEEE Transactions on Image Processing, vol. 10, no. 2, pp. 266-277, 2001. . https://doi.org/10.1109/83.902291
- X. F. Wang, D. S. Huang, and H. Xu, "An efficient local Chan-Vese model for image segmentation," Pattern Recognit., vol. 43, no. 3, pp. 603-618, 2010. https://doi.org/10.1016/j.patcog.2009.08.002
- I. Kiruba Rajia and K. K. Thyagharajan, "An analysis of segmentation techniques to identify herbal leaves from complex background," Procedia Comput. Sci., vol. 48, no. C, pp. 589-599, 2015. https://doi.org/10.1016/j.procs.2015.04.140
- Y. Yuan, Y.-W. Chen, C. Dong, H. Yu, and Z. Zhu, "Hybrid method combining superpixel, random walk and active contour model for fast and accurate liver segmentation," Comput. Med. Imaging Graph., vol. 70, pp. 119-134, Dec. 2018. https://doi.org/10.1016/j.compmedimag.2018.08.012
- P. Getreuer, "Chan-Vese Segmentation," Image Process. Line, vol. 2, pp. 214-224, 2012. https://doi.org/10.5201/ipol.2012.g-cv
- R. Eckhorn, H. J. Reitboeck, M. Arndt, and P. Dicke, "Feature Linking via Synchronization among Distributed Assemblies: Simulations of Results from Cat Visual Cortex," Neural Comput., vol. 2, no. 3, pp. 293-307, 1990. https://doi.org/10.1162/neco.1990.2.3.293
- J. L. Johnson, "Pulse-coupled neural nets: translation, rotation, scale, distortion, and intensity signal invariance for images," Appl. Opt., vol. 33, no. 26, p. 6239, Sep. 1994. https://doi.org/10.1364/AO.33.006239
- C. A. Priya, T. Balasaravanan, and A. S. Thanamani, "An efficient leaf recognition algorithm for plant classification using support vector machine," in Proc. of Int. Conf. Pattern Recognition, Informatics Med. Eng. PRIME 2012, pp. 428-432, 2012.
- A. Caglayan, O. Guclu, and A. B. Can, "A plant recognition approach using shape and color features in leaf images," Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 8157 LNCS, no. PART 2, pp. 161-170, 2013.
- J. Hossain and M. A. Amin, "Leaf shape identification based plant biometrics," in Proc. of 2010 13th Int. Conf. Comput. Inf. Technol. ICCIT 2010, no. Iccit, pp. 458-463, 2010.