Rapid Determination of Ginkgolic Acids in Ginkgo biloba Leaf Using Online Column Switching High-Performance Liquid Chromatography-Diode Array Detection and Confirmation by Liquid Chromatography-tandem Mass Spectrometry

  • Received : 2013.07.29
  • Accepted : 2013.09.12
  • Published : 2013.12.20


In this study, an improved method for the quantitative analysis of ginkgolic acids (GAs) in Ginkgo biloba leaf extract was developed. The samples were extracted with a mixture of chloroform and 50 % ethanol, after which the chloroform extract was dried and reconstituted in methanol. GAs with 13:0, 15:1, and 17:1 in the extract were successfully separated within 40 min and determined with high throughput performance using an online column-switching HPLC method using an SP column C8 SG80 ($4.6{\times}150mm$, $5{\mu}m$) and a Cadenza 5CD C18 column ($4.6{\times}150mm$, $3{\mu}m$). The developed HPLC method was validated for Ginkgo biloba leaf extract. The validation parameters were specificity, linearity, precision, accuracy, and limits of detection and quantitation (LODs and LOQs, respectively). It was found that all of the calibration curves showed good linearity ($r^2$ > 0.9993) within the tested ranges. The LODs and LOQs were all lower than $0.04{\mu}g/mL$. The established method was found to be simple, rapid, and high throughput for the quantitative analysis of GAs in ten commercial Ginkgo biloba leaf extract and dietary supplements. The samples were also analyzed in LC-electrospray ionization (ESI) tandem mass spectrometry (MS/MS) - multiple-ion reaction monitoring (MRM) mode to confirm the identification results that were obtained by the column switching HPLC-DAD method. The developed method is considered to be suitable for the routine quality control and safety assurance of Ginkgo biloba leaf extract.


Ginkgo biloba leaf extract is well recognized as an ingredient in various pharmaceuticals and dietary supplements in America, Europe, and Asia owing to the many types of bioactive constituents it contains.1,2 Flavonol glycosides, terpene trilactones, ginkgolides, and proanthocyanidins are included in the extract, and they contribute to improved cognition and memory. They also exert therapeutic effects against cerebrovascular disease, peripheral vascular disease, sexual dysfunction, and affective.3,4 However, alkylphenols (ginkgolic acids, ginkgols, and bilobols) contained in the extract are categorized as hazardous constituents because they have allergenic, cytotoxic, mutagenic, and slight neurotoxic properties.5,6 In particular, draft monographs by the 1999 US and European Pharmacopeias recommend that ginkgolic acids (GAs) be limited to 5 ppm, and this recommendation is followed by most manufacturers.7,8

Several analytical approaches have been reported for the determination of GAs in Ginkgo biloba leaf extract.9-11 The most common approach to analyze GAs in Ginkgo biloba leaf involves extraction followed by a multistep purification using solid-phase extraction (SPE) with various adsorbents and liquid-liquid extraction. In these previous studies, the difficulties and limits of quantitative analysis for GAs in Ginkgo biloba have been reported. The main analytical problems in the determination of GAs are their low concentrations in Ginkgo biloba and the complicated matrix of the sample. In most cases, the detection of trace amounts of GAs can be easily masked by the high-concentration of plant constituents; thus, the analytes of interest remain undetected. It is therefore important to develop a rapid analytical method that is suitable for the precise quantification of these substances at the low levels in which they are found in samples.

There have been numerous studies for the analysis of hazardous constituents in ginkgo leaf extract. Approximately 400 relevant studies have been published since 2001. In addition, several analytical methods, such as HPLC,12,13 GC,14 TLC,15 NMR,16,17 and ELISA18 have been introduced. Recently, the use of LC/MS and NMR assays for GA analysis has been found to be accurate, but expensive and rather difficult to subject to routine quality control.19-23 In addition, GC/MS assays are complicated because they require a tedious sample preparation procedure, including multistep cleanup and chemical derivatization.24 Although the HPLC methods that are combined with various detectors have been found to be appropriate in the analysis of GAs, they are only applicable for a very limited range of applications due to the low concentration of GAs and the sample matrix effect. For sample cleanup, liquid-liquid extraction, solid-phase extraction or column chromatography is necessarily required. Such sample cleanup methods could be un favorable choices for the routine analysis of GAs. In this study, a column switching HPLC method as an alternative solution was applied to significantly simplify the analytical procedure and enable it to be automate routine GA analysis.

Moreover, the identification of trace GAs is a very difficult task due to the significant matrix effect of Ginkgo biloba extract. LC-electrospray ionization (ESI)-MS/MS techniques have been applied for the accurate identification and quantification of several compounds in complex plant extracts.25,26 Recently, UHPLC-MS/MS has been successfully applied for the determination of many compounds in Ginkgo biloba leaf.27In particular, multiple-ion reaction monitoring (MRM) mode can greatly increase the sensitivity and selectivity of target analytes, even in complicated sample matrices. The combination of column switching HPLC-DAD and LC-MS/ MS provides powerful information on the quality control and confirmation of GAs in Ginkgo leaf extract.

In this study, a rapid and simple column switching HPLC method was established to reduce the matrix effect and enhance the detection sensitivity of GAs. The developed method was validated in terms of linearity, precision, accuracy, limits of detection (LODs), and limits of quantitation (LOQs). The validated method was applied to an assay of 10 samples from Ginkgo biloba extract and supplements. In addition, all of the samples were analyzed by LCESI- MS/MS to confirm the GAs data that was obtained by the column switching HPLC method.


Materials and Methods

Solvents and Reagents. HPLC-grade chloroform, ethanol, and methanol were obtained from Duksan (Kyunggi, Korea), and acetonitrile (ACN) was purchased from SK chemical (Daejeon, Korea). Ultra-pure water (18.2 MΩ) was prepared using a TKA purification system (Germany), and phosphoric acid was obtained from Junsei (Japan). All of the solvents used for HPLC were filtered through a 0.45 μm nylon membrane filter (Whatman) and degassed prior to use.

Standard Preparation. Standard ginkgolic acid I and II were purchased from Chromadex (Irvine, CA). In ginkgolic acid I (Chromadex 00007165), ginkgolic acids with C13:0 and C15:1 were combined, but only C15:1 had accurate purity information (86.8%). Therefore, the peak retention times of ginkgolic acid with C13:0 and C15:1 were verified using standard ginkgolic acid I, while ginkgolic acid with C15:1 was used to quantify ginkgolic acid C13:0 and C15:1. Ginkgolic acid II (Chromadex 00007170) contains C17:1, and its purity was 96.4%. The chemical structures of the GAs are indicated in Figure 1.

The stock solution of each compound was prepared as follows: 5-10 mg standard was accurately weighed and then placed into a 100 mL volumetric flask. Next, methanol was added and the solution was diluted to volume with the same solvent. Each stock solution was stored at 4 ℃ and the gradient dilutions were made with methanol to produce working stock solutions with concentrations of 0.01-10.0 μg/mL.

Figure 1.Chemical structures of GAs investigated in this study.

Figure 2.Analytical procedure for the determination of GAs in Ginkgo biloba by HPLC-DAD and LC-ESI-MS/MS.

Sample Preparation. The sample used in this study was Ginkgo biloba leaf extract that was obtained from Naturex (Avignon, France), which was extracted with 60% ethanol. The content of flavonol glycoside in the extract was 24%. Briefly, a 1 g aliquot of Ginkgo biloba leaf extract was dissolved in 50 mL 50% ethanol. The solution was then extracted with 50 mL chloroform, after which the chloroform extracts were transferred to a round bottom flask. The aqueous layer was subsequently re-extracted with 10 mL ethanol and 50 mL chloroform. After the samples were combined with chloroform, the extracts were evaporated to dryness. The dry residue was reconstituted in 5 mL methanol, and it was filtered through a 0.45 μm syringe filter prior to analysis (Figure 2).

Instrumental Conditions.

HPLC Conditions: Chromatographic separation was accomplished using a SPLC (Shiseido, Japan) that consisted of a pump (SP 3101), autosampler (SP 3133), PDA detector (SP 3017), column oven (SP 3004), degasser (SP 3010), and switching valve (SP 3011). The HPLC system was controlled by EZChromelite. Analytes were separated using a columnswitching HPLC system. The precolumn was a SP MF C8 SG80 (4.6 × 150 mm, i.d.; particle size, 5 μm, Shiseido), and the separation column was a Cadenza 5CD C18 (4.6 × 150 mm, i.d.; particle size, 3 μm, Imtakt). The on-line dual column HPLC system is depicted in Figure 3. The gradient elution program consisted of a mobile phase that was composed of water containing 0.1% phosphoric acid (A), acetonitrile (B), and 70% acetonitrile containing 0.1% phosphoric acid. The gradient program used is shown in Table 1. The flow rate was 1 mL/min and the temperature of the column oven was 40 ℃. The wavelength used for UV detection was 210 nm and the injection volume was 50 μL.

Figure 3.On-line dual column (switching) HPLC system.

Table 1.Solvent A was water containing 0.1% phosphoric acid, Solvent B was ACN, and Solvent C was 70% ACN containing 0.1% phosphoric acid. Solvents A and B were linked to a dual pump and connected to the separation column, and Solvent C was linked to a single pump that was connected to the pre-column. The valve switching time was 2.9 to 4.0 min.

LC/MS/MS Conditions. Mass spectrometric analyses were conducted on an Agilent 6460 triple quadrupole mass spectrometer using an Agilent 1290 infinity series HPLC composed of a 1290 binary pump (G4220B), 1290 auto sampler (G4226A), and 1290 thermostat column compartment (G1316C). Instrument control was conducted using Mass Hunter. Analytes were detected using a triple quadrupole mass spectrometry detector. Separation was conducted on a Unison UK-C18 column (100 × 2 mm, i.d.; particle size, 3 μm, Imtakt). The gradient elution program consisted of a mobile phase that was composed of water containing 0.1% formic acid (A) and acetonitrile (B). The gradient started with 30% A and 70% B. During the first minute, the composition of B was maintained at 70%. From 1 to 10 min the composition of B was increased to 95%, where it was maintained from 10 to 15 min. The composition of B was decreased to 70% at 15.1 min and then allowed to equilibrate to the initial composition until 18 min. The flow rate was set at 0.3 mL/min, the temperature of the column oven was 40 ℃, and the injection volume was 2 μL. The instrument was operated in the ESI negative ion mode and the detection of the ions was conducted in multiple-ion reaction monitoring

Table 2.* Frag: Fragmentor voltage (V). * CE: Collision energies

The optimized mass spectrometric parameters were as follows: the gas temperature was 320 ℃, the gas flow was 9 L min−1, the nebulizer pressure was 45 psi, the sheath gas temperature was 350 ℃, the sheath flow was 11 mL/min, and the capillary voltage was set at 4000 V.

Validation of the Method. The method that was developed for the analysis of Ginkgo biloba leaf extract (Naturax, France) was validated according to the following ICH guidelines28: Validation of analytical procedures: Text and Methodology Q2(R1) (ICH 2005). The specificity, linearity, precision, recovery, and detection and quantitation limits (DLs and QLs) were all investigated.


Results and Discussion

Extraction and LC Analysis of GAs. Sample preparation was conducted according to the method described in the Korean Pharmaceutical Codex (KPC 2007)29 by dissolving 1 g Ginkgo biloba leaf extract in 50% ethanol, and then extracting the GAs twice using chloroform (ICH 2005). After the combined chloroform extracts were evaporated to dryness, the dried residue was reconstituted in 5 mL methanol. The final solution was then filtered through a 0.45 μm syringe prior to analysis.

Due to their low polarity, GAs dissolve easily in organic solvents such as hexane, chloroform, and methanol.8 However, a large amount of interferences were co-extracted when such solvents were used, which interfered with the analysis of GAs by HPLC. As a typical example, the detection of minor GAs in the crude extract was masked by matrix interferences (Figure 4).

Although several organic solvents have been used for the extraction of GAs from plants, the extract should be further purified using liquid-liquid extraction, solid-phase extraction, or column chromatography with various adsorbents due to a large amount of co-extracting interferences. However, these methods that are used for the purification of GAs are unfavorable because of the tedious procedure for routine quality control.

Figure 4.HPLC chromatograms of (a) standard GA mixture and (b) GAs extracted from the samples using an SP C18 UG 120 column. The identified peaks are as follows: 1. GA with 13:0, 2. GA with 15:1, and 3. GA with 17:1.

In this study, a dual column HPLC method was applied for the effective elimination of interferences and the separation GAs from the matrix. A C8 column was used as the precolumn because conventional C18-HPLC is known to be somewhat difficult for the separation of the GAs pairs with C13:0/C15:1 and C15:0/C17:1. An SP column C8 SG80 (4.6 × 150 mm, 5 μm) and an Agilent Zorbax C8 column (4.6 × 150 mm, 5 μm) were tested and compared. The peaks of GA 13:0, 15:1, and 17:1 were well separated within the range of 2.9 to 4.0 minutes on C8 SG80 column. Conversely, the peaks of the three GAs appeared in the 7 to 19 min range on the Zorbax column. Thus, the C8 SG80 column was selected as the pre-column, taking into consideration the analysis time and separation efficiency.

However, the dual column HPLC running time could be significantly lengthened if a 250 mm analytical column is used. To reduce the LC elution time and maintain separation efficiency, a column with a relatively short length and small particle size should be used. A Cadenza 5CD C18 column (4.6 × 150 mm, 3 μm) was selected as a second column. In addition, a gradient elution program was applied with a mobile phase consisting of water containing 0.1% phosphoric acid (A) and ACN (B) to effectively elute GA 13:0 and GA15:1, and to reduce the peak broadening of GA17:1, as shown in Figure 5(a). By using an on-line dual column, three GAs in the extract were successfully separated from the matrix, and interferences could be reduced without applying a purification step (Figure 5(b)).

Figure 5.HPLC chromatograms of (a) standard GA mixture and (b) GAs extracted from sample using dual column. The identified peaks are as follows:1. GA with 13:0, 2. GA with 15:1, and 3. GA with 17:1.

The main UV absorption of GAs has been found to occur at 210 nm and 310 nm. Specifically, the GA analysis wavelength was determined to be 210 nm by the United States and European Pharmacopeias (European Pharmacopoeia Commission 1999),30 but 310 nm by the Deutsches Arzneibuch (DAB).31 At 310 nm, the baseline is more stable because there is no influence of the mobile phase UV cut-off; however, the peak sensitivity at 310 nm is lower than that at 210 nm. Therefore, 210 nm was selected as the detection wavelength for high sensitivity detection of GAs. However, considering the mobile phase UV cut off, acetonitrile should be selected as mobile phase B.

Method Validation.

Linearity: To determine the accuracy of the method, standard calibration curves were prepared by injecting mixed standard solutions at six concentration levels. The peak areas of the standards were then plotted against the concentrations, which were expressed as μg/mL. Linear regression parameters were examined three times over a three day period.

As shown in Table 3, the correlation coefficient was greater than 0.9993, indicating a high correlation between the amount of GAs and the measured concentration. The measured concentration range was 0.01-10.0 μg/mL.

Detection Limit and Quantitation Limit. The limits of detection (LOD) and quantitation (LOQ) were determined based on the standard deviation of the response and the slope, which is one of the LOD and LOQ measurements in the ICH guidelines. The LOD and LOQ were calculated as described by Eq. (1).

Table 3.Calibration curves, correlation coefficients, LODs and LOQs of ginkgolic acids obtained by HPLC-DAD

where σ = the standard deviation of the response, and S = the slope of the calibration curve

To calculate σ, a calibration curve of the sample with GAs that had a concentration similar to the LOD, LOQ was studied. The standard deviation of the y-intercepts of the regression lines was used as σ.

The three levels of Ginkgo biloba leaf extract containing 0.04-1.0 μg/mL GAs were analyzed in triplicate. The LOD and LOQ of GAs were less than 9.9 ng/mL and 39.3 ng/mL, respectively, as indicated in Table 3. These results indicate that the developed method is sufficiently sensitive for the determination of GAs in the Ginkgo biloba leaf extract.

Precision. The repeatability of the method was demonstrated by analyzing six replicates of the Ginkgo biloba leaf extract. In addition, the procedure was repeated eighteen times over a three day period to investigate the intermediate precision. The results of the repeatability and intermediate precision examination are shown in Table 4. Relative standard deviation (RSD, %) of the repeatability ranged from 1.5 to 5.7%, and the intermediate precision ranged from 2.2 to 6.3%. The RSD values were found to be within the acceptable criteria and indicated that the method had relatively high precision.

Accuracy. To determine the accuracy of the method, Ginkgo biloba leaf extract was analyzed in four groups. In three groups, the samples were spiked with a standard solution of known concentration to create a low, middle, and high level (0.2, 0.5, 1.0 μg/mL), while one group was prepared as a reference sample. Each group was then analyzed in triplicate. The percent recovery was calculated by comparing the actual area and the theoretical area of the spiked and unspiked sample and the area of the three standard solutions. The accuracy data of the GAs are listed in Table 5. The percent recovery ranged from 90.2% to 99.4% with 0.4- 4.8% RSD, indicating that this method is reasonably accurate.

Table 4.Precision data of GAs in Ginkgo biloba leaf extract

Table 5.Accuracy data of GAs in Ginkgo biloba leaf extract

Method Application. The developed method was used to analyze ten samples of Ginkgo biloba leaf extract and dietary supplements containing 10-20% Ginkgo leaf extract.

The level of GAs found in the samples is summarized in Table 6. Each GA was quantified at a level of 0.04 μg/mL by an established method. One (#5) of the ten samples was found to have GAs in concentrations higher than the restriction level of 5 ppm.

The GA specificity of the standard and sample was tested by LC/MS/MS and HPLC-DAD. Specifically, the retention time and UV spectra of the GAs in the standard and the sample were checked by HPLC analysis. In some cases, it was difficult to accurately identify the GAs in the sample extract by HPLC due to the presence of similar alkylphenols such as cardanols and urushiols analogues.

For the accurate identification of GAs extracted from Ginkgo biloba leaf, HPLC-ESI-MS/MS was applied in the negative ion mode. The precursor ions of [M-H]− for GAs with C13:0, C15:1, and C17:1 were set as m/z 319, m/z 345, and m/z 373. The product ions of [M-H]− for GAs produced common [M-H-CO2]− and m/z 119 [C6H3COO]− and 133 [C7H5COO]− ions. In MRM experiments, [M-H]− for individual GAs were selected as precursor ions and common product ions were selected for ion transition monitoring. As shown in Figure 5, three GAs in the sample extract were clearly detected by the LC-MS/MS-MRM mode.

Figure 6.TIC and MRM chromatograms of GAs extracted from Ginkgo biloba by LC-ESI-MS/MS.

A specific precursor-product ion transition monitoring in the MRM mode could significantly enhance the analytical sensitivity and selectivity for the analysis of trace amounts of GAs in complex matrices. As can be seen in Figure 5, the sensitivity and selectivity of GAs in the sample extract were greatly improved and no significant interferences were observed. Thus, three GAs with C13:0, C15:1, and C17:1 in the sample extract could be accurately identified by the peak area ratio of individual GAs that were detected in the MRM chromatograms.



This study was conducted to develop a high throughput method of analyzing GAs in Ginkgo biloba leaf extract. An HPLC switching system was used to selectively separate the trace amount of GAs that exists in the leaf extract and to significantly reduce the sample matrix effect. The method was successfully validated in terms of specificity, linearity, precision, recovery, detection and quantitation limit.

The column switching HPLC method was found to be simple, easy, and reproducible for the determination of GAs in the sample extract, and it could be applied for the high throughput analysis of GAs by the elimination of a tedious sample workup. The determination of GAs in sample extract was possible up to 0.04 μg/mL, reflecting the high sensitivity of the developed method. This method was successfully applied for the routine quality control of various samples and supplements containing Ginkgo biloba leaf extract. Moreover, the GAs in the sample extract were successfully confirmed by LC-MS/MS-MRM, providing complementary data for the column switching HPLC results.

Along with existing studies for GA analysis, the developed method is expected to be an alternative solution for the routine quality control of GAs in Ginkgo biloba leaf extract and other commercial supplements.


  1. Kressmann, S.; Müller, W. E.; Blume, H. H. J. Pharm. Pharmacol. 2002, 54(5), 661-669.
  2. McKenna, D. J.; Jones, K.; Hughes, K. Altern. Ther. Health. Med. 2001, 7(5), 70-86, 88-90.
  3. Valli, G.; Giardina, E. G. J. Am. Coll. Cardiol. 2002, 39(7), 1083- 1095.
  4. Mahadevan, S.; Park, Y. J. Food Sci. 2008, 73(1), 14-19.
  5. Farlow, M. R.; Miller, M. L,; Pejovic, V. Dement, Geriatr. Cogn. Disord. 2008, 25, 408-422.
  6. Ware, W. R. Integr. Cancer. Ther. 2009, 8, 22-28.
  7. Van Beek, T. A. J. Chromatogr. A 2002, 967(1), 21-55.
  8. Van Beek, T. A.; Montoro, P. J. Chromatogr. A 2009, 1216(11), 2002-2032.
  9. Ndjoko, K.; Wolfender, J. L.; Hostettmann, K. J. Chromatogr. B Biomed. Sci. Appl. 2000, 744(2), 249-255.
  10. Xia, H.; Wang, X.; Li, L.; Wang, S.; Guo, C.; Liu, Y.; Yu, L.; Jiang, H.; Zeng, S. J. Chromatogr. B 2010 878(28), 2701-2706.
  11. Maciej, A. P.; Kevin, D.; Samantha, S.; Brain, C.; John, J. B. J. Agric. Food Chem. 2011 59, 10879-10886.
  12. Wang, L.; Jia, Y.; Pan, Z.; Mo, W.; Hu, B. J. Anal. Appl. Pyrol. 2009, 85(1-2), 66-71.
  13. Chen, J.; Yan, T.; Xu, T. J. Sep. Sci. 2012, 35, 1094-1101.
  14. Choi, Y. H.; Choi, H. K.; Peltenburg-Looman, A. M.; Lefeber, A. W.; Verpoorte, R. Phytochem. Anal. 2004, 15(5), 325-330.
  15. Gray, D. E.; Messer, D.; Porter, A.; Hefner, B.; Logan, D.; Harris, R. K.; Clark, A. P.; Algaier, J. A.; Overstreet, J. D.; Smith, C. S. J. AOAC Int. 2007, 90(5), 1203-1209.
  16. Sara, A.; Jerzy, W.; Jaroszewski, T.; Robert, V.; Dan, S. Metabolomics 2010, 6, 292-302.
  17. Hiroshi, G.; Toyonobu, U. Phytochem. Anal. 2012, 23(1), 84-87.
  18. ZiZkova, P.; ViskuPi ova, J.; Horakova, L'. Interdiscip. Toxicol. 2010, 3(4), 132-136.
  19. He, J.; Xie, B. J. Chromatogr. A 2002, 943(2), 303-309.
  20. Van Beek, T. A.; Wintermans, M. S. J. Chromatogr. A 2001, 930(1-2), 109-117.
  21. Fuzzati, N.; Pace, R.; Villa, F. Fitoterapia. 2003, 74(3), 247-256.
  22. Yang, X.-M.; Zhang, X.; Chen, Y.; Liu, F. Chromatographia 2009, 69(5-6), 593-596.
  23. Kim, S. H.; Kim, D. H.; Park, J. H.; Oh, M. H.; Cho, C. H.; Baek, J. H.; Cho, J. H.; Kim, T. B.; Lee, K. Y.; Kim, Y. C.; Sung, S. H. J. Pharmacogn. 2008, 39(3), 218-222.
  24. Van Beek, T. A. J. Chromatogr. A 2009, 1216(11), 2002-2032.
  25. Yuan, Y.; Jian, P.; Zeyu, W.; Ailing, H. A. Z. J. Chromatogr. Sci. 2013, 51, 266-272.
  26. Daniela, I. S.; Venera, S.; Victor, D.; Andrei, M. J. Pharm. Biomed. Anal. 2009, 50, 459-468.
  27. Xin, Y.; Guisheng, Z.; Yuping, T.; Zhenhao, L.; Shulan, S.; Dawei, Q.; Jin-Ao, D. Molecules 2013, 18, 3050-3059.
  28. ICH Guideline; Validation of analytical procedures: Text and Methodology Q2(R1). 2005; p p6-17.
  29. Korean Pharmaceutical Codex, 3rd ed.; Korea Food and Drug Administration: 2007; pp 1165-1166.
  30. Pharmeuropa 11 European Pharmacopoeia Commission 1999; p 333.
  31. Deutsches Arzneibuch, Vol 2. Methoden der Biologie. Stuttgart, Deutscher Apotheker Verlag, 1996.

Cited by

  1. Plants, Extracts, and Dietary Supplements vol.62, pp.50, 2014,
  2. Antioxidant and antigenotoxic activities of Ginkgo biloba L. leaf extract are retained after in vitro gastrointestinal digestive conditions pp.1436-6215, 2019,