• International Journal of Computer Science & Network Security
• /
• v.21 no.12spc
• /
• pp.549-555
• /
• 2021

Machine Learning(ML) splits data into 3 parts, which are usually 60% for training, 20% for validation, and 20% for testing. It just splits quantitatively instead of selecting each set of data by a criterion, which is very important concept for the adequacy of test data. ML measures a model's accuracy by applying a set of validation data, and revises the model until the validation accuracy reaches on a certain level. After the validation process, the complete model is tested with the set of test data, which are not seen by the model yet. If the set of test data covers the model's attributes well, the test accuracy will be close to the validation accuracy of the model. To make sure that ML's set of test data works adequately, we design an experiment and see if the test accuracy of model is always close to its validation adequacy as expected. The experiment builds 100 different SVM models for each of six data sets published in UCI ML repository. From the test accuracy and its validation accuracy of 600 cases, we find some unexpected cases, where the test accuracy is very different from its validation accuracy. Consequently, it is not always true that ML's set of test data is adequate to assure a model's quality.

• Proceedings of the Korean Society for Noise and Vibration Engineering Conference
• /
• 2014.10a
• /
• pp.347-349
• /
• 2014

Model verification and validation (V&V) is a current research topic to build computational models with high predictive capability by addressing the general concepts, processes and statistical techniques. The hypothesis test for validity check is one of the model validation techniques and gives a guideline to evaluate the validity of a computational model when limited experimental data only exist due to restricted test resources (e.g., time and budget). The hypothesis test for validity check mainly employ Type I error, the risk of rejecting the valid computational model, for the validity evaluation since quantification of Type II error is not feasible for model validation. However, Type II error, the risk of accepting invalid computational model, should be importantly considered for an engineered products having high risk on predicted results. This paper proposes a technique named as the response-adaptive experimental design to reduce Type II error by adaptively designing experimental conditions for the validation experiment. A tire tread block problem and a numerical example are employed to show the effectiveness of the response-adaptive experimental design for the validity evaluation.

• Transactions of the Korean Society of Mechanical Engineers A
• /
• v.31 no.1 s.256
• /
• pp.55-61
• /
• 2007

Recently simulation model becomes an essential tool for analysis and design of a system but it is often expensive and time consuming as it becomes complicate to achieve reliable results. Therefore, high-fidelity simulation model needs to be replaced by an approximate model, the so-called metamodel. Metamodeling techniques include 3 components of sampling, metamodel and validation. Cross-validation approach has been proposed to provide sequnatially new sample point based on cross-validation error but it is very expensive because cross-validation must be evaluated at each stage. To enhance the cross-validation of metamodel, sequential sampling method using candidate points and representative cross-validation is proposed in this paper. The candidate and representative cross-validation approach of sequential sampling is illustrated for two-dimensional domain. To verify the performance of the suggested sampling technique, we compare the accuracy of the metamodels for various mathematical functions with that obtained by conventional sequential sampling strategies such as maximum distance, mean squared error, and maximum entropy sequential samplings. Through this research we team that the proposed approach is computationally inexpensive and provides good prediction performance.

• The Korean Journal of Applied Statistics
• /
• v.25 no.1
• /
• pp.139-152
• /
• 2012

The result of the analysis of a population pharmacokinetic model can directly influence the decision of the dose level applied to the targeted patients. Therefore the validation procedure of the final model is very important in this area. This paper reviews the validation methods of population pharmacokinetic models from a statistical viewpoint. In addition, the whole procedure of the analysis of population pharmacokinetics, from the base model to the final model (that includes various validation procedures for the final model) is tested with real clinical data.

• Steel and Composite Structures
• /
• v.17 no.6
• /
• pp.835-850
• /
• 2014

The fire resistance of composite steel and concrete structures may be determined by using the simplified methods provided in EN 1994-1-2. For the particular situations not covered by the standard, an advanced calculation model might be applied, using special purpose programs for the analysis of structures in fire. The validation of these programs has always been an important issue for software developers, but also for designers and authorities. Clause 4.4.4 from EN 1994-1-2 refers to the validation of the advanced calculation models and states that these models must be validated through relevant test results. The paper presents the calculation of fire resistance of the composite columns in a high-rise building built in Romania, and focusses on the validation of the calculation model (computer program SAFIR), for this particular case. This validation, asked by the Romanian authorities, considers the available experimental results of a fire test, performed on a similar composite steel-concrete column.

• Geomechanics and Engineering
• /
• v.28 no.5
• /
• pp.437-454
• /
• 2022

The hydro-mechanical (HM) two-way sequential coupling in the TOUGH2-FLAC3D linking algorithm is validated completely and successfully in both M to H and H to M directions, which are initiated by mechanical surface loading for geomechanical validation and hydrological groundwater pumping for hydrogeological validation, respectively. For such complete and successful validation, a TOUGH2-FLAC3D linked numerical model is developed first by adopting the TOUGH2-FLAC3D linking algorithm, which uses the two-way (fixed-stress split) sequential coupling scheme and the implicit backward time stepping method. Two geomechanical and two hydrogeological validation problems are then simulated using the linked numerical model together with basic validation strategies and prerequisites. The second geomechanical and second hydrogeological validation problems are also associated with the Mandel effect and the Noordbergum and Rhade effects, respectively, which are three phenomenally well-known but numerically challenging HM effects. Finally, sequentially coupled numerical solutions are compared with either analytical solutions (verification) or fully coupled numerical solutions (benchmarking). In all the four validation problems, they show almost perfect to extremely or very good agreement. In addition, the second geomechanical validation problem clearly displays the Mandel effect and suggests a proper or minimum geometrical ratio of the height to the width for the rectangular domain to maximize agreement between the numerical and analytical solutions. In the meantime, the second hydrogeological validation problem clearly displays the Noordbergum and Rhade effects and implies that the HM two-way sequential coupling scheme used in the linked numerical model is as rigorous as the HM two-way full coupling scheme used in a fully coupled numerical model.

• Korean Chemical Engineering Research
• /
• v.59 no.4
• /
• pp.532-541
• /
• 2021

Recently, research on the application of artificial intelligence in the chemical process has been increasing rapidly. However, overfitting is a significant problem that prevents the model from being generalized well to predict unseen data on test data, as well as observed training data. Cross validation is one of the ways to solve the overfitting problem. In this study, the time-series cross validation method was applied to optimize the number of batch and epoch in the hyperparameters of the prediction model for the 2,3-BDO distillation process, and it compared with K-fold cross validation generally used. As a result, the RMSE of the model with time-series cross validation was lower by 9.06%, and the MAPE was higher by 0.61% than the model with K-fold cross validation. Also, the calculation time was 198.29 sec less than the K-fold cross validation method.

• Coupled systems mechanics
• /
• v.4 no.1
• /
• pp.37-65
• /
• 2015

Modeling and simulation of mechanical response of infrastructure object, solids and structures, relies on the use of computational models to foretell the state of a physical system under conditions for which such computational model has not been validated. Verification and Validation (V&V) procedures are the primary means of assessing accuracy, building confidence and credibility in modeling and computational simulations of behavior of those infrastructure objects. Validation is the process of determining a degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model. It is mainly a physics issue and provides evidence that the correct model is solved (Oberkampf et al. 2002). Our primary interest is in modeling and simulating behavior of porous particulate media that is fully saturated with pore fluid, including cyclic mobility and liquefaction. Fully saturated soils undergoing dynamic shaking fall in this category. Verification modeling and simulation of fully saturated porous soils is addressed in more detail by (Tasiopoulou et al. 2014), and in this paper we address validation. A set of centrifuge experiments is used for this purpose. Discussion is provided assessing the effects of scaling laws on centrifuge experiments and their influence on the validation. Available validation test are reviewed in view of first and second order phenomena and their importance to validation. For example, dynamics behavior of the system, following the dynamic time, and dissipation of the pore fluid pressures, following diffusion time, are not happening in the same time scale and those discrepancies are discussed. Laboratory tests, performed on soil that is used in centrifuge experiments, were used to calibrate material models that are then used in a validation process. Number of physical and numerical examples are used for validation and to illustrate presented discussion. In particular, it is shown that for the most part, numerical prediction of behavior, using laboratory test data to calibrate soil material model, prior to centrifuge experiments, can be validated using scaled tests. There are, of course, discrepancies, sources of which are analyzed and discussed.

• Journal of the Korea Society of Computer and Information
• /
• v.27 no.11
• /
• pp.13-18
• /
• 2022

In this paper, we proposed VW-FedAVG(Validation based Weighted FedAVG) which updates the global model by weighting according to performance verification from the models of each device participating in the training. The first method is designed to validate each local client model through validation dataset before updating the global model with a server side validation structure. The second is a client-side validation structure, which is designed in such a way that the validation data set is evenly distributed to each client and the global model is after validation. MNIST, CIFAR-10 is used, and the IID, Non-IID distribution for image classification obtained higher accuracy than previous studies.

• Journal of Korean Institute of Industrial Engineers
• /
• v.6 no.1
• /
• pp.27-38
• /
• 1980

This paper presents the validation of a stochastic model of muscle fatigue during static muscle contractions. Forty four laboratory experiments, covering eleven test conditions for two trained subjects, were run in order to estimate fatigue and recovery rates, based on EMG observations. The validation of the model was made by comparing the model predictions to the experimental fatigue time. The validation study supports that the stochastic model of muscle fatigue accurately represents the underlying fatigue process. The study also provides support that the fatigue model can be used as a monitor of individual muscle capabilities.