• Title/Summary/Keyword: Rare-event

Search Result 179, Processing Time 0.021 seconds

Comparison of Bias Correction Methods for the Rare Event Logistic Regression (희귀 사건 로지스틱 회귀분석을 위한 편의 수정 방법 비교 연구)

  • Kim, Hyungwoo;Ko, Taeseok;Park, No-Wook;Lee, Woojoo
    • The Korean Journal of Applied Statistics
    • /
    • v.27 no.2
    • /
    • pp.277-290
    • /
    • 2014
  • We analyzed binary landslide data from the Boeun area with logistic regression. Since the number of landslide occurrences is only 9 out of 5000 observations, this can be regarded as a rare event data. The main issue of logistic regression with the rare event data is a serious bias problem in regression coefficient estimates. Two bias correction methods were proposed before and we quantitatively compared them via simulation. Firth (1993)'s approach outperformed and provided the most stable results for analyzing the rare-event binary data.

Finding Association Rules based on the Significant Rare Relation of Events with Time Attribute (시간 속성을 갖는 이벤트의 의미있는 희소 관계에 기반한 연관 규칙 탐사)

  • Han, Dae-Young;Kim, Dae-In;Kim, Jae-In;Song, Myung-Jin;Hwang, Bu-Hyun
    • The KIPS Transactions:PartD
    • /
    • v.16D no.5
    • /
    • pp.691-700
    • /
    • 2009
  • An event means a flow which has a time attribute such as the a symptom of patients, an interval event has the time period between the start-time-point and the end-time-point. Although there are many studies for temporal data mining, they do not deal with discovering knowledge from interval event such as patient histories and purchase histories. In this paper, we suggest a method of temporal data mining that finds association rules of event causal relationships and predicts an occurrence of effect event based on discovered rules. Our method can predict the occurrence of an event by summarizing an interval event using the time attribute of an event and finding the causal relationship of event. As a result of simulation, this method can discover better knowledge than others by considering a lot of supports of an event and finding the significant rare relation on interval events which means an essential cause of an event, regardless of an occurrence support of an event in comparison with conventional data mining techniques.

A Study on the Adjustment of Posterior Probability for Oversampling when the Target is Rare (목표 범주가 희귀한 자료의 과대표본추출에 대한 연구)

  • Kim, U.N.;Lee, S.K.;Choi, J.H.
    • The Korean Journal of Applied Statistics
    • /
    • v.24 no.3
    • /
    • pp.477-484
    • /
    • 2011
  • When an event of target variable is rare, a widespread strategy is to build a model on the sample that disproportionally over-represents the events, that is over-sampled. Using the data over-sampled from the original data set, the predicted values would be biased; however, it can be easily corrected to represent the population. In this study, we investigate into the relationship between the proportion of rare event on a data-mart and the model performance using real world data of a Korean credit card company. Also, we use the methods for adjusting of posterior probability for over-sampled data of the offset method and the weighted method. Finally, we compare the performance of the methods using real data sets.

Efficient Flash Memory Access Power Reduction Techniques for IoT-Driven Rare-Event Logging Application (IoT 기반 간헐적 이벤트 로깅 응용에 최적화된 효율적 플래시 메모리 전력 소모 감소기법)

  • Kwon, Jisu;Cho, Jeonghun;Park, Daejin
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.14 no.2
    • /
    • pp.87-96
    • /
    • 2019
  • Low power issue is one of the most critical problems in the Internet of Things (IoT), which are powered by battery. To solve this problem, various approaches have been presented so far. In this paper, we propose a method to reduce the power consumption by reducing the numbers of accesses into the flash memory consuming a large amount of power for on-chip software execution. Our approach is based on using cooperative logging structure to distribute the sampling overhead in single sensor node to adjacent nodes in case of rare-event applications. The proposed algorithm to identify event occurrence is newly introduced with negative feedback method by observing difference between past data and recent data coming from the sensor. When an event with need of flash access is determined, the proposed approach only allows access to write the sampled data in flash memory. The proposed event detection algorithm (EDA) result in 30% reduction of power consumption compared to the conventional flash write scheme for all cases of event. The sampled data from the sensor is first traced into the random access memory (RAM), and write access to the flash memory is delayed until the page buffer of the on-chip flash memory controller in the micro controller unit (MCU) is full of the numbers of the traced data, thereby reducing the frequency of accessing flash memory. This technique additionally reduces power consumption by 40% compared to flash-write all data. By sharing the sampling information via LoRa channel, the overhead in sampling data is distributed, to reduce the sampling load on each node, so that the 66% reduction of total power consumption is achieved in several IoT edge nodes by removing the sampling operation of duplicated data.

Rare Disaster Events, Growth Volatility, and Financial Liberalization: International Evidence

  • Bongseok Choi
    • Journal of Korea Trade
    • /
    • v.27 no.2
    • /
    • pp.96-114
    • /
    • 2023
  • Purpose - This paper elucidates a nexus between the occurrence of rare disaster events and the volatility of economic growth by distinguishing the likelihood of rare events from stochastic volatility. We provide new empirical facts based on a quarterly time series. In particular, we focus on the role of financial liberalization in spreading the economic crisis in developing countries. Design/methodology - We use quarterly data on consumption expenditure (real per capita consumption) from 44 countries, including advanced and developing countries, ending in the fourth quarter of 2020. We estimate the likelihood of rare event occurrences and stochastic volatility for countries using the Bayesian Markov chain Monte Carlo (MCMC) method developed by Barro and Jin (2021). We present our estimation results for the relationship between rare disaster events, stochastic volatility, and growth volatility. Findings - We find the global common disaster event, the COVID-19 pandemic, and thirteen country-specific disaster events. Consumption falls by about 7% on average in the first quarter of a disaster and by 4% in the long run. The occurrence of rare disaster events and the volatility of gross domestic product (GDP) growth are positively correlated (4.8%), whereas the rare events and GDP growth rate are negatively correlated (-12.1%). In particular, financial liberalization has played an important role in exacerbating the adverse impact of both rare disasters and financial market instability on growth volatility. Several case studies, including the case of South Korea, provide insights into the cause of major financial crises in small open developing countries, including the Asian currency crisis of 1998. Originality/value - This paper presents new empirical facts on the relationship between the occurrence of rare disaster events (or stochastic volatility) and growth volatility. Increasing data frequency allows for greater accuracy in assessing a country's specific risk. Our findings suggest that financial market and institutional stability can be vital for buffering against rare disaster shocks. It is necessary to preemptively strengthen the foundation for financial stability in developing countries and increase the quality of the information provided to markets.

Mining Association Rule for the Abnormal Event in Data Stream Systems (데이터 스트림 시스템에서 이상 이벤트에 대한 연관 규칙 마이닝)

  • Kim, Dae-In;Park, Joon;Hwang, Bu-Hyun
    • The KIPS Transactions:PartD
    • /
    • v.14D no.5
    • /
    • pp.483-490
    • /
    • 2007
  • Recently mining techniques that analyze the data stream to discover potential information, have been widely studied. However, most of the researches based on the support are concerned with the frequent event, but ignore the infrequent event even if it is crucial. In this paper, we propose SM-AF method discovering association rules to an abnormal event. In considering the window that an abnormal event is sensed, SM-AF method can discover the association rules to the critical event, even if it is occurred infrequently. Also, SM-AF method can discover the significant rare itemsets associated with abnormal event and periodic event itemsets. Through analysis and experiments, we show that SM-AF method is superior to the previous methods of mining association rules.

A New Fast Simulation Technique for Rare Event Simulation

  • Kim, Yun-Bae;Roh, Deok-Seon;Lee, Myeong-Yong
    • Proceedings of the Korea Society for Simulation Conference
    • /
    • 1999.04a
    • /
    • pp.70-79
    • /
    • 1999
  • Importance Sampling (IS) has been applied to accelerate the occurrence of rare events. However, it has a drawback of effective biasing scheme to make the estimator from IS unbiased. Adaptive Importance Sampling (AIS) employs an estimated sampling distribution of IS to the systems of interest during the course of simulation. We propose Nonparametric Adaptive Importance Sampling (NAIS) technique which is nonparametrically modified version of AIS and test it to estimate a probability of rare event in M/M/1 queueing model. Comparing with classical Monte Carlo simulation, the computational efficiency and variance reductions gained via NAIS are substantial. A possible extension of NAIS regarding with random number generation is also discussed.

  • PDF

A Simple Approach to Calculate CDF with Non-rare Events in Seismic PSA Model of Korean Nuclear Power Plants (국내 원자력발전소 지진 PSA의 CDF 과평가 방지를 위한 비희귀사건 모델링 방법 연구)

  • Lim, Hak Kyu
    • Journal of the Korean Society of Safety
    • /
    • v.36 no.5
    • /
    • pp.86-91
    • /
    • 2021
  • Calculating the scrutable core damage frequency (CDF) of nuclear power plants is an important component of the seismic probabilistic safety assessment (SPSA). In this work, a simple approach is developed to calculate CDF from minimal cut sets (MCSs) with non-rare events. When conventional calculation methods based on rare event approximations are employed, the CDF of industry SPSA models is significantly overestimated by non-rare events in the MCSs. Recently, quantification algorithms using binary decision diagrams (BDDs) have been introduced to prevent CDF overestimation in the SPSA. However, BDD structures are generated from a small part of whole MCSs due to limited computational memory, and they cannot be reviewed due to their complicated logic structure. This study suggests a simple approach for scrutinizing the CDF calculation based on whole MCSs in the SPSA system analysis model. The proposed approach compares the new results to outputs from existing algorithms, which helps in avoiding CDF overestimation.