• Journal of Korea Water Resources Association
• /
• v.54 no.11
• /
• pp.933-942
• /
• 2021

The surface image velocimetry was developed to measure river flow velocity safely and effectively in flood season. There are a couple of methods in the surface image velocimetry. Among them the spatio-temporal image velocimetry is in the spotlight, since it can estimate mean velocity for a period of time. For the spatio-temporal image velocimetry analyzes a series of images all at once, it can reduce analyzing time so much. It, however, has a little drawback to find out the main flow direction. If the direction of spatio-temporal image does not coincide to the main flow direction, it may cause singnificant error in velocity. The present study aims to propose a new method to find out the main flow direction by using a fast Fourier transform(FFT) to a spatio-temporal (image) volume, which were constructed by accumulating the river surface images along the time direction. The method consists of two steps; the first step for finding main flow direction in space image and the second step for calculating the velocity magnitude in main flow direction in spatio-temporal image. In the first step a time-accumulated image was made from the spatio-temporal volume along the time direction. We analyzed this time-accumulated image by using FFT and figured out the main flow direction from the transformed image. Then a spatio-temporal image in main flow direction was extracted from the spatio-temporal volume. Once again, the spatio-temporal image was analyzed by FFT and velocity magnitudes were calculated from the transformed image. The proposed method was applied to a series of artificial images for error analysis. It was shown that the proposed method could analyze two-dimensional flow field with fairly good accuracy.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.6 no.10
• /
• pp.2632-2649
• /
• 2012

Most approaches to human action recognition is limited due to the use of simple action datasets under controlled environments or focus on excessively localized features without sufficiently exploring the spatio-temporal information. This paper proposed a framework for recognizing realistic human actions. Specifically, a new action representation is proposed based on computing a rich set of descriptors from keypoint trajectories. To obtain efficient and compact representations for actions, we develop a feature fusion method to combine spatial-temporal local motion descriptors by the movement of the camera which is detected by the distribution of spatio-temporal interest points in the clips. A new topic model called Markov Semantic Model is proposed for semantic feature selection which relies on the different kinds of dependencies between words produced by "syntactic " and "semantic" constraints. The informative features are selected collaboratively based on the different types of dependencies between words produced by short range and long range constraints. Building on the nonlinear SVMs, we validate this proposed hierarchical framework on several realistic action datasets.

• Journal of KIISE:Software and Applications
• /
• v.31 no.11
• /
• pp.1534-1542
• /
• 2004

A novel vision-based scheme of extracting real-time traffic information parameters is presented. The method is based on a region classification followed by a spatio-temporal image analysis. The detection region images for each traffic lane are classified into one of the three categories: the road, the vehicle, and the shadow, using statistical and structural features. Misclassification in a frame is corrected by using temporally correlated features of vehicles in the spatio-temporal image. Since only local images of detection regions are processed, the real-time operation of more than 30 frames per second is realized without using dedicated parallel processors, while ensuring detection performance robust to the variation of weather conditions, shadows, and traffic load.

• Korean Journal of Remote Sensing
• /
• v.36 no.5_1
• /
• pp.807-821
• /
• 2020

The objective of this study is to evaluate the applicability of representative spatio-temporal fusion models developed for the fusion of mid- and low-resolution satellite images in order to construct a set of time-series high-resolution images for crop monitoring. Particularly, the effects of the characteristics of input image pairs on the prediction performance are investigated by considering the principle of spatio-temporal fusion. An experiment on the fusion of multi-temporal Sentinel-2 and RapidEye images in agricultural fields was conducted to evaluate the prediction performance. Three representative fusion models, including Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM), SParse-representation-based SpatioTemporal reflectance Fusion Model (SPSTFM), and Flexible Spatiotemporal DAta Fusion (FSDAF), were applied to this comparative experiment. The three spatio-temporal fusion models exhibited different prediction performance in terms of prediction errors and spatial similarity. However, regardless of the model types, the correlation between coarse resolution images acquired on the pair dates and the prediction date was more significant than the difference between the pair dates and the prediction date to improve the prediction performance. In addition, using vegetation index as input for spatio-temporal fusion showed better prediction performance by alleviating error propagation problems, compared with using fused reflectance values in the calculation of vegetation index. These experimental results can be used as basic information for both the selection of optimal image pairs and input types, and the development of an advanced model in spatio-temporal fusion for crop monitoring.

• Journal of Broadcast Engineering
• /
• v.9 no.3
• /
• pp.214-224
• /
• 2004

In this paper, an adaptive spatio-temporal predictive coding based on the H.264 is proposed for 3D immersive media encoding, such as 3D image processing, 3DTV, and 3D videoconferencing. First, we propose a spatio-temporal predictive coding using the same view and inter-view images for the two TPPP, IBBP GOP (group of picture) structures 4hat are different from the conventional simulcast method. Second, an 2D inter-view direct mode for the efficient prediction is proposed when the proposed spatio-temporal prediction uses the IBBP structure. The 2D inter-view direct mode is applied when the temporal direct mode in B(hi-Predictive) picture of the H.264 refers to an inter-view image, since the current temporal direct mode in the H.264 standard could no: be applied to the inter-view image. The proposed method is compared to the conventional simulcast method in terms of PSNR (peak signal to noise ratio) for the various 3D test video sequences. The proposed method shows better PSNR results than the conventional simulcast mode.

• The Transactions of The Korean Institute of Electrical Engineers
• /
• v.65 no.10
• /
• pp.1731-1737
• /
• 2016

In this paper, a method is proposed to recognize human actions as nonverbal expression; the proposed method is composed of two steps which are action representation and action recognition. First, MHI(Motion History Image) is used in the action representation step. This method includes segmentation based on depth information and generates spatio-temporal templates to describe actions. Second, CNN(Convolution Neural Network) which includes feature extraction and classification is employed in the action recognition step. It extracts convolution feature vectors and then uses a classifier to recognize actions. The recognition performance of the proposed method is demonstrated by comparing other action recognition methods in experimental results.

• Journal of Korea Multimedia Society
• /
• v.23 no.8
• /
• pp.977-985
• /
• 2020

Aiming at the problem of complex feature extraction and low accuracy in human action recognition, this paper proposed a network structure combining batch normalization algorithm with GoogLeNet network model. Applying Batch Normalization idea in the field of image classification to action recognition field, it improved the algorithm by normalizing the network input training sample by mini-batch. For convolutional network, RGB image was the spatial input, and stacked optical flows was the temporal input. Then, it fused the spatio-temporal networks to get the final action recognition result. It trained and evaluated the architecture on the standard video actions benchmarks of UCF101 and HMDB51, which achieved the accuracy of 93.42% and 67.82%. The results show that the improved convolutional neural network has a significant improvement in improving the recognition rate and has obvious advantages in action recognition.

• The KIPS Transactions:PartB
• /
• v.14B no.5
• /
• pp.351-360
• /
• 2007

Video Completion refers to a computer vision technique which restores damaged images by filling missing pixels with suitable color in a video sequence. We propose a new video completion technique to fill in image holes which are caused by removing an unnecessary object in a video sequence, where two objects cross each other in the presence of camera motion. We remove the closer object from a camera which results in image holes. Then these holes are filled by color information of some others frames. First of all, spatio-temporal volumes of occluding and occluded objects are created according to the centroid of the objects. Secondly, a temporal search technique by voxel matching separates and removes the occluding object. Finally. these holes are filled by using spatial search technique. Seams on the boundary of completed pixels we removed by a simple blending technique. Experimental results using real video sequences show that the proposed technique produces new completed videos.

• Proceedings of the IEEK Conference
• /
• 1998.10a
• /
• pp.871-874
• /
• 1998

This paper presents a new morphological spatio-temporal segmentation algorithm. The algorithm incorporates intensity and motion information simultaneously, and uses morphological tools such as morphological filters and watershed algorithm. The procedure toward complete segmetnation consists of three steps: joint marker extraction, boundary decision, and motion-based region fusion. By incorporating spatial and temporal information simultaneously, we can obtain visually meaningful segmentation results. Simulation results demonstrates the efficiency of the proposed method.

• Korean Journal of Remote Sensing
• /
• v.37 no.6_3
• /
• pp.1931-1942
• /
• 2021

As the utility of an optical satellite image with a high spatial resolution (i.e., fine-scale) has been emphasized, recently, various studies of the land surface monitoring using those have been widely carried out. However, the usefulness of fine-scale satellite images is limited because those are acquired at a low temporal resolution. To compensate for this limitation, the spatiotemporal data fusion can be applied to generate a synthetic image with a high spatio-temporal resolution by fusing multiple satellite images with different spatial and temporal resolutions. Since the spatio-temporal data fusion models have been developed for mid or low spatial resolution satellite images in the previous studies, it is necessary to evaluate the applicability of the developed models to the satellite images with a high spatial resolution. For this, this study evaluated the applicability of the developed spatio-temporal fusion models for KOMPSAT-3A and Sentinel-2 images. Here, an Enhanced Spatial and Temporal Adaptive Fusion Model (ESTARFM) and Spatial Time-series Geostatistical Deconvolution/Fusion Model (STGDFM), which use the different information for prediction, were applied. As a result of this study, it was found that the prediction performance of STGDFM, which combines temporally continuous reflectance values, was better than that of ESTARFM. Particularly, the prediction performance of STGDFM was significantly improved when it is difficult to simultaneously acquire KOMPSAT and Sentinel-2 images at a same date due to the low temporal resolution of KOMPSAT images. From the results of this study, it was confirmed that STGDFM, which has relatively better prediction performance by combining continuous temporal information, can compensate for the limitation to the low revisit time of fine-scale satellite images.