In this paper, we propose a deep learning structure suitable for embedded system. The flame detection process of the proposed deep learning structure consists of four steps : flame area detection using flame color model, flame image classification using deep learning structure for flame color specialization, $N{\times}N$ cell separation in detected flame area, flame image classification using deep learning structure for flame shape specialization. First, only the color of the flame is extracted from the input image and then labeled to detect the flame area. Second, area of flame detected is the input of a deep learning structure specialized in flame color and is classified as flame image only if the probability of flame class at the output is greater than 75%. Third, divide the detected flame region of the images classified as flame images less than 75% in the preceding section into $N{\times}N$ units. Fourthly, small cells divided into $N{\times}N$ units are inserted into the input of a deep learning structure specialized to the shape of the flame and each cell is judged to be flame proof and classified as flame images if more than 50% of cells are classified as flame images. To verify the effectiveness of the proposed deep learning structure, we experimented with a flame database of ImageNet. Experimental results show that the proposed deep learning structure has an average resource occupancy rate of 29.86% and an 8 second fast flame detection time. The flame detection rate averaged 0.95% lower compared to the existing deep learning structure, but this was the result of light construction of the deep learning structure for application to embedded systems. Therefore, the deep learning structure for flame detection proposed in this paper has been proved suitable for the application of embedded system.