The present paper deals with extraction of figures and characters from their background using the knowledge of color. At each pixel of the image on the CRT sent from a video camera, RGB values are transformed into the values in another color system, HSI, where "H" denotes hue;"S" denotes saturation;"I" denotes intensity. Representing color in HSI color space is advantageous, since a human feels color mainly in hue with the aid of brightness and purity. Comparing HSI data thus obtained with the masked original image detects noise-free edges included in the orginal image. Then setting a set of HSI thresholds and changing it identifies the portion of image of the same color. This color information is used in recongnizing characters and figures as an auxiliary system of a hierachical figure categorization method for characters and figures recognition.cters and figures recognition.