In this paper, the decentralized suboptimal $H_2$ filtering problem is considered. An additional term is added to the centralized optimal $H_2$ filter so that the whole filter is decentralized. We derive a sufficient condition for existence of such decentralized filters. By employing the solution procedure for the exact model matching problem, we obtain a set of decentralized $H_2$ filters, and choose a suboptimal filter from this set of decentralized $H_2$ filters. Naturally the resulting filter is guaranteed to be stable.