Abstract: A novel approach of descriptor design for local appearance matching is presented. Because MSER region detector selects only the most stable regions which results in high repeatability, we choose it to detect image regions covariant to image transformation, which are then used as interest regions for computing descriptors. To get more distinctive feature vector to characterize the local image appearance, our descriptor consists of two main parts: (1) Affine Invariant Fourier Descriptor (AIFD) calculated based on MSER since AIFD possess invariant properties under translation, scaling, rotation and shearing and (2) color moments and GLCM based texture calculated from normalized patches. We deduce a fast patch normalization process which starts from a set of polygonal image regions output from MSER. Texture and color are the intrinsic properties of object and robust to affine transformation. We assessed our descriptor based on public image datasets which contains structured and textured scenes in different viewpoint as well as illumination change scenes. The experimental results showed that our descriptor obtained higher matching score comparing to several published descriptors.