Abstract: This study presents the system of annotation of the medical images of the patients which is based on ontologies of the image and of the patient which is a base ontology since the information of this last represent the legends for these images. This system uses for a specification of the ontology and the annotation, the SHOE (Simple HTML Ontology Extensions) language.