Abstract: Although, many fragile video watermarking techniques have been proposed as an effective solution to content authentication problems, they cannot effectively distinguish between legal attacks and illegal attacks. In order to solve this problem, this paper proposes a novel semi-fragile video watermarking algorithm for H.264 video. Traditional fragile video watermarking methods often select the motion vectors as the embedding positions, while our scheme adopt the intra-prediction residuals after Integer DCT (Discrete Cosine Transform) and quantization to be embedding locations. During the watermark embedding process, we modify the standard H.264 quantizer and we embed the watermark based on the dither modulation technique in order to extract the watermark blindly. In the watermark extraction process, this paper adopts a threshold to distinguish between illegal attacks and legal attacks performed on the video clip. Experimental results demonstrate the effectiveness and feasibility of the proposed algorithm and the proposed scheme can survive legal recompression operations and recognize illegal filtering operations.