Evaluating the Accuracy of the Copyleaks AI Video Detector

Using a dual-team system, we have designed our evaluation process to ensure top-level quality, standards, and reliability. We utilize two independent departments to evaluate the model: our Data Science and QA teams. Each team works autonomously with its own unique evaluation data and tools, ensuring that results remain unbiased, objective, and accurate.

It is essential to note that all testing data is strictly separated from the model's training data. We evaluate our models only on fresh content that our AI Video Detector has never encountered. To ensure our testing remains relevant, we continuously update our datasets to include media generated by the latest GenAI models.

For every model release, our Data Science and QA teams independently curate testing datasets. Each dataset consists of a large number of files with a strict ground-truth label indicating its origin, as follows:

• Fully-Human: Authentic video and audio produced through traditional methods (whether captured by physical recording devices or created via standard digital animation and VFX) containing zero generative AI alterations.
• Fully-AI: Content synthesized entirely by generative AI networks, including foundational text- and image-to-video models, deepfake algorithms, and advanced voice cloning engines.

Testing is executed directly against the Copyleaks API, which processes visual and auditory tracks entirely independent of one another. While our core benchmark testing isolates Fully-Human and Fully-AI media to ensure strict, measurable performance metrics, this independent dual-track architecture guarantees the system accurately handles hybrid content (e.g., an authentic human video dubbed with an AI-generated voiceover). Rather than blending these results, we evaluate the API outputs for the video and audio tracks completely independently, ensuring absolute transparency for each specific medium.

The evaluation is conducted on media that meets our optimal technical requirements:

• Duration: Between 2 seconds and 1 hour.
• Resolution: A minimum of 360x360 pixels.
• Frame Rate: A minimum of 16 FPS.
• Audio/File Size: Under 0.5 GB.
• Supported Formats: Standard containers including .mp4, .mov, .avi, .webm, and other professional formats defined in our documentation.

Video and audio tracks are dynamic mediums; a single file can contain a complex mix of authentic recording and AI-generated splicing. To provide the most robust detection, we analyze content on a continuous timeline. For our users, this dynamic analysis offers a highly practical benefit: the ability to see granular, frame-by-frame timestamps that pinpoint exactly when suspected manipulation occurs.

However, when evaluating the core performance and reliability of our models, we focus our benchmark testing on the macro File-Level verdict. This file-level metric evaluates the media as a whole, calculating the total duration of synthetic manipulation to categorize the complete video or audio file as either "AI-Generated" or "Human." This strict, macro-level evaluation reflects how most organizations deploy our system for high-confidence triage, policy enforcement, and automated moderation.

To provide absolute clarity on our system's reliability at this file level, we measure our performance using three primary, industry-standard metrics:

• AI Detection Rate (True Positive Rate): The percentage of actual AI-generated content that our system successfully catches.
• False Positive Rate (FPR): The percentage of authentic human content that is incorrectly identified as AI. A FPR of 1% means the True Negative Rate (TNR) is 99%, ensuring authentic creators are rarely penalized.
• Precision: The accuracy of our positive AI flags. A Precision of 99% means that out of every 100 files we flag as AI, 99 are truly AI, ensuring users can act on our alerts with high confidence.

Our Data Science and QA teams independently conducted tests on large, diverse datasets containing tens of thousands of video and audio files. These datasets encompass varying resolutions, recording qualities, capturing devices, and generative AI models (including deepfake generators, voice cloning, and text-to-speech engines) which are considered state-of-the-art at the time of publication.

The following results demonstrate the system's ability to accurately categorize complete files based on their overall composition, providing a clear picture of its high-confidence triage capabilities.

While our detection system achieves state-of-the-art results, no technology is perfect. Our models can occasionally misinterpret complex temporal or acoustic segments, leading to incorrect assessments.

The Copyleaks AI Video and Audio Detector is specifically engineered to identify synthetic footprints left by the latest generative AI tools, and is not currently designed to detect traditional media alterations, including:

• Manual Splicing & Creative Editing: Standard transitions, cuts, or arrangements made by a human editor using traditional software (e.g., Adobe Premiere, Final Cut Pro, or Audacity).
• Traditional VFX & CGI: Computer-generated imagery or sound effects created through standard 3D modeling, animation pipelines, or foley work rather than generative neural networks.
• Non-Generative Adjustments: Photographic filters, color grading, sharpening, cropping, speed adjustments (timelapse/slow-motion), or basic audio equalization (EQ) and noise reduction.

Furthermore, because the generative AI landscape evolves at a rapid pace, the detection of content created by models released after our most recent update may be less consistent until those tools are integrated into our training datasets. Additionally, certain highly specialized manipulations (such as specific face-swapping applications or musical AI generations) are currently part of our active research and development and may not be fully covered in this release.

It should be noted that the Audio Detector is designed to detect AI usage in speech; other use cases (such as AI-generated music) should not expect similar levels of accuracy compared to the numbers shown above.

We are deeply committed to the continuous improvement of our detection engines. Your insights play a crucial role in our mission to ensure digital authenticity. If you encounter a specific AI model, voice-cloning tool, or emerging manipulation technique you would like to see supported, we warmly encourage you to share your feedback so we can prioritize it in our upcoming development cycles.