URL Verification System
How a fact-checking team turned hours of manual source-checking per article into a 30-second verdict, with an AI Employee that shows its evidence.
URL Verification System
“What used to take a trained analyst three hours and a stack of source databases now takes under thirty seconds. We needed scores our editors could defend in print, not numbers a model thought looked right. Dojo Labs built it so every confidence score is computed from evidence the analyst can read on the page.”
URL Verification System
Measurable Outcomes
that drive ROI.
< 30s
Time to Verdict
~95%
Time Reduced Per Evaluation
99.5%
Platform Uptime
40%
Faster via Dedup
By integrating our computation layer, URL Verification System transformed from a services-heavy model to a scalable, automated platform.
Client Overview
About URL Verification System
The URL Verification System is an AI content credibility platform built for organizations operating in high-stakes information environments: media companies, educational institutions, research facilities, and fact-checking outfits that have to evaluate the reliability of online content at the pace of the news cycle. The pitch was direct: take an analyst's hours of source-checking and turn it into a confidence score in 30 seconds, with every part of that score backed by evidence the analyst can audit.
Industry
Media, EdTech, Research, Fact-Checking
Client
URL Verification System
Target Users
Organizations, researchers, journalists, general users
Engagement Type
Full-Stack AI Platform, Build & Integration
The Problem
The Challenge
Before we came in, evaluating the credibility of a single article took hours of expert time. Misinformation does not announce itself: sophisticated false content mimics the structure, tone, and citation style of credible journalism. So a trained analyst would manually pull source databases, cross-reference claims, check publication histories, and write up a verdict, while the news cycle that triggered the request had already moved on.
Hours of expert time per article for initial credibility evaluation
Manual cross-referencing across multiple disconnected source databases
Existing tools producing opaque scores with no underlying evidence
Real-time news cycles outpacing manual verification
No scalable path to systematic content verification across an organization
The Core Problem
A small team cannot fact-check at the pace of the news cycle if every article eats hours of an analyst's time. They needed a Employee that does the source-checking, cross-referencing, and bias analysis itself and returns a verdict in seconds, with the evidence visible and auditable underneath so an editor can defend it in print.
What We Built
Our Solution
We built an AI Employee, a multi-stage credibility platform, that takes a URL and returns a fully documented verdict in under 30 seconds, doing the source-checking a trained analyst used to spend hours on. Every score is backed by evidence the analyst can read on the page.
01. Intelligent Web Scraping & Content Extraction
We built a multi-technology extraction layer that handles the messy reality of the web: JavaScript-heavy single-page applications, paywalls, dynamic content, weird metadata. Playwright handles JS-rendered pages, BeautifulSoup handles static, and a content-hash dedup step cuts processing overhead by 40%.
Playwright headless browser for JavaScript-rendered and dynamic content
Comprehensive metadata harvesting including publication dates and author profiles
WHOIS integration for domain age and ownership verification
Content hash deduplication reducing processing overhead by 40%
02. Source Reputation Intelligence
We built a structured domain reputation database with dynamic trust scoring that evaluates the publication source independently of the content. A piece can read credible and still come from a domain registered last week with no track record. The score reflects both.
Comprehensive domain reputation database with multi-factor trust scoring
WHOIS domain age and ownership verification wired in automatically
Social media presence analysis as a secondary credibility signal
Dynamic threshold adjustment based on domain type and subject matter
03. NLP Content Analysis Engine
We built a custom NLP layer that processes extracted content across multiple analytical dimensions in parallel: logical consistency, emotional manipulation patterns, bias detection. BERT and OpenAI handle reading comprehension and pattern recognition. Their output feeds the scoring engine, it does not become the score.
Input: extracted article text, metadata, and publication context
Processing: BERT bias detection, OpenAI pattern analysis, manipulation flagging
Output: logical consistency score, manipulation flags, claim list, bias indicators
04. Multi-Source Cross-Reference Verification
We built a real-time cross-reference layer that takes specific factual claims and checks them against external sources: Google SERP, Perplexity, established databases, reliable news outlets. Contradictions get surfaced. Citation quality gets graded.
Google SERP and Perplexity API integration for real-time source comparison
Automated factual claim extraction with targeted cross-reference checking
Contradiction detection between article claims and verified external sources
Citation quality analysis assessing reliability of referenced sources
05. Confidence Score Computation & Evidence Generation
This is the part that matters. Every input from the upstream layers (reputation, NLP, cross-reference, citation quality) feeds into a deterministic weighted scoring algorithm. The final confidence number is computed, with full evidence documentation surfaced alongside it. Same article in, same score out, every time.
Deterministic weighted confidence calculation from multi-layer verified inputs
Risk-adjusted scoring with topic-sensitive threshold calibration
Full evidence documentation with specific content highlights and annotations
Progressive status updates so users see what the engine is doing in real time
Tech Stack
Technologies Used
| Layer | Technology | Role |
|---|---|---|
| Backend Services | Cloud Functions + Cloud Run | URL handling, scraping, analysis services |
| Web Scraping | BeautifulSoup4, Playwright | Static and JS-rendered content extraction |
| NLP / AI Layer | OpenAI + BERT | Pattern analysis, bias detection, claim extraction |
| Compute Layer | Deterministic Python Engine | All confidence score calculation and aggregation |
| External Verification | Google SERP + Perplexity API | Real-time cross-reference and contradiction detection |
| Frontend | React.js | Browser extension and web application interfaces |
| Cloud Platform | Google Cloud Platform | Serverless plus containerized, auto-scaling |
| Security | HTTPS + Secure API Auth | End-to-end encryption, rate limiting, audit trails |
Why We Built a Computation Layer
A credibility score is consequential. Editors decide what to publish on it. Researchers cite sources on it. Educators evaluate content on it. The score has to be defensible, which means it has to be auditable, which means it has to be computed against evidence rather than generated.
An LLM asked to produce a confidence number will produce something that reads like a credibility score. But that number is the model's prediction of what such a score should look like given the prompt, shaped by training data and prompt wording. Run the same article through the same model an hour later and you may get a different number, with no way to explain why.
So we routed every score through a deterministic engine. OpenAI and BERT handle reading and pattern recognition, that is what they are good at. The scoring layer takes those structured findings, applies the weighted formula, and produces the final number. Same inputs, same score. The reasoning is visible. The evidence is attached. The result holds up to review.
The Transformation
Before & After Dojo Labs
Before
Hours of expert time per article for initial credibility evaluation
Manual cross-referencing across disconnected source databases
Black-box tools producing scores with no underlying evidence
Verification bottlenecks under high content volume
Specialist training required to operate the tools
After
Full assessment in under 30 seconds
Automated real-time cross-reference against reliable sources
Every score backed by specific evidence and content highlights
Auto-scaling platform handling concurrent requests
Accessible browser extension usable by non-specialist team members
Roadmap
What's Next
We architected the platform from day one to extend without touching the core scoring infrastructure. Phase 2 expands detection range and organizational integration:
Adaptive misinformation models with continuous retraining on emerging manipulation patterns
Organization-level verification history with trend analysis and pattern identification
API tier for direct integration into media and research platforms
Custom thresholds so organizations define their own credibility cutoffs
Batch verification for processing entire content watchlists in parallel
Phase 2 adds capability, the core scoring layer stays untouched.
Ready to build something like this?
Book a 30-minute call. We'll discuss where your AI handles numbers, identify hallucination risks, and map out your computation layer.
Book a Free Discovery Call