title: "Why We Built a Browser-Based Video Processing Tool" description: "The technical story behind VideosKit: how WebCodecs API, mediabunny, and modern browser capabilities enabled us to build video tools that are 3-5x faster than ffmpeg.wasm with zero server-side processing." date: "2026-03-18" author: "VideosKit Team" tags: ["webcodecs", "mediabunny", "browser-apis", "performance", "video-processing"] category: id: "engineering" name: "Engineering" featured: true draft: false locale: "en" readingTime: 8 cover: "https://images.unsplash.com/photo-1555066931-4365d14bab8c?w=800&h=400&fit=crop" type: "blog"
Why We Built a Browser-Based Video Processing Tool
When we set out to build video tools for the web, the conventional wisdom was clear: video processing belongs on the server. Upload the file, run FFmpeg, return the result. Simple, proven, and deeply limited by bandwidth, cost, and privacy.
We thought there had to be a better way. Turns out, there is — and it's been sitting in our browsers all along.
The Status Quo: ffmpeg.wasm
For the past few years, ffmpeg.wasm has been the go-to solution for in-browser video processing. It compiles FFmpeg to WebAssembly, giving you access to the full FFmpeg toolkit right in the browser. It works, but it comes with significant drawbacks:
- Large bundle size: ffmpeg.wasm core is ~25MB, with format-specific codecs pushing it to 30MB+
- Slow initialization: Loading a 25MB WebAssembly module before any processing starts
- Single-threaded by default: No access to hardware acceleration
- Memory constraints: WebAssembly linear memory limits (4GB) create issues with large files
- No hardware decoding: Software-only decode/encode, missing GPU acceleration entirely
In our benchmarks, a 50MB MP4 video compression at 70% quality took 45-60 seconds with ffmpeg.wasm. Not terrible, but not great either.
The WebCodecs Approach
The WebCodecs API is a relatively new browser API that provides direct access to the browser's native media codecs — the same ones used for video playback on YouTube, Netflix, and every other streaming platform. Key advantages:
- Hardware acceleration: Uses GPU for decoding and encoding where available
- Zero bundle overhead: No WebAssembly, no compiled binaries — it's a native browser API
- Fast initialization: Available immediately, no loading time
- Full format support: H.264, H.265, VP8, VP9, AV1 — whatever your browser supports
The catch? WebCodecs is low-level. You need to handle demuxing, frame-by-frame encoding, container formatting, and everything in between. That's a lot of plumbing.
Enter mediabunny
We built mediabunny — a lightweight library that wraps WebCodecs with a simple, ergonomic API. It handles the complex parts:
- Automatic demuxing: Detects container format and extracts video/audio streams
- Frame-level control: Access individual frames for precise editing
- Smart re-encoding: Configurable quality presets with real-time progress
- Memory-efficient streaming: Processes frames in chunks, not all at once
Performance Comparison
| Metric | ffmpeg.wasm | mediabunny (WebCodecs) |
|---|---|---|
| Bundle size | 25-30MB | ~15KB |
| Initialization | 3-5s | Instant |
| 50MB compression (70%) | 45-60s | 10-20s |
| Hardware acceleration | ❌ | ✅ |
| Memory usage (50MB file) | ~800MB | ~300MB |
3-5x faster across the board, with dramatically lower memory usage and zero bundle overhead.
Privacy: The Killer Feature
Performance is great, but here's what really sets browser-based processing apart: privacy by architecture.
With server-based tools, you're trusting someone else with your data. Even well-intentioned services have servers that can be breached, subpoenaed, or misconfigured. With VideosKit:
- Files never leave your device — there is literally no server endpoint for file uploads
- No cookies, no tracking, no analytics on file content
- Works offline once loaded — process videos on an airplane, in a restricted network, anywhere
- Enterprise-friendly for sensitive content — no data residency or compliance concerns
The Development Challenges
Building this wasn't without challenges:
Large File Handling
Browsers don't give you random file access. We use the File API + streaming to process videos frame-by-frame without loading the entire file into memory. This required careful buffer management and garbage collection tuning.
Cross-Browser Compatibility
WebCodecs is supported in Chrome 94+, Edge 94+, and Safari 16.4+. Firefox added support in version 130. We detect capability and gracefully fall back to a message for unsupported browsers.
Memory Management
Video frames are large — a single 4K frame is ~33MB uncompressed. We implemented frame pooling and streaming pipelines to keep memory usage under control, even with 4K content.
Progress Reporting
Users need to know how long to wait. We built a callback system that reports progress based on frame position, giving accurate real-time updates.
Open Source
VideosKit and mediabunny are both open source. Check out the code:
- VideosKit: github.com/nicely-gg/video-tools
- mediabunny: github.com/nicely-gg/mediabunny
Contributions, issues, and feedback are welcome.
The Future of Browser-Based Video
WebCodecs is still evolving. AV1 encoding, WebGPU-powered processing, and improved MediaSource Extensions will make browser-based video even more powerful. We're excited to be pushing the boundaries of what's possible.
If you haven't tried VideosKit yet, visit videoskit.cc. Compress, convert, trim, or inspect a video — all in your browser, all for free.