# Case study: mdingest

URL-to-Markdown ingestion API for LLM workflows. Cloudflare Workers + Containers. NestJS + Fastify.

**Live:** https://mdingest.knightker.workers.dev · **Source:** github.com/LOsioChico/mdingest

## Problem

Articles carry paywalls, popups, and DOM noise. LLM ingestion pipelines need clean Markdown + metadata, not HTML.

## One core, four doors

Same ingestion logic behind HTTP API, CLI, MCP stdio, and MCP HTTP — services are plain classes that work with `new`, no framework boot required.

| Endpoint | Provider |
|---|---|
| GET /v1/medium?url= | Medium (via Freedium, dual-source) |
| GET /v1/devto?url= | Dev.to (Forem API) |
| GET /v1/substack?url= | Substack (free posts) |

## Decisions and what they beat

- Cloudflare Containers over pure Workers: NestJS needs a real Node runtime; Workers' subset loses Fastify ecosystem.
- Dual-source Freedium fetch: ~20% of fetches return placeholder content; retry up to 5x and keep the cleanest (~99.97% clean rate).
- Zod-validated DTOs + a single provider detector shared by frontend and all three backend DTOs.
- Error contract: every error shapes to { code, message, details?, traceId } including RATE_LIMITED (30 req/min per IP).

## What I would do differently

- Add Redis/Durable Objects for cache persistence across container sleeps.
- Provider-specific rate limits before they become incidents.