Solving Real-Time RTSP-to-WebRTC Video Latency & Keycloak Auth Bottlenecks

The Engineering Problem

In modern IoT and industrial surveillance applications, handling video feeds and access control at scale creates two major infrastructure bottlenecks:

1. RTSP Stream Latency: Standard IP cameras broadcast over RTSP (Real-Time Streaming Protocol). Browsers cannot natively play RTSP streams without high-latency HLS transposing (8–15 second delay) or heavy client plugins.
2. Keycloak Authentication Spikes: When hundreds of concurrent clients request stream access, token verification requests against Keycloak endpoints create CPU throttling and HTTP 429 rate limit exceptions.

During my work developing containerized video infrastructure and microservices in Germany, I engineered a production solution that dropped stream latency to under 350ms while scaling authentication seamlessly.

---

1. Converting RTSP to Low-Latency WebRTC / HLS

To eliminate the 10+ second HLS buffer delay without requiring custom browser extensions, we introduced a media gateway container utilizing go2rtc and ffmpeg inside Docker.

+----------------+      RTSP       +-------------------+     WebRTC / WSS     +----------------+
| IP Camera Feed | --------------> | Media Gateway Container | ------------------> | Web Browser UI |
+----------------+  (H.264/AAC)   +-------------------+    (< 350ms Latency)   +----------------+

Key Configuration Insights

• Protocol Fallback: The gateway negotiates WebRTC via WebSocket signalling first. If network NAT environments block UDP candidate pairs, it seamlessly falls back to Low-Latency HLS (LL-HLS).
• Hardware Acceleration: Docker containers pass host GPU devices (/dev/dri) directly to ffmpeg workers, reducing CPU overhead by 65%.

---

2. Keycloak JWT Authentication & Caching

Instead of validating every incoming video segment against Keycloak's /protocol/openid-connect/userinfo or introspect endpoints, we implemented a public-key validation model at the API Gateway level.

import jwt
from jwt import PyJWKClient

# Cache Keycloak JWKS (JSON Web Key Set) locally to eliminate remote auth calls
jwks_client = PyJWKClient("https://auth.example.com/realms/production/protocol/openid-connect/certs")

def verify_request_token(token_string: str):
    try:
        signing_key = jwks_client.get_signing_key_from_jwt(token_string)
        payload = jwt.decode(
            token_string,
            signing_key.key,
            algorithms=["RS256"],
            audience="media-service",
            options={"verify_exp": True}
        )
        return payload
    except jwt.PyJWTError as e:
        raise PermissionError(f"Unauthorized token: {str(e)}")

Operational Impact

• Zero Auth Latency Overhead: Local RS256 signature verification takes < 1ms instead of a 120ms round-trip network call to Keycloak.
• Resilience: Even during Keycloak service restarts or high load, existing valid JWTs continue to authenticate smoothly.

---

Technical Takeaways

1. Use WebRTC over RTSP for true real-time streaming (< 500ms latency).
2. Validate JWTs statelessly via RS256 JWKS caching rather than calling Keycloak /introspect per request.
3. Pass hardware device mounts (/dev/dri) into containerized video processing pipelines to free up system CPU for application logic.