H.265/HEVC Support in RTMP: From Extended Specifications to Practical Implementation

Since its inception in 2002, RTMP (Real-Time Messaging Protocol) has been the de facto standard for live streaming push, owing to its low latency, stable connections, and broad device compatibility. However, the traditional RTMP specification only supports the H.264 video codec. With the advent of the 4K/8K ultra-high-definition era, H.265/HEVC, offering 30%-50% higher compression efficiency than H.264, has become a key technology for reducing bandwidth costs and improving video quality. How to support H.265 while maintaining the advantages of the RTMP ecosystem has become a critical topic in the streaming media field.

Technical Evolution: From Private Extensions to Enhanced RTMP

Limitations of Traditional Extension Schemes: CodecID Extension

Before the introduction of the Enhanced RTMP specification, CDN vendors commonly supported H.265 by extending the CodecID field in the VideoTag – setting CodecID to 12 to identify HEVC. While this approach solved the basic problem of "having or not having" support, it had significant drawbacks: the 4-bit CodecID space was limited and could not accommodate expansion needs for more modern codecs like VP9 and AV1. Furthermore, implementation differences among vendors led to cross-platform compatibility issues.

Enhanced RTMP: A Standardized Solution

The Enhanced RTMP specification, officially released in July 2023, completely resolved this issue through the FourCC extension mechanism. Its core design includes:

  • 1.IsExHeader Flag: An extension header identifier is added to the first byte of the VideoTagHeader. When this flag is set to 1, the FourCC mode is enabled.
  • 2.FourCC Codec Identifier: Four-character codes (e.g., hvc1 for HEVC) are used to replace the original CodecID, reserving ample space for future codecs (VP9, AV1, etc.).
  • 3.PacketType Extensions: New types are defined, such as SequenceStart (for transmitting the HEVCDecoderConfigurationRecord) and CodedFrames (for carrying complete NALU frames), maintaining consistency with H.264's processing logic.

This design achieves backward compatibility – legacy players can still recognize H.264, while players supporting Enhanced RTMP can seamlessly decode HEVC.

Practical Challenges: Key Issues in Implementing H.265 over RTMP

1. Client Compatibility Bottlenecks

Despite the clear advantages of H.265's encoding efficiency, client-side support remains imperfect. Chrome browser has supported HEVC hardware decoding since version 105, but users need to verify its availability via chrome://gpu. Safari's support for HEVC is relatively good, but it relies on the operating system's underlying hardware decoding capabilities.

2. Fragmentation of the Playback Ecosystem

Support for H.265 decoding varies significantly across different platforms:

  • Browser Side: mpegts.js already supports Enhanced RTMP HEVC, but hls.js and dash.js have not yet fully caught up.
  • Mobile Side: Android MediaCodec and iOS VideoToolbox provide good support for H.265 hardware decoding, but lower-end devices may experience performance bottlenecks.
  • Players: VLC supports H.265 over HTTP-TS/SRT, but its support for HEVC over RTMP/FLV remains limited.

3. Server-Side and Toolchain Adaptation

FFmpeg has natively supported HEVC over RTMP since version 6.0, but earlier versions required patches. OBS has supported HEVC over RTMP since version 29, but it requires hardware encoder support. Open-source servers like SRS 6.0 have fully embraced Enhanced RTMP, but compatibility verification with CDN vendors is still necessary during actual deployment.

Happytimesoft RTMP Server's Practical Solution

Addressing the ecosystem fragmentation issues mentioned above, Happytimesoft RTMP Server provides a comprehensive solution for practical implementation:

  • Comprehensive Codec Compatibility: Supports both H.264 and H.265 video encoding, along with audio formats like AAC and G.711, meeting the decoding requirements of various terminals.
  • Protocol Conversion Capability: Can convert H.265 streams transmitted via RTSP or SRT to RTMP in real-time, enabling unified access for heterogeneous devices.
  • Lightweight Cross-Platform Design: Developed in C/C++, it can run on resource-constrained devices like embedded Linux, making it suitable for H.265 push streaming at edge nodes.
  • Dual-Mode Support: Compatible with both the traditional CodecID extension mode used by domestic CDN vendors and the latest Enhanced RTMP international specification, ensuring stable operation across different CDN environments.

Future Outlook

The emergence of Enhanced RTMP revitalizes the RTMP protocol within the new generation of audio-visual systems. As more efficient codecs like AV1 and H.266 become widespread, the FourCC extension mechanism reserves ample space for the future evolution of RTMP. For developers, choosing H.265 requires weighing the benefits against the costs: in scenarios with clear terminal support for H.265 (such as internal surveillance systems or controlled remote teaching environments), it can be confidently adopted to save up to 50% on bandwidth costs; for public-facing distribution scenarios, fallback solutions like H.264 or server-side transcoding should be considered.

The true technical value lies not in rigid adherence to protocols, but in the ability to operate stably within complex and ever-changing engineering scenarios, creating tangible value for users. The journey of H.265 over RTMP, from private extensions to international specifications, is a vivid microcosm of streaming media technology evolving from being merely "usable" to truly "well-crafted."