How HDR Restoration Works

A technical deep-dive into the AI-powered process behind HDRPop

Understanding HDR and Gain Maps

To understand how HDRPop restores HDR, it's important to first understand what HDR actually means in the context of modern smartphone photography.

What is HDR?

High Dynamic Range (HDR) refers to images that can display a wider range of luminance values than traditional Standard Dynamic Range (SDR) images. While SDR images are typically limited to brightness levels between 0.1 and 100 nits, HDR images can represent brightness levels from 0.001 to 10,000+ nits.

Modern HDR displays (like iPhone XDR screens, iPad Pros, and high-end monitors) can reproduce this extended range, making highlights significantly brighter while maintaining deep shadows. The result is more realistic, impactful images that better match what the human eye sees in real life.

What is a Gain Map?

The gain map is the secret sauce that enables backward-compatible HDR images. It's a technique standardized in ISO 21496-1 (also known as Ultra HDR JPEG or JPEG-R).

Here's how it works:

  • Base Image (SDR): A standard JPEG that looks normal on regular displays. This is what older devices and apps see.
  • Gain Map (HDR Metadata): A smaller secondary image stored in JPEG metadata that describes how much brighter each pixel should be on HDR displays. It's essentially a "brightness multiplier map."
  • HDR Reconstruction: When an HDR-capable device opens the image, it combines the base image with the gain map to reconstruct the full HDR version with extended dynamic range.

Key Insight: The gain map is what AI editors strip away. They process the base SDR image and output a new JPEG without the HDR metadata, losing the brightness enhancement data in the process.

The HDR Restoration Pipeline

HDRPop's restoration process involves several stages, from upload to final HDR output. Here's the complete technical pipeline:

1

Image Upload & Validation

Your photo is uploaded via HTTPS with end-to-end encryption. We validate the file format (JPEG, PNG, etc.), check for corruption, and enforce size limits (20MB max, configurable pixel count). Images are stored temporarily in an isolated filesystem with automatic cleanup.

2

Preprocessing & Normalization

The image is decoded and converted to a standardized color space (typically RGB in linear space). We apply tone mapping to normalize exposure and ensure the image is in the optimal range for the AI model. Images larger than 4096px are resized to prevent memory issues while maintaining quality.

3

GMNet AI Model Inference

The preprocessed image is fed into the GMNet (Gain Map Network) deep learning model. This convolutional neural network was trained on thousands of HDR image pairs and learned to predict the missing gain map from SDR input. The model analyzes luminance, color gradients, and local contrast to generate a high-quality gain map prediction.

4

Gain Map Generation & Refinement

The raw model output is post-processed to ensure physical plausibility (no negative gains, smooth transitions) and perceptual quality. We apply edge-preserving smoothing and validate against HDR-VDP (Visual Difference Predictor) metrics to ensure the gain map looks natural on HDR displays.

5

Ultra HDR JPEG Encoding

The base image and gain map are combined into an ISO 21496-1 compliant Ultra HDR JPEG file. The gain map is stored in the JPEG's XMP metadata using the standard format recognized by iOS, Android, and other platforms. The file is then compressed with high-quality JPEG encoding (quality 95).

6

Validation & Delivery

The final HDR JPEG is validated for structural correctness (proper JPEG markers, valid XMP metadata, correct gain map dimensions) and queued for download. We surface key metadata and readiness status in the results page, then make the file available immediately. The original and temporary files are scheduled for automatic deletion within 24 hours.

The GMNet AI Model

Architecture & Training

GMNet is a deep convolutional neural network designed specifically for inverse tone mapping and gain map prediction. It was published in peer-reviewed research at ICLR 2025 and represents the state-of-the-art in HDR restoration.

Key architectural features:

  • Dual-branch architecture: The model uses separate global and local branches to capture both overall scene lighting and fine-grained details.
  • Global branch: Processes downsampled images to understand overall luminance distribution and scene characteristics.
  • Local branch: Focuses on high-resolution details using residual blocks and attention mechanisms.
  • Attention mechanisms: Uses spatial and channel attention to focus on important regions for HDR enhancement.

Research Citation: "Learning Gain Map for Inverse Tone Mapping" (ICLR 2025). The model achieves state-of-the-art performance on HDR reconstruction benchmarks with an average PSNR of 38.2 dB and SSIM of 0.96 on test datasets.

Model Performance

On our production infrastructure, GMNet typically processes images in 2-3 seconds on average, depending on resolution:

Image Resolution Processing Time Memory Usage
1080p (1920x1080) ~1.5 seconds ~600 MB
4K (3840x2160) ~2.5 seconds ~1.2 GB
8K+ (downscaled to 4K) ~3 seconds ~1.5 GB

Quality Assurance

Validation Metrics

Every processed image goes through automated quality checks before delivery:

  • Structural Validation: Verify JPEG structure, XMP metadata presence, and gain map dimensions
  • Gain Map Range: Check that gain values are physically plausible (typically 1.0x to 8.0x)
  • File Size: Confirm output is reasonable (not corrupted or too large)
  • Degenerate Detection: Ensure gain map has sufficient variation (not all zeros or all max)

Fallback System: If GMNet fails or is unavailable, we automatically fall back to our synthetic gain map generator that creates reasonable HDR enhancement based on luminance analysis.

Edge Cases & Limitations

While GMNet is highly accurate, there are some scenarios where HDR restoration may be less effective:

  • Extremely dark or bright images: Photos with very narrow dynamic range (e.g., completely black frames) don't benefit much from HDR enhancement.
  • Heavily edited/stylized images: Images with extreme filters, posterization, or artistic effects may not restore naturally since they diverge significantly from natural photography.
  • Low-resolution images: Images below 800x600 pixels may not have enough detail for the model to generate high-quality gain maps.
  • Synthetic/CGI images: Computer-generated images without natural lighting may produce less realistic HDR effects.

Platform Compatibility

Ultra HDR JPEG Support

The ISO 21496-1 Ultra HDR JPEG format is supported by:

Platform HDR Display Support Notes
iOS 14+ (Photos app) ✅ Full Support iPhone XS and newer, iPad Pro (2018+)
macOS (Preview, Photos) ✅ Full Support Macs with XDR displays (Pro Display XDR, MacBook Pro 2021+)
Android 14+ ✅ Full Support Requires HDR-capable display hardware
Windows 11 ⚠️ Limited Requires HDR10-capable display and compatible app
Web Browsers ❌ Not Supported Browsers render base SDR image only (no gain map)

Backward Compatibility: Ultra HDR JPEGs look like normal high-quality JPEGs on devices that don't support HDR. There's no downside to using HDR—it's purely additive.

Privacy & Security

Data Handling Pipeline

Your photos flow through a security-conscious pipeline designed to minimize risk:

  1. Encrypted Upload: HTTPS with TLS 1.3, end-to-end encryption
  2. Temporary Storage: Files stored in isolated temp directories with restricted permissions
  3. Processing in Memory: Image data loaded into RAM for processing, never written to persistent storage unnecessarily
  4. Automatic Deletion: Cron job runs every hour to delete files older than 24 hours
  5. No Training Data: Your photos are never used for model training or retained beyond processing
  6. Zero Logging of Image Content: We log technical metadata (resolution, format, processing time) but never log image content

Technical Implementation Details

Direct Ultra HDR Creation

Unlike other solutions that rely on external libraries, HDRPop directly constructs Ultra HDR JPEG files by creating Multi-Picture Format (MPF) containers with:

  • SDR Base Image: The main JPEG that looks normal on regular displays
  • Gain Map Image: Auxiliary JPEG containing the brightness enhancement data
  • MPF Metadata: Links the two images together according to CIPA DC-X007 standard
  • HDR Metadata: Gain map parameters (min/max values, gamma, offsets)

Model Architecture Details

The GMNet implementation uses a sophisticated dual-branch architecture:

  • Global Branch: Downsampling layers (down1, down2) + 5 residual blocks for scene understanding
  • Local Branch: High-resolution processing with 3 sets of 5 residual blocks each
  • Attention Modules: Spatial and channel attention (SQ_Module) for selective enhancement
  • Mask Estimation: Learned masks to focus enhancement on appropriate regions
  • Output Generation: PixelShuffle upsampling + final convolution for gain map production

Future Improvements

Roadmap

We're continuously improving HDRPop. Here's what's on the horizon:

  • Custom Model Training: Fine-tuning GMNet on specific photography styles (portraits, landscapes, architecture) for even better results
  • Video HDR Support: Extending the pipeline to handle video files with HDR10 and Dolby Vision standards
  • Real-Time Preview: Using WebGPU to simulate HDR tone mapping in browsers (challenging due to display limitations)
  • Batch Processing API: RESTful API for developers to integrate HDR restoration into their own workflows
  • Custom Gain Map Adjustments: User controls for fine-tuning HDR intensity and tone curve

Technical Resources

For Developers

Interested in the technical details or want to build on top of HDRPop?

  • ISO 21496-1 Specification: Official standard for Ultra HDR JPEG format
  • GMNet Research Paper: "Learning Gain Map for Inverse Tone Mapping" (ICLR 2025)
  • libultrahdr Library: Open-source library for reading/writing Ultra HDR JPEGs
  • HDR-VDP Quality Metric: Perceptual quality metric for HDR images

Questions? If you're a developer or researcher interested in HDR technology, feel free to reach out at [email protected]. We love talking about HDR, tone mapping, and computational photography!

Ready to See It in Action?

Upload your first photo and experience AI-powered HDR restoration

Try HDRPop Free →