↓ Skip to main content
EN

日本語

English

  1. Projects/

WACV 2026: DM3Net: Dual-Camera Super-Resolution via Domain Modulation and Multi-scale Matching

·485 words·3 mins
GUAN CONG
Author
GUAN CONG
A student studying Computer Science at Waseda University. Interested in Machine Learning, Natural Language Processing, and Computer Vision.
Table of Contents
A new image restoration approach for the era of multi-camera smartphones

Overview
#

alt text
In recent years, multi-camera devices such as smartphones and action cameras can simultaneously capture images with different focal lengths, such as wide-angle and telephoto.

However, these images have field of view (FoV) mismatches and resolution/color domain differences, making high-definition reconstruction difficult through simple image fusion.

The proposed DM³Net (Dual-camera SR via Domain Modulation and Multi-scale Matching) addresses this challenge by combining “multi-stage feature matching” with “global domain alignment,” achieving super-resolution performance that surpasses conventional methods.

Background and Challenges
#

In smartphone dual cameras:

  • Wide-angle camera: Covers a wide field of view but has low resolution
  • Telephoto camera: Has a narrow field of view but contains high-resolution details

The goal of Dual-Camera Super-Resolution is to use the telephoto image as a “reference image” to enhance the wide-angle image to high definition. Conventional methods (e.g., DCSR, KeDuSR) transfer high-frequency information through patch-level feature matching, but had problems with reduced detail reproduction due to scale constraints.

Proposed Method: Key Components of DM³Net
#

alt text
DM³Net consists of the following three elements:

1. Multi-Scale Matching
#

Calculates feature patch similarity at multiple receptive field scales (1/4, 1/2, 1) and selects the optimal reference patch. This significantly improves structural consistency and detail reproduction compared to conventional single-scale matching.

alt text
Furthermore, a “Key Pruning” strategy was introduced to remove redundant patches with high similarity. This reduced GPU memory consumption by approximately 17% while cutting inference time to one-third.

2. Domain Modulation
#

Wide-angle images (LR) and telephoto images (Ref) have domain differences due to different capture pipelines. Therefore, DM³Net learns global domain embeddings (z, zgt) from each pair (LR-GT, LRc-Ref). By introducing a “domain-aware loss (L_domain)” that aligns the distributions of both and adaptively incorporating it into the reconstruction process, smooth mapping to the high-resolution domain is achieved.

3. Integrated Reconstruction
#

Finally, high-frequency features reconstructed through matching and domain-modulated embedding information are integrated to generate visually natural and high-fidelity HR images.

Experimental Results
#

alt text
On three real-world datasets (DuSR-Real, RealMCVSR-Real, CameraFusion-Real), DM³Net achieved the following performance:
alt text
In particular, it maintained high accuracy even in non-overlapping regions (Corner Region), confirming robust operation against partial field of view and color differences.

Ablation Studies and Discussion
#

  • Multi-scale: PSNR improves as scales increase (27.75 dB).
  • Domain modulation: Best performance when using both z and zgt.
  • Key Pruning: Optimal balance at threshold 0.7 and sampling interval 16.

From these results, it is analyzed that DM³Net’s performance improvement comes from the combination of improved local matching accuracy and introduction of global domain alignment.

Applications and Future Prospects
#

DM³Net is directly applicable to multi-lens capture environments such as smartphones, drones, and vehicle-mounted cameras. In particular, it shows high potential in Computational Photography technology that integrates different angles of view, resolutions, and ISP processing across hardware.

In the future, model lightweighting and real-time processing are targeted, with application to diffusion-based restoration and video extension expected.