Introduction

Imaging systems—from smartphone cameras to medical MRI scanners and self-driving car sensors—produce measurements that often look nothing like the final images humans view. The raw data may be encoded in ways that are impossible to interpret directly, yet artificial intelligence (AI) can extract valuable information from these measurements. Traditionally, we evaluate imaging systems by metrics such as resolution or signal-to-noise ratio (SNR), but these metrics treat quality factors independently and cannot capture the true usefulness of the data. Alternatively, training neural networks to reconstruct or classify images conflates hardware performance with algorithm quality. This guide presents a step-by-step method for directly evaluating and optimizing imaging systems based on the information content they provide, as described in the NeurIPS 2025 paper “Information-Driven Design of Imaging Systems.” By estimating mutual information directly from noisy measurements, you can compare systems, optimize designs, and achieve state-of-the-art results with less memory and compute, without the need for task-specific decoders.

What You Need

Before starting, ensure you have the following prerequisites:

• Imaging system description – The encoder (optical system) that maps objects to noiseless images.
• Noisy measurements – Actual sensor outputs corrupted by noise (e.g., photon shot noise, read noise).
• Noise model – A probabilistic description of how noise corrupts the measurements (e.g., Gaussian, Poisson).
• Object model (optional) – While our method avoids explicit object models, having a statistical prior can improve estimation if available.
• Computational tools – A framework for high-dimensional mutual information estimation (e.g., neural network-based estimators such as MINE or InfoNCE).
• Basic understanding of information theory – Familiarity with mutual information and the concept of uncertainty reduction.

Step-by-Step Guide

Step 1: Describe Your Imaging System

Start by formally defining the imaging system as an encoder. This encoder maps objects (e.g., scenes, tissues, obstacles) to noiseless images. In practice, you need to know the optical transfer function (OTF), sensor response, and any sampling patterns. For example, an MRI scanner collects data in k-space (frequency domain) using specific pulse sequences. Write down the forward model:

Noiseless image = Encoder(object)

This step is crucial because it sets the basis for what information can be transmitted. Record the physical constraints (lens diffraction, pixel size, noise floor).

Step 2: Collect Noisy Measurements

Run your imaging system to obtain actual measurements. These are the raw outputs before any processing—think raw sensor data from a smartphone camera before demosaicing, or frequency-space data from an MRI scanner before reconstruction. Store multiple measurements if possible, as our method works with a set of noisy samples. Note that these measurements may look nothing like the final images; that’s fine because we focus on information, not visual quality.

Example: For a self-driving car’s LiDAR, collect point clouds under various lighting conditions.

Step 3: Characterize the Noise Model

Identify the dominant sources of noise and describe them probabilistically. Common noise models include:

• Gaussian noise – additive, sensor read noise.
• Poisson noise – signal-dependent, shot noise.
• Combined models – e.g., Gaussian + Poisson.

You do not need to know the true object distribution; only the conditional distribution of measurements given the noiseless image. For example, if noise is additive Gaussian, then: Measurement | Noiseless Image ~ Normal(Noiseless Image, σ²). Estimate σ from calibration data.

Step 4: Estimate Mutual Information from Measurements

This is the core step. We use the noisy measurements and noise model to estimate the mutual information I(Object; Measurement)—how much the measurement reduces uncertainty about the object. Our approach avoids the need for explicit object models or unconstrained channel capacity calculations. Instead, we directly estimate information from the sample pairs:

1. Prepare data pairs: If you have ground truth objects (synthetic or controlled), pair each noisy measurement with its corresponding object. If not, use repeated measurements of the same object (e.g., multiple shots of a static scene).
2. Choose estimator: Use a neural network-based mutual information estimator (e.g., the MINE estimator or InfoNCE). Alternatively, for low-dimensional cases, a k-nearest neighbor estimator can work.
3. Train the estimator: Feed pairs (object, measurement) into the estimator to learn the mutual information. Thanks to the noise model, the estimator learns the mapping even if objects are high-dimensional.
4. Compute the estimate: After training, apply the estimator to your test data to obtain a scalar value—the information content in nats or bits.

This single number captures the combined effect of resolution, noise, sampling, and all other factors. A blurry, noisy image that preserves essential features can have higher mutual information than a sharp clean image that loses those features.

Step 5: Compare or Optimize Imaging Systems

With the mutual information metric, you can now evaluate different imaging system designs. Two systems with the same mutual information are equivalent in their ability to distinguish objects, even if their measurements look completely different. Use the metric to:

• Compare hardware designs: A lens with better resolution but higher noise vs. a lens with lower resolution but less noise—which yields more information? Compute and compare.
• Optimize parameters: Vary system parameters (aperture, exposure time, sensor gain, sampling pattern) and compute mutual information for each combination. Choose the design that maximizes the metric.
• Validate against end-to-end learning: Our paper shows that maximizing mutual information produces designs matching state-of-the-art end-to-end methods (which train a decoder for a specific task) while requiring less memory and compute. So you can skip training task-specific decoders.

Example: In an MRI system, optimize the k-space trajectory to maximize information per unit time.

Tips for Success

• Embrace the “weird” measurements: The information metric works even if the raw data is not human-interpretable. Don’t rely on visual inspection—trust the numerical estimate.
• Do not conflate hardware and software: Traditional approaches train a neural network on top of a fixed hardware design, making it hard to separate the contributions. Our method evaluates hardware alone, so you can improve the optics separately from the processing.
• Watch out for estimator bias: Mutual information estimators can be biased, especially with limited data. Use techniques like cross-validation or ensemble estimation to reduce bias.
• Generalize across domains: The same framework works for smartphone imaging, MRI, LiDAR, and more. The key is having a noise model and access to raw measurements—not final images.
• Start simple: Test on a synthetic system (e.g., a simple Gaussian channel) to validate that your estimator returns the correct information value before applying to a real system.
• Use the metric for co-design: Once you have an information estimate, you can jointly optimize hardware and a post-processing algorithm (if needed) by using the information metric as the objective, avoiding task-specific constraints.

By following these steps, you can directly evaluate and optimize imaging systems based on what truly matters: the information they provide. This approach unifies previously separate quality metrics, accounts for noise and resolution simultaneously, and eliminates the need for expensive end-to-end training. For full technical details and experimental results across four imaging domains, refer to the NeurIPS 2025 paper.