Challenges with 3D Reconstruction of Dune Areas Using MRO CTX Data Due to Variable Lighting Conditions

[squidward-qwq]

Hello Community,

I am currently engaged in a project that involves 3D reconstruction of dune areas using data from the Mars Reconnaissance Orbiter (MRO) Context Camera (CTX). I have encountered a significant challenge due to the variable lighting conditions in these regions. The images produced by following the guidelines in section 8.3 of the MRO CTX documentation result in poor quality, as demonstrated in the attached image.

In an attempt to improve the results, I have applied knowledge from section 6, 'The next steps', of the documentation to refine my algorithm. Unfortunately, the improvements are not substantial, and areas with significant lighting changes still exhibit null values.

Additionally, I discovered an algorithm based on ASP called "CASP-GO" which generates stereo image pairs. However, when I try to use the provided stereo.ctx configuration directly, I encounter numerous errors in the terminal. Below is the configuration I attempted to use:

# -*- mode: sh -*-

# Pre-Processing / stereo_pprc
################################################################

# Pre-alignment options
#
# Available choices are (however not all are supported by all sessions):
#    NONE           (Recommended for anything map projected)
#    EPIPOLAR       (Recommended for Pinhole Sessions)
#    HOMOGRAPHY     (Recommended for ISIS wide-angle shots)
#    AFFINEEPIPOLAR (Recommended for ISIS narrow-angle and DG sessions)
alignment-method none

# Intensity Normalization
# force-use-entire-range true      
# Use entire input range to disable normalization, i.e. use raw values to pass directly to correlation algorithms.

# individually-normalize false
# nodata-value -32768 will not be used if it is less than the hard coded actual nodata value, though it will always displayed as -32768 in ArcMap for ASP derivatives.
# nodata-optimal-threshold-factor 0.1 
# skip-image-normalization true

# Select a preprocessing filter:
#
# 0 - None
# 1 - Subtracted Mean
# 2 - Laplacian of Gaussian (recommended)
prefilter-mode 2

# Kernel size (1-sigma) for pre-processing
#
# Recommend 1.4 px for Laplacian of Gaussian
# Recommend 25 px for Subtracted Mean
prefilter-kernel-width 1.4

# Integer Correlation / stereo_corr
################################################################
corr-seed-mode 1
# Select a cost function to use for initialization:
#
# 0 - absolute difference (fast)
# 1 - squared difference  (faster .. but usually bad)
# 2 - normalized cross correlation (recommended)
cost-mode 0

# Initialization step: correlation kernel size
corr-kernel 55 55

# Initializaion step: correlation search range
#
# Uncomment the following to use explicit search ranges. Otherwise, a
# value will be choosen for you.
# corr-search -100 -100 100 100
# corr-search -125 -3 125 3
# xcorr-threshold 2
#use-local-homography true DO NOT USE as it creates tile artefact

# Subpixel Refinement / stereo_rfne
################################################################

# Subpixel step: subpixel modes
#
# 0 - disable subpixel correlation (fastest)
# 1 - parabola fitting (draft mode - not as accurate)
# 2 - affine adaptive window, bayes EM weighting (slower, but much more accurate)

# disable-v-subpixel
subpixel-mode 2
subpixel-em-iter 15
subpixel-affine-iter 5
# subpixel-pyramid-levels 3

# Subpixel step: correlation kernel size
subpixel-kernel 75 75
#25 25
#
# Post Filtering / stereo_fltr
################################################################

# Fill in holes up to 100,000 pixels in size with an inpainting method
# disable-fill-holes
#
# filter-mode 1
# rm-half-kernel 7 7
# max-mean-diff 1
# enable-fill-holes true
# fill-holes-max-size 100000
#100000

# Automatic "erode" low confidence pixels
# rm-min-matches 60
# rm-threshold 3
# rm-cleanup-passes 1

# Triangulation / stereo_tri
################################################################

# Size max of the universe in meters and altitude off the ground.
# Setting both values to zero turns this post-processing step off.
near-universe-radius 0.0
far-universe-radius 0.0

[oleg-alexandrov]

Sorry for the pain. Stereo cannot work, even in principle, with images with different illumination. There is not much we can do about that. Stereo works based on correlation of images that are the same except for perspective change. CTX has a lot of data. It is strongly suggested to get stereo pairs with similar illumination. Consider also using HiRISE. CASP-GO is listed in the experimental section. It will not help improve CTX processing.

…

On Thu, May 9, 2024 at 7:12 AM squidward ***@***.***> wrote: Hello Community, I am currently engaged in a project that involves 3D reconstruction of dune areas using data from the Mars Reconnaissance Orbiter (MRO) Context Camera (CTX). I have encountered a significant challenge due to the variable lighting conditions in these regions. The images produced by following the guidelines in section 8.3 of the MRO CTX documentation result in poor quality, as demonstrated in the attached image. image.png (view on web) <https://github.com/NeoGeographyToolkit/StereoPipeline/assets/88065797/c39b002d-099b-4bc9-bbb2-1dc02cd77131> image.png (view on web) <https://github.com/NeoGeographyToolkit/StereoPipeline/assets/88065797/be4a5e07-bb8b-4868-8904-9c4b8de95314> In an attempt to improve the results, I have applied knowledge from section 6, 'The next steps', of the documentation to refine my algorithm. Unfortunately, the improvements are not substantial, and areas with significant lighting changes still exhibit null values. Additionally, I discovered an algorithm based on ASP called "CASP-GO" which generates stereo image pairs. However, when I try to use the provided stereo.ctx configuration directly, I encounter numerous errors in the terminal. Below is the configuration I attempted to use: # -*- mode: sh -*- # Pre-Processing / stereo_pprc################################################################ # Pre-alignment options## Available choices are (however not all are supported by all sessions):# NONE (Recommended for anything map projected)# EPIPOLAR (Recommended for Pinhole Sessions)# HOMOGRAPHY (Recommended for ISIS wide-angle shots)# AFFINEEPIPOLAR (Recommended for ISIS narrow-angle and DG sessions) alignment-method none # Intensity Normalization# force-use-entire-range true # Use entire input range to disable normalization, i.e. use raw values to pass directly to correlation algorithms. # individually-normalize false# nodata-value -32768 will not be used if it is less than the hard coded actual nodata value, though it will always displayed as -32768 in ArcMap for ASP derivatives.# nodata-optimal-threshold-factor 0.1 # skip-image-normalization true # Select a preprocessing filter:## 0 - None# 1 - Subtracted Mean# 2 - Laplacian of Gaussian (recommended) prefilter-mode 2 # Kernel size (1-sigma) for pre-processing## Recommend 1.4 px for Laplacian of Gaussian# Recommend 25 px for Subtracted Mean prefilter-kernel-width 1.4 # Integer Correlation / stereo_corr################################################################ corr-seed-mode 1# Select a cost function to use for initialization:## 0 - absolute difference (fast)# 1 - squared difference (faster .. but usually bad)# 2 - normalized cross correlation (recommended) cost-mode 0 # Initialization step: correlation kernel size corr-kernel 55 55 # Initializaion step: correlation search range## Uncomment the following to use explicit search ranges. Otherwise, a# value will be choosen for you.# corr-search -100 -100 100 100# corr-search -125 -3 125 3# xcorr-threshold 2#use-local-homography true DO NOT USE as it creates tile artefact # Subpixel Refinement / stereo_rfne################################################################ # Subpixel step: subpixel modes## 0 - disable subpixel correlation (fastest)# 1 - parabola fitting (draft mode - not as accurate)# 2 - affine adaptive window, bayes EM weighting (slower, but much more accurate) # disable-v-subpixel subpixel-mode 2 subpixel-em-iter 15 subpixel-affine-iter 5# subpixel-pyramid-levels 3 # Subpixel step: correlation kernel size subpixel-kernel 75 75#25 25## Post Filtering / stereo_fltr################################################################ # Fill in holes up to 100,000 pixels in size with an inpainting method# disable-fill-holes## filter-mode 1# rm-half-kernel 7 7# max-mean-diff 1# enable-fill-holes true# fill-holes-max-size 100000#100000 # Automatic "erode" low confidence pixels# rm-min-matches 60# rm-threshold 3# rm-cleanup-passes 1 # Triangulation / stereo_tri################################################################ # Size max of the universe in meters and altitude off the ground.# Setting both values to zero turns this post-processing step off. near-universe-radius 0.0 far-universe-radius 0.0 — Reply to this email directly, view it on GitHub <#433>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKDU3H7PKMBKQ4YQMLXHJLZBN73XAVCNFSM6AAAAABHO4Q53CVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZWGYZTKNBVGE> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[squidward-qwq]

Thank you for your response, Oleg. However, I have a question regarding the DEMs of Martian dune areas. I noticed that some product data, such as those from LPL (Lunar & Planetary Laboratory), feature clear representations of dune sections. Could you please explain how they manage to generate DEMs of dunes without noise?

[oleg-alexandrov]

I don't know what software they use or what images they use. If you can identify those dunes on Mars, can try to search for images of that area, and pick some pairs with similar illumination and good stereo convergence angle.

It is also suggested experimenting with mapprojection and some of our stereo algorithms (https://stereopipeline.readthedocs.io/en/latest/next_steps.html).

Dunes can be tricky as they may somewhat different from different perspectives, unlike solid rock, and not sure if they have a clear pattern or are just uniform in color and texture, which could also be problematic.

[squidward-qwq]

Okay, thank you very much