How to put a si with BRDF attribute in to an image block.

[Start1er]

Hi, I want to use custom defined Integrator here to render depth, normal, albedo. But the aov_integrator failed as the Integrator sample() function interface was not matching aov_integrator, So I decided to write recturn the AOV in python interface the following was:

def render_aov(self, scene: mi.Scene,sensor: mi.Sensor,seed=0,spp=0):
        from matplotlib import pyplot as plt
        import numpy as np
        with dr.suspend_grad():
            sampler, spp = self.prepare(sensor=sensor,seed=seed, spp=spp, aovs=self.aovs())
            ray, weight, pos, _ = self.sample_rays(scene, sensor, sampler)
            
            # compute the intersection
            si = scene.ray_intersect(ray)
            active = si.is_valid()
            si = dr.select(si.is_valid(), si, dr.zeros(mi.SurfaceInteraction3f))
            bsdf = si.bsdf()
            roughness = bsdf.eval_roughness(si, active)
            albedo = bsdf.eval_diffuse_reflectance(si,active)
            # Create a film block to store albedo with RGB channels
            block = sensor.film().create_block()
            block.set_coalesce(block.coalesce() and spp >= 4)
            # create a film block to store roughness metallic ect. with one channel
            block_size = sensor.film().size()
            block_offset = mi.ScalarPoint2i(0, 0)
            block_1 = mi.ImageBlock(size=block_size, offset=block_offset, channel_count=2)
            block_1.set_coalesce(block.coalesce() and spp >= 4)
            
            # Store the albedo and roughness in the film block
            albedo_alpha = dr.select(active, mi.Float(1), mi.Float(0))
            roughness_alpha = dr.select(active, mi.Float(1), mi.Float(0))
            block.put(pos, ray.wavelengths, albedo, albedo_alpha)
            block_1.put(pos, ray.wavelengths, roughness, roughness_alpha)
            
            # Develop the film to get the final images
            sensor.film().put_block(block)
            sensor.film().put_block(block_1)
            albedo_image = sensor.film().develop()
            roughness_image = sensor.film().develop()

            return {
                    'albedo': albedo_bitmap,
                    'roughness': roughness_bitmap
                }

• The question is the var block works fine when I put albedo with 3 channel RGB into it. But the roughness is only have one channel, so I create with block_1 which I want to put one channel. But it failed, I was struggling by how to render one channel AOV into a image block such as roughness, depth?
• Any response would be helpful

[Start1er]

By the way, the reason why I customly render aovs in python is when I use the mitsuba offitial here aov_integrator interface plugin

    aov_integrator = mi.load_dict({
            'type': 'aov',
            'aovs': 'albedo:albedo, normals:sh_normal, dd.y:depth',
            'integrator': {'type': 'sdf_prb_reparam'},
        })
     aovs = mi.render(scene, seed=frame, sensor=sensor, integrator=aov_integrator,
                                spp=spp)

The integrator.render() function here raised a error:

Traceback (most recent call last):
 Lib\site-packages\mitsuba\python\util.py", line 522, in render
    return dr.custom(_RenderOp, scene, sensor, params, integrator,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Lib\site-packages\drjit\router.py", line 5776, in custom
    output = inst.eval(**{ k: _dr.detach(v) for k, v in kwargs.items() })
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Lib\site-packages\mitsuba\python\util.py", line 377, in eval
    return self.integrator.render(
           ^^^^^^^^^^^^^^^^^^^^^^^
TypeError: SdfPrbReparamIntegrator.sample() missing 3 required positional arguments: 'state_in', 'reparam', and 'active'

It looks like the mitsuba offitial aov_integrator interface has not match the custom integrator by this work's integrator.
So I maybe reimplement the aov by an additional python function

[njroussel]

Hi @Start1er

The SDF integrator that you've been linking to (this one), was written for an older version of Mitsuba. Since the time that this was implemented, the interfaces have changed a bit, hence the error messages you've been seeing.

The lastest release, does support aovs in custom Python integrators natively through mi.render(). So you don't need to re-implement a render_aov, but you should rather change your custom integrator to use the new interfaces. Have a look at the integrators in this folder if you need examples of the new interfaces.

[Start1er]

Hi，
Thanks for your kindly help! Also I will try you advice to change the Integrator interface to adapt the new version. But this gona a little bit complex as new version integrator delete the interface for reparameterizations algorithm. So I think render_aov is more suitable? So I also wonder If I put one channel attribute of BRDF (depth, roughness) into RGB image block like albedo. Does this only copy to extending three channels value and do not change original value. If that was true, I can retrive for one of RGB for that one channel value. Or Is there any other way to put one channel into image block?
like:
Could this get me one channel attribute BRDF into image block.

block = sensor.film().create_block()
block.set_coalesce(block.coalesce() and spp >= 4)
roughness_alpha = dr.select(active, mi.Float(1), mi.Float(0))
block.put(pos, ray.wavelengths, roughness, roughness_alpha)
sensor.film().put_block(block)
roughness_rgb = sensor.film().develop()
roughness_one_channel = roughness_rgb[:,:,0]

The problem is If a eval_attribute_1() for some of value like roughness or depth. I have no idea how to put it into a image block