update viewer state in loop

[sheridana]

when running a neuroglancer instance from an IPython notebook, is it possible to refresh the viewer state each iteration in a loop? simple example:

## cell 1
viewer = neuroglancer.Viewer()

## cell 2
def create_batch(data):
    
    data =  skimage.util.random_noise(
        data,
        var=random.uniform(0, 0.5)
    )
    
    layer = neuroglancer.LocalVolume(
                data=data,
                offset=[0,]*3,
                voxel_size=[1,]*3)

    layer_data = neuroglancer.ImageLayer(
                    source=layer
                )
        
    return layer_data

## cell 3
data = zarr.open(sample, 'r')['volumes/raw'][0:100,0:100,0:100]

## cell 4
layer_data = create_batch(data)

with viewer.txn() as s:
    s.layers['raw'] = layer_data   

which works as expected, each time cell 4 is run, the viewer is updated. however, wrapping cell 4 content in a loop fails when retrieving data:

for i in range(3):
    layer_data = create_batch(data)

    with viewer.txn() as s:
        s.layers['raw'] = layer_data

which throws Error retrieving metadata for volume... 404: Not Found

tried invalidating the layer, adding a time delay, deleting the layer before adding, and updating the viewer state, but no success. the desired behavior is to refresh the data in the browser as it is updated in the loop. is this possible?

[jbms]

I'm not able to reproduce this issue with the current version of Neuroglancer. I see from your code though that you are using an older version, from before Oct 2019. Quite a lot has changed since then, and it is possible that this particular bug was fixed by one of the changes since then.

[sheridana]

sorry, should have clarified the versions.

steps to replicate (tested on ubuntu 18.04, conda 4.8.5) using publicly hosted fib25 dataset:

1. clean conda env with python 3 + jupyter

2. pip install neuroglancer==1.1.2 scikit-image==0.17.2 cloud-volume

3. run jupyter notebook

cells:

##cell 1

import cloudvolume
import neuroglancer
import numpy as np
import random
import skimage

##cell 2

neuroglancer.set_server_bind_address('0.0.0.0')

##cell 3

viewer = neuroglancer.Viewer()
viewer ## open in new window

##cell 4

vol = cloudvolume.CloudVolume('https://storage.googleapis.com/neuroglancer-public-data/flyem_fib-25/image')

offset = [int(i/2) for i in vol.info['scales'][0]['size']]

data = vol[
    offset[0]:offset[0]+100,
    offset[1]:offset[1]+100,
    offset[2]:offset[2]+100]

data = np.transpose(data[...,0], [2,1,0])

##cell 5

def create_batch(data,offset):
    
    data =  skimage.util.random_noise(
        data,
        var=random.uniform(0, 0.3)
    )
    
    layer = neuroglancer.LocalVolume(
                data=data,
                offset=offset,
                voxel_size=[8,]*3)

    layer_data = neuroglancer.ImageLayer(
                    source=layer
                )
            
    return layer_data

##cell 6

# works as expected, shows single batch
with viewer.txn() as s:
    layer_data = create_batch(data,offset)
    s.layers['raw'] = layer_data

##cell 7

# works but only shows single batch
for i in range(2):
    with viewer.txn() as s:
        layer_data = create_batch(data,offset)
        s.layers['raw'] = layer_data

##cell 8

# fails retrieving data for first batch, adds second batch
for i in range(2):
    layer_data = create_batch(data,offset)
    with viewer.txn() as s:
        s.layers['raw'] = layer_data

also tried with invalidating the layer, but same results. desired behavior:

1st pass:

• create data
• add data to viewer (should see this as it's added)

2nd pass:

• create data
• update data in viewer (should see the updated data)

maybe it's easier to see what's happening with a pause:

import time

for i in range(3):
    layer_data = create_batch(data,offset)
    with viewer.txn() as s:
        s.layers['raw'] = layer_data
        time.sleep(1)

the viewer tries to retrieve data for each iteration before appending but fails to find the data. it should retrieve the data then append, repeat. i assume this is because of the asynchronous nature, but am wondering if there is a simple workaround? (not a javascript expert). if it's something that's solved in the current neuroglancer version i would still be interested in seeing the relevant code

[jbms]

The fix was in 90291e8 , I had forgotten about that issue.

[sheridana]

yep that solved it, thanks a lot!