Issues with hnn2 installation/setup

[salvadord]

From Chris Bailey @cjayb:

"Hi Salva

Re. mpi: I just meant that your GH page doesn’t mention it as a dependency

Re. extgauss note that it’s sort of a relic, not included in any of the “official” HNN models (we decided to omit it for now in the new hnn-core API). The param-file you’re using in the example sets weights to zero.

I upgraded NetPyNE to 0.9.9.1 from pip, and pulled the latest updates to the hnn2 branch off GH. There’s something fishy about how netpyne uses Matplotlib on macOS: import netpyne leads to

Warning: no DISPLAY environment variable.
--No graphics will be displayed.

I traced this to netpyne.init.py. As you know, the macOS and Qt backends don’t require an X subsystem, so it’s a bit odd to see this behaviour. Any ideas?

The spike histogram plots still open up in Bokeh, but look reasonable (meaning external drives are doing their thing). However, simulate_trials seems to be creating an empty simulation (“Connections: 0”), which obviously doesn’t do much (see snippet of output below and trials-folder attached). Given that you get the code to work, the problem must be with my installation. I think we should probably move such discussions to GH instead of polluting inboxes ;)

On the bright side, iplotRaster() now runs without error (though no plot is produced). You win some…

/Chris

Loading custom mechanism files from /Users/au210321/src/git/hnn2/hnn_models/hnn_neocortex/x86_64/.libs/libnrnmech.so

Start time: 2021-03-04 10:22:10.531743

Creating network of 24 cell populations on 1 hosts...
Number of cells on node 0: 1354
Done; cell creation time = 0.83 s.

Gathering data...
Done; gather time = 0.22 s.

Analyzing...
Cells: 1354
Connections: 0 (0.00 per cell)
Plotting 3D cell shape ...
Saving batch to /Users/au210321/src/git/hnn2/hnn_models/hnn_neocortex/data/trials/trials_batch.json ...
(0,) (0,)
('hnn_params', 'prng_seedcore') = 0
Saving simConfig to /Users/au210321/src/git/hnn2/hnn_models/hnn_neocortex/data/trials/trials_0_cfg.json ...
Running job /Users/au210321/src/git/hnn2/hnn_models/hnn_neocortex/data/trials/trials_0
mpirun -n 2 nrniv -python -mpi /Users/au210321/src/git/hnn2/hnn_models/hnn_neocortex/init.py simConfig=/Users/au210321/src/git/hnn2/hnn_models/hnn_neocortex/data/trials/trials_0_cfg.json netParams=/Users/au210321/src/git/hnn2/hnn_models/hnn_neocortex/data/trials/trials_netParams.py"

[salvadord]

Chris from the trial output logs seems there is an mpi error:

Fatal error in MPI_Init_thread: Invalid group, error stack:
MPIR_Init_thread(586)..............:
MPID_Init(224).....................: channel initialization failed
MPIDI_CH3_Init(105)................:
MPID_nem_init(324).................:
MPID_nem_tcp_init(175).............:
MPID_nem_tcp_get_business_card(401):
MPID_nem_tcp_init(373).............: gethostbyname failed, d42672 (errno 0)
(unknown)(): Invalid group

• I'm puzzled because I thought before you were able to run ok the mpi sims since you got the dipole plots. I assume the mod folder is compiled? did you change anything else after updating netpyne and hnn2?

• Are you able to run other NEURON sims using mpiexec -n 2 nrniv -mpi -python [script.py] ?

You can test it using this simple script:

cat >test.py <<EOL
from neuron import h
pc = h.ParallelContext()
id = int(pc.id())
nhost = int(pc.nhost())
print ('I am %d of %d'%(id, nhost))
pc.barrier()
h.quit()
EOL

• If that works, are you able to run a netpyne script in parallel, eg. mpiexec -n 2 nrniv -mpi -python HHTut_run.py (you can clone examples from here: https://github.com/Neurosim-lab/netpyne/tree/development/examples)

[salvadord]

For the display issue, what is the output of echo $DISPLAY? Do you have XQuartz (https://www.xquartz.org/) installed?

A possible workaround if $DISPLAY is empty is to use set it with any value (eg. export DISPLAY=1$).

We had to add that check for displays to avoid errors when running on HPCs, but maybe it doesn't generalize well to your setup.

[cjayb]

My bad: my conda-environment got botched due to (presumably) polluting my base. I nuked the lot and reinstalled.

I tracked the MPI issue to this:

• conda install mpi4py pulls mpich, for which simple tests like the one you proposed work, but NEURON fails
• on macOS (at least mine!), conda install openmpi is needed

The Warning about missing DISPLAY comes from NEURON, I think. I have a relatively new Mac, on which I haven't installed XQuartz yet. I've solved this for now by commenting out matplotlib.use("agg") in netpyne.__init__, as it should not be necessary to install XQuartz to run MPL.

After some fiddling, I finally managed to get some plots done in ipython line by line (the plot_X-functions seem to remain in the background if I run the whole script in one go). Left plot is hnn2, right one HNN/hnn-core:

How do you set the seeds? Does each GID get a unique seed, or do you set a global one? If the former, then it should be possible to set a seedcore to quantitatively reproduce HNN GUI. Note "possible" not "easy" ;)

[salvadord]

Great that almost working now!

re: display - curious if installing XQuartz would solve the issue or not.

re: remain in the background - maybe related to the DISPLAY issue as well? are you running with 'python -i'? Btw, I think for the plots you can set the argument 'saveFig=True' so they generate png files.

re: seeds - there is a global seed parameter, but a different seed is then generated for each gid. As you say it might be possible but not easy, since I believe we might be using different RNG functions. However, in the past I did reproduce almost exactly the hnn plots by feeding in the same spike times for each external feed -- see https://drive.google.com/file/d/1K1frR-zAzk0yx74S9N0Qrhw-nzUMqhtj/view?usp=sharing.