Visualisation

!pip install vfb-connect --upgrade

# Import libs and initialise API objects
from vfb_connect.cross_server_tools import VfbConnect
import pandas as pd

vc = VfbConnect()  

import pymaid
import navis

# Needed because deepnote doesn't support fancy progress bars yet
navis.set_pbars(jupyter=False) 
pymaid.set_pbars(jupyter=False)

# Connect to the VFB CATMAID server hosting the FAFB data
rm = pymaid.connect_catmaid(server="https://fafb.catmaid.virtualflybrain.org/",  
                            api_token=None, max_threads=10)

# Test call to see if connection works 
print(f'Server is running CATMAID version {rm.catmaid_version}')

WARNING: Could not load OpenGL library.
INFO  : Global CATMAID instance set. Caching is ON. (pymaid)
Server is running CATMAID version 2020.02.15-905-g93a969b37

Plotting

navis lets you plot neurons in 2d using matplotlib (nice for figures), and in 3d using either plotly when in a notebook environment like Deepnote or using a vispy-based 3D viewer when using a Python terminal. The vispy solution won’t work in Deepnote so we will focus on matplotlib’s 2d and plotly for 3d.

# We'll use a FAFB neuron, retrieved using navis as an example
n = pymaid.get_neurons(16) # retrieves a CatMaid neuron object with skeleton id = 16 (See the Mapping notebook for how to find )
n

INFO  : Cached data used. Use `pymaid.clear_cache()` to clear. (pymaid)
INFO  : Cached data used. Use `pymaid.clear_cache()` to clear. (pymaid)

# We can plot this in 2D
navis.plot2d(n)

navis.plot2d(m)

(<Figure size 432x432 with 1 Axes>, <Axes3DSubplot:>)

# Or 3D
navis.plot3d(n)

Navigation:

• left click and drag to rotate (select “Orbital rotation” above the legend to make your life easier)
• mousewheel to zoom
• middle-mouse + drag to translate
• click legend items (single or double) to hide/unhide

Customization:

The above plots are very basic examples but there are a ton of ways to tweak things to your liking. For a full list of parameters check out the docs for plot2d and plot3d. (Hint - you can view documentation by floating over a method name in Deepnote)

Let’s for example change the colors. In general, colors can be:

a string - e.g. “red” or just “r” an rgb/rgba tuple - e.g. (1, 0, 0) for red

navis.plot3d(n, color='red')

Excercise

Float over the

# CATMAID neuron objects include connectors (synapses)  here pre-(red) and postsynapses (blue).
navis.plot3d(n, width=1000, connectors=True, color='k') #

Now let’s try an example with multiple neurons

# Pulling (annotations on) neurons from a single dataset on CATMAID

bates = pymaid.find_neurons(annotations='Paper: Bates and Schlegel et al 2020') 
# Viewing first 10 neurons:
bates[0:10]

INFO  : Found 583 neurons matching the search parameters (pymaid)

# Plotting multiple neurons, we get a default multi-color pallete

bates[0:10].plot3d(hover_name=True)

What if we want to use color to group neurons in some way?

The following example illustrates colouring by lineage, where lineage is derived from standardised neuron names on FAFB CATMAID

da1

# On CATMAID, we need to know something about the structure of names and parse them to get types:

import re 
prog = re.compile("Uniglomerular(.*?) DA1 ") 

# Match all neuron names in `bates` against that pattern
is_da1 = list(map(lambda x: prog.match(x) != None, bates.name))

# Subset list of neurons
da1 = bates[is_da1]
da1.head()

# Add lineage attribute to all da1 neurons based on regex against name
for n in da1:
    # Split name into components and keep only the tract
    n.lineage = n.name.split(' ')[3]    

import seaborn as sns
import numpy as np 

# Get all unique lineages (we expect only two)
lineages = np.unique(da1.lineage) 

# Generate a color per lineage
lin_cmap = dict(zip(lineages, sns.color_palette('muted', len(lineages))))

# Make a dictionary mapping skid to colour appropriate to lineage
lineage_map = dict(zip(da1.id, da1.lineage)) 
neuron_cmap = {i: lin_cmap[l] for i, l in lineage_map.items()}

navis.plot3d(da1, color=neuron_cmap, legend_group=lineage_map, hover_name=True, width=1000)

We can do the same thing using VFB, looking up the relevant terms on the web site and then using these to categorise CATMAID neurons

# With VFB, we can query by type

from vfb_connect.cross_server_tools import gen_short_form # TODO - make this arg on oc.get_instances

def vfb_type_2_skids(vfb_type):
    # Get IDs (short_forms) for instances of type
    da1_from_vfb = map(gen_short_form, vc.oc.get_instances(vfb_type, query_by_label=True))
    # Find which neurons are in catmaid_fafb and return 
    da1_skid_lookup = vc.neo_query_wrapper.vfb_id_2_xrefs(da1_from_vfb, db='catmaid_fafb', reverse_return=True)
    # Convert skids to ints 
    da1_skids = da1_skid_lookup.keys()
    return list(da1_skids)

da1_fafb = vfb_type_2_skids("'adult antennal lobe projection neuron DA1'")

Running query: FBbt:00048096
Query URL: http://owl.virtualflybrain.org/kbs/vfb/instances?object=FBbt%3A00048096&prefixes=%7B%22FBbt%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_%22%2C+%22RO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_%22%2C+%22BFO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FBFO_%22%7D&direct=False
Query results: 31
/root/venv/lib/python3.7/site-packages/vfb_connect/neo/query_wrapper.py:279: UserWarning:

The following IDs do not match DB &/or id_type constraints: {'VFB_0010121x', 'VFB_00101203', 'VFB_0010122z', 'VFB_00101202', 'VFB_jrchjtdd', 'VFB_jrchjtdg', 'VFB_00101201', 'VFB_0010122p', 'VFB_0010124l', 'VFB_00101205', 'VFB_jrchjtde', 'VFB_0010123b', 'VFB_jrchjtda', 'VFB_0010122y', 'VFB_jrchjtdc', 'VFB_00101204', 'VFB_00101199', 'VFB_0010124e', 'VFB_0010122k', 'VFB_00104624', 'VFB_jrchjtdh', 'VFB_jrchjtdb', 'VFB_00102297', 'VFB_00101200', 'VFB_jrchjtdf', 'VFB_00102763', 'VFB_0010126e', 'VFB_00102294', 'VFB_00104629', 'VFB_0010122m', 'VFB_00103943'}

# Inspect subclasses:
pd.DataFrame.from_records(vc.get_subclasses("'adult antennal lobe projection neuron DA1'", summary=True))

Running query: FBbt:00048096
Query URL: http://owl.virtualflybrain.org/kbs/vfb/subclasses?object=FBbt%3A00048096&prefixes=%7B%22FBbt%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_%22%2C+%22RO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_%22%2C+%22BFO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FBFO_%22%7D&direct=False
Query results: 2

# Get skids for DA1_lPNs and DA1_vPNs
da1_l_fafb = vfb_type_2_skids("'DA1_lPN'")
da1_v_fafb = vfb_type_2_skids("'DA1_vPN'")

is_da1 = list(map(lambda x: x in da1_fafb, bates.skeleton_id))
da1 = bates[is_da1]

Running query: FBbt:00067363
Query URL: http://owl.virtualflybrain.org/kbs/vfb/instances?object=FBbt%3A00067363&prefixes=%7B%22FBbt%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_%22%2C+%22RO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_%22%2C+%22BFO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FBFO_%22%7D&direct=False
Query results: 22
/root/venv/lib/python3.7/site-packages/vfb_connect/neo/query_wrapper.py:279: UserWarning:

The following IDs do not match DB &/or id_type constraints: {'VFB_00101203', 'VFB_00101202', 'VFB_0010122z', 'VFB_jrchjtdd', 'VFB_jrchjtdg', 'VFB_00101201', 'VFB_0010122p', 'VFB_0010124l', 'VFB_00101205', 'VFB_jrchjtde', 'VFB_0010123b', 'VFB_jrchjtda', 'VFB_0010122y', 'VFB_jrchjtdc', 'VFB_00101204', 'VFB_00101199', 'VFB_0010124e', 'VFB_0010122k', 'VFB_jrchjtdb', 'VFB_00101200', 'VFB_jrchjtdf', 'VFB_0010126e'}

Running query: FBbt:00067372
Query URL: http://owl.virtualflybrain.org/kbs/vfb/instances?object=FBbt%3A00067372&prefixes=%7B%22FBbt%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_%22%2C+%22RO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_%22%2C+%22BFO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FBFO_%22%7D&direct=False
Query results: 9
/root/venv/lib/python3.7/site-packages/vfb_connect/neo/query_wrapper.py:279: UserWarning:

The following IDs do not match DB &/or id_type constraints: {'VFB_00103943', 'VFB_jrchjtdh', 'VFB_0010121x', 'VFB_00102763', 'VFB_00104624', 'VFB_00102294', 'VFB_00104629', 'VFB_0010122m', 'VFB_00102297'}

# Generate a palette 
palette = sns.color_palette('muted', 2)
# Make a dictionary 
lin_cmap = { 'lPN': palette[0], 'vPN': palette[1]}

# Make a dictionary with key = skid & value = color by lineage
lineage_map = {n: 'lPN' for n in da1_l_fafb}
lineage_map.update({n: 'vPN' for n in da1_v_fafb})

neuron_cmap = {n: lin_cmap[l] for n, l in lineage_map.items()}

navis.plot3d(da1, color=neuron_cmap, hover_name=True, legend_group=lineage_map, width=1000)

Using VFB templates and data to plot neurons from multiple sources in a single template space.

VFB allows us to pull neurons from multiple sources and display them in a single template space

You can browse the various templates available on the VFB site, or get a summary:

pd.DataFrame.from_records(vc.neo_query_wrapper.get_templates(summary=True))

DA1_manifest = vc.get_images_by_type("'adult antennal lobe projection neuron DA1'", template = 'JRC2018Unisex', image_folder = 'DA1', stomp=True)

Running query: FBbt:00048096
Query URL: http://owl.virtualflybrain.org/kbs/vfb/instances?object=FBbt%3A00048096&prefixes=%7B%22FBbt%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_%22%2C+%22RO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_%22%2C+%22BFO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FBFO_%22%7D&direct=False
Query results: 31
/root/venv/lib/python3.7/site-packages/vfb_connect/neo/query_wrapper.py:220: UserWarning:

No 'swc' file found for 'ALv1_P02(DA1)_0_2018U'.

DA1_manifest[0:3]

We can then plot the neurons, coloring by data_source

# Read skeletons from file
nl = navis.read_swc('DA1') 

# The neuron names correspond to the filenames
# To make it easier, we will manually add IDs from the manifest
ids = dict(zip(DA1_manifest.filename.map(lambda x: x.replace('.swc', '')).values,
               DA1_manifest.accession.values)) 
for n in nl:
    n.id = ids[n.name]

# Add data source 
ds = DA1_manifest.set_index('accession').data_source.to_dict()
for n in nl:
    n.source = ds[n.id]

source_cmap = dict(zip(np.unique(nl.source),
                       sns.color_palette('muted', len(np.unique(nl.source)))))

source_map = dict(zip(nl.id, nl.source))
neuron_cmap = {n.id: source_cmap[n.source] for n in nl}

navis.plot3d(nl, color=neuron_cmap, hover_name=True, legend_group=source_map, width=1000)

Adding volumes

VFB has regions of interest defined for every template. These can be browsed on our template ROI browser:

or the results of queries available from each template. The ROIs for JRC2018Unisex come from Janelia (the work of Kazonori Shinomiya) and align with domains on the Hemibrain following standard cross-registration.

# Hackish!
jrc2018u = trimesh.load_remote('https://github.com/schlegelp/navis-flybrains/raw/main/flybrains/meshes/JRC2018U.ply')
jrc2018u = navis.Volume(jrc2018u, color=(.85, .85, .85, .3), name='JRC2018U')
jrc2018u

<navis.Volume(name=JRC2018U, color=(0.85, 0.85, 0.85, 0.3), vertices.shape=(11943, 3), faces.shape=(24558, 3))>

# Get an image (obj volume) of the mushroom body calyx on JRC2018Unisex: 

SMP = vc.neo_query_wrapper.get_images(['VFB_00102170'], template = 'JRC2018Unisex', image_folder='SMP', image_type='obj', stomp=True) # TODO 
SMP

# Get an image (obj volume) of the mushroom body calyx on JRC2018Unisex: 

MB = vc.get_images_by_type("'VFB_00101567'", template = 'JRC2018Unisex', image_folder='MB', image_type='obj', stomp=True) # TODO 
MB

Running query: FBbt:00003685
Query URL: http://owl.virtualflybrain.org/kbs/vfb/instances?object=FBbt%3A00003685&prefixes=%7B%22FBbt%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_%22%2C+%22RO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_%22%2C+%22BFO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FBFO_%22%7D&direct=False
Query results: 6

# Plot this along with the neurons in the previous example
import trimesh

# TBA # discovery step

test = trimesh.load('SMP/SMP_on_JRC2018Unisex_adult_brain.obj')
mesh = navis.Volume(test)
mesh.color = (230, 230, 230, .2)
mesh.name = "SMP on JRC2018Unisex adult brain"
navis.plot3d([nl, mesh], color=neuron_cmap, hover_name=True, legend_group=source_map)

# Plot this along with the neurons in the previous example
import trimesh

# TBA # discovery step

#test = trimesh.load('MB/CA_on_JRC2018Unisex_adult_brain.obj')
#mesh = navis.Volume(test)
#mesh.color = (230, 230, 230, .2)
#mesh.name = "ME on JRC2018Unisex adult brain"
navis.plot3d([nl, jrc2018u], color=neuron_cmap, hover_name=True, legend_group=source_map)

Excercise

Choose some neurons to display and render by classification. Write some code based on the previous examples to achieve this.