Virtual Fossil Fragmenter

Use Computer Vision and 3D Modeling to Identify Fossil Fragments of Mollusk Shells and Trilobites

Background

You can read more about trilobites in the Trilobite Bits challenge

Large collections of marine fossils from ancient seabeds can be found in deposits of limestone, and often the specimens within have been broken into fragments. If rock deposit is unlithified or only partially lithified (= the grains are not strongly cemented together), the fossils can be sieved out of the deposit. In other cases, it is sometimes possible to use an acidic preparation to dissolve the rock surrounding the fossils, leaving behind a variety of fossil pieces from as many as a dozen or more different taxa. These collections are a great "snapshot" of the ancient marine ecosystem, showing which taxa could be found together in the same locality.

Examples of small fossils in limestone.

But with fragments, it can be difficult to determine if each came from a unique individual or if each individual is represented by many parts. It next becomes a painstaking process to pick through all the fragments to not only identify which taxa are present in the sample, but to determine how many individual specimens from a particular taxon there are. Luckily, because of unique features in the form and structure (morphology) of some of these taxa, we can estimate the abundances of some different taxa from fragmented collections. For example, each Lirobittium rugatum has one aperture and each Cyclocardia occidentalis has one umbo. Therefore if we only count the specimens with apertures or umbos, we know the sample includes only unique individuals.

(Note that each Cyclocardia individual had two shells that were mirror images of each other--thus each individual technically could have contributed two specimens to the collection. Because of this researchers sometimes consider dividing the number of specimens with umbos by 2 when comparing abundances, especially if there are many specimens of the same size. But either way we still need to know the number of specimens with umbos!)

An AMNH collection of fossil fragments from a single rock sample.

---

Solutions

We want to identify these fragmented pieces of mollusk shells and trilobite exoskeleon. The theory is that we could take a 3D model of an organism, programmatically/algorithmically "shatter" the model into pieces, then render and randomly rotate the shattered pieces to generate a training set of images that could possibly identify the fragment using computer vision.

Complete fossil exoskeleton of the trilobite Cryptolithus (here Cryptolithus lorettensis), order Asaphida

We see these as the solutions to this challenge, in order of increasing difficulty:

• A solution that recognizes that every fragment on a control image belongs to one taxon

• A solution that recognizes that there are different fragments on a group image with several taxa

• A solution that correctly recognizes ALL the different taxa on a group image

• A solution that can identify which region of an individual taxa a fragment comes from

• A solution that can detect unique characteristics of individual fragments (umbo, aperture) and return a count of those individuals

Subproject 1: 400-million-year-old trilobites

Fragments of trilobite species Cryptolithus cranidia (here Cryptolithus tesselatus) in two control images on a 1cm grid

What is provided:

• 3D models of Cryptolithus tesselatus 1) cephalon (head), 2) thorax, 3) pygidium (tail)

• Set of images of just cephala, different orientations, different degrees of fragmentation

• Set of images of known mix of specimens including all Cryptolithus tesselatus parts, different orientations, different degrees of fragmentation

WHAT PROPORTION OF FRAGMENTS CAN BE IDENTIFIED AS CRYPTOLITHUS?

Subproject 2: 2-million-year-old mollusk communities

What is provided:

• 3D models of 1) Lirobittium rugatum (gastropod), 2) Cyclocardia occidentalis (bivalve).

• Set of images of just specimens of Lirobittium rugatum, different orientations, different degrees of fragmentation

• Set of images of just specimens of Cyclocardia occidentalis, different orientations, different degrees of fragmentation

• Set of images of known mix of specimens including Lirobittium rugatum and Cyclocardia occidentalis, different orientations, different degrees of fragmentation

HOW MANY OF THE FRAGMENTS ARE IDENTIFIABLE AS LIROBITTIUM OR CYCLOCARDIA?

HOW MANY OF THE FRAGMENTS INCLUDE THE APERTURE (FOR LIROBITTIUM) OR THE UMBO (FOR CYCLOCARDIA)?

---

Resources

Libraries using booleans to split things into fragments

• CSG.js: Constructive Solid Geometry (CSG) JavaScript library
• Cork: Cork is designed to support Boolean operations between triangle meshes (C++)

Volumetric manipulation

• OpenVDB: OpenVDB is an Academy Award-winning C++ library comprising a hierarchical data structure and a suite of tools for the efficient manipulation of sparse, time-varying, volumetric data discretized on three-dimensional grids.

Image segmentation

• Mathworks Segmentation methods in image processing and analysis: code samples and how-tos.
• ImageJ Trainable Weka Segmentation: Neural network trainable image segmentation
• Point Cloud Library: PCL has a bunch of tools to deal with 3D point cloud data and might be used to match or segment objects.

Check out Online Resources and Data Sets for more general purpose software and utilities.

---

Challenge owner: Melanie Hopkins