Virtual Fossil Fragmenter

Use Computer Vision and 3D Modeling to Identify Fossil Fragments of Shells, Bivalves, Trilobites

Background

You can read more about trilobites in the Trilobite Bits challenge

Large collections of marine fossils from ancient seabeds can be found in deposits of limestone, and often the specimens within have been broken into fragments. If rock deposit is unlithified or only partially lithified (= the grains are not strongly cemented together), the fossils can be sieved out of the deposit. In other cases, it is sometimes possible to use an acidic preparation to dissolve the rock surrounding the fossils, leaving behind a variety of fossil pieces from as many as a dozen or more different species. These collections are a great "snapshot" of the ancient marine ecosystem, showing which species could be found together in the same locality.

Examples of small fossils in limestone.

But with fragments, it can be difficult to determine if each came from a unique individual or if each individual is represented by many parts. It next becomes a painstaking process to pick through all the fragments to not only identify which species (or taxa) are present in the sample, but to determine how many individual specimens from a particular taxon there are. Luckily, because of unique features in the form and structure (morphology) of some of these species, we can estimate the abundances of some different species from fragmented collections. For example, each Lirobitticum has one aperture and each Cyclocardia has one umbo. Therefore if we only count the specimens with apertures or umbos, we know the sample includes only unique individuals.

An AMNH collection of fossil fragments from a single rock sample.

---

Solutions

We want to identify these fragmented pieces of shells, bivalves, and trilobites. The theory is that we could take a 3D model of an organism, programmatically/algorithmically "shatter" the model into pieces, then render and randomly rotate the shattered pieces to generate a training set of images that could possibly identify the fragment using computer vision.

Complete fossil exoskeleton of the trilobite Cryptolithus (here Cryptolithus lorettensis), order Asaphida

We see these as the solutions to this challenge, in order of increasing difficulty:

• A solution that recognizes that every fragment on a control image belongs to one taxon / species

• A solution that recognizes that there are different fragments on a group image with several taxa

• A solution that correctly recognizes ALL the different taxa on a group image

• A solution that can identify which region of an individual species a fragment comes from

• A solution that can detect unique characteristics of individual fragments (umbo, aperture) and return a count of those individuals

Subproject 1: 400-million-year-old trilobites

Fragments of trilobite species Cryptolithus cranidia (here Cryptolithus tesselatus) in two control images on a 1cm grid

What is provided:

• 3D models of Cryptolithus tesselatus 1) cephalon (head), 2) thoracic segments, 3) pygidium (tail)

• Set of images of just cephala, different orientations, different degrees of fragmentation

• Set of images of known mix of specimens including all Cryptolithus tesselatus parts, different orientations, different degrees of fragmentation

WHAT PROPORTION OF FRAGMENTS CAN BE IDENTIFIED AS CRYPTOLITHUS?

Subproject 2: 2-million-year-old mollusk communities

What is provided:

• 3D models of 1) Lirobitticum rugatum (gastropod), 2) Cyclocardia occidentalis (bivalve), need to be stl or ply files?

• Set of images of just specimens of Lirobitticum rugatum, different orientations, different degrees of fragmentation

• Set of images of just specimens of Cyclocardia occidentalis, different orientations, different degrees of fragmentation

• Set of images of known mix of specimens including Lirobitticum rugatum and Cyclocardia occidentalis, different orientations, different degrees of fragmentation

HOW MANY OF THE FRAGMENTS ARE IDENTIFIABLE AS LIROBITTICUM OR CYCLOCARDIA?

HOW MANY OF THE FRAGMENTS INCLUDE THE APERTURE (FOR LIROBITTICUM) OR THE UMBO (FOR CYCLOCARDIA)?

---

Resources

Libraries using booleans to split things into fragments

• CSG.js: Constructive Solid Geometry (CSG) JavaScript library
• Cork: Cork is designed to support Boolean operations between triangle meshes

C++ volumetric manipulation

• OpenVDB: OpenVDB is an Academy Award-winning C++ library comprising a hierarchical data structure and a suite of tools for the efficient manipulation of sparse, time-varying, volumetric data discretized on three-dimensional grids.

Image segmentation

• Mathworks Segmentation methods in image processing and analysis: code samples and how-tos.
• ImageJ Trainable Weka Segmentation: Neural network trainable image segmentation
• Point Cloud Library: PCL has a bunch of tools to deal with 3D point cloud data and might be used to match or segment objects.

Check out Online Resources and Data Sets for more general purpose software and utilities.

---

Challenge owner: Melanie Hopkins