Why microscopy and not flow cytometry?

One means to collect single-cell data is flow cytometry. Modern instruments are powerful but (i) cannot interrogate individual cells repeatedly to produce time series for each cell, (ii) cannot collect a great deal of light due to the short time (typically microseconds) that the cell passes the detector, and (iii) typically do not capture images of the cells, making it difficult to analyze cell shape, size and intracellular localization of fluorescence.

Optical microscopy can compensate for some limitations of flow cytometry by providing abilities to revisit individual cells over time, collect emitted light for long times and capture cell images with high resolution.

Many automated microscope-based cytometry relies on two commercial software packages, Metamorph (Molecular Devices Corporation) and ImagePro (Media Cybernetics, Inc.), to operate the microscopes, collect the data and analyze them. These packages, often used together with more general purpose analysis programs, such as Matlab (The Mathworks, Inc.) and Labview (National Instruments Corporation), probably constitute the state of the art in commercial software used for these purposes. Likewise, open-source projects can provide valuable tools for image analysis. Examples include the Open Microscopy Environment (OME), which provides file formats and metadata standards for microscope images, Image J/Fiji, a Java-based package of microscope image analysis tools, and CellProfiler, a great pipeline-based batch construction tool.

Remarks on ground truth creation:

When one decide to provide the ground truth for a set of images, the question arises: what is a threshold to mark a cell? Sometimes in dense cell colonies it is difficult to say if we should penalize/reward an algorithm for missing/finding a 'cell' because it is barely visible. It is important to find a good threshold for that. Cells near the edges of the image are another problem as many algorithms discard these cells by design, so they should not be penalized for that. In order to deal with it we additionally marked the cells near the edges so the evaluation does not reward for finding them nor penalized for missing them.

The images below show which cells we mark (green) which we don't (red) and which cells are considered 'border' cells (orange):

For every test set two additional test subsets were created in order to test how the performance of the tool changes when:
  • the image sequence of lower temporal resolution is used (denoted with suffix 't')
  • the images used are noisy (denoted with suffix 'n')
Lower temporal resolution was achieved by taking every other image thus doubling the time interval (up to 6 minutes). Noisy images were created by adding Gaussian noise to the original sequence using ImageJ's Gaussian noise option with standard deviation set to 400.

Further ideas about program comparison

Some further ideas how comparisons are performed here.