Real-time neuronal traces:
hours → seconds.

66×
faster inference
60s → 0.91s
27h→25m
full pipeline run time
1wk→5h
training time. Even after training on
100× more samples
Labeled neurons on top of the single-color channel they were labeled from. (Missing neurons are often not labeled enough in training data to have high confidence predictions.)

Why C. elegans?

C. elegans is a 1 mm roundworm with 302 stereotyped, individually-named neurons. It's small enough to map completely, but still surprisingly capable: finding food, avoiding harm, adapting when hungry or sick, associating specific temperatures or smell as positive or negative stimuli. 

We record the entire brain, as it senses, processes, and acts.

Rethinking the Approach

A moving worm results in blurry, dim, single-color frames. The old pipeline non-rigidly warps every frame onto a crisp, four-color reference of an immobilized worm, then labels that reference. The warp is expensive and fundamentally cannot run online. Instead, I made it possible to label each moving frame directly, removing alignment altogether.

To do that, I needed a network which could perform on the lower quality freely moving (FM) data. This meant being able to train on the hundreds of thousands of volumes we'd recorded in the lab, however the current network took days to train on just 81 volumes.

BEFORE
Each moving frame → register to a data-dense reference → label the reference. Accurate, but hours behind the experiment.
AFTER · ACLL
Label each moving frame directly, on its single dim channel. No reference, no warping — fast enough to run during the recording.
Training cut a week → 5 hours — even training on 100× more data; the full run will happen later
More data to learn mono-color, dim, moving frames — allowing direct labeling
Moving labeling removed the need for registration — no reference warp needed
Merged Post-Processing & Inference —  Sheer correction, Channel Alignment, Segmentation, Trace Extraction
Faster inference and Direct Trace Output —  1.2s after folding in the post-processing compution

Inference-Specific Improvements

a.
Trained on sheared data directly
Removed a standalone shear-correction step (>1s) by adapting ACL to the raw geometry.
b.
Smart channel alignment 
From a few leading frames instead of per-frame — a 2.5s/frame cost dropped to near zero.
c.
ROIs straight from predictions
Extracted regions of interest from the network's output algorithmically, removing an entire segmentation stage.
d.
Custom acquisition integration
A custom NIS Elements hook pulls frames off the microscope in real time, working with the existing system and procedures.

Testing & Tradeoffs

Dropping the segmentation step makes ACLL's traces noisier than the old ground-truth pipeline, but they are aimed at entirely different and complementary use cases.

ACLL enables entirely new experiments, but also provides a backup for experiments when immobilization or alignment fails (a common occurrence). Since more time and compute will almost always win out, ANTSUN still remains the benchmark C. elegans pipeline.

Network confidence correlates with accuracy across held-out datasets, providing automatic flagging of inaccurate predictions. Additionally, intra-recording quality is more stable than inter-recording, which is a useful characteristic.

The Scope of My Work

As a solo project,  I built on the work of Atanas, Lu (2025), overhauling the pipeline, from architecture to training to live deployment.

—Network architecture, loss, & hyperparams
—Training efficiency, memory usage, & scaling
—Inference-time systems & memory optimization
—Live NIS Elements acquisition integration
—Validation & confidence tooling for lab use
All neurons (scroll) - ANTSUN traces, ACLL traces | Confidence & Correctness across frames and recordings
Full trace stack — every neuron's activity for a whole recording