Caltech researchers have developed a new Artificial Intelligence tool called CellSAM, designed to identify cells in images across a broad range of biological applications. The work emerged from a collaboration between the laboratories of David Van Valen, assistant professor of biology and biological engineering, and Yisong Yue, professor of computing and mathematical sciences. A paper describing the research appears in the journal Nature Methods.
Imaging plays a central role in biology, from identifying cancerous cells in biopsies to observing immune cells such as macrophages as they track and destroy pathogens. Distinguishing and labeling individual cells in images and videos has traditionally been a labor-intensive process done manually or with algorithms that require significant correction. CellSAM is intended to replace that fragmented approach with a single model that can work across many different use cases, helping researchers identify cell types, determine where they are located, and analyze how they interact with neighboring cells.
Biological images vary widely and can reveal very different phenomena, including tumor cells hidden among tissues and bacteria that secrete sticky antibiotic-resistant goo. At the same time, advances in imaging technology are generating increasingly large biological datasets. Researchers describe CellSAM as the first model that can be applied to myriad biological imaging tasks, making it easier to study the complex dynamics that shape disease and treatment response, including why a cancer immunotherapy may work for one person but not for another.
The CellSAM algorithm was trained on vast amounts of biological images that had been labeled by hand. The team plans to continue improving the model by training it on more types of biological data. The tool is currently available for researchers to use for free. Yue said the approach could make it possible to investigate biological questions at scales that were previously impractical, including tracking millions of cells across many conditions to study rare cell states and subtle changes in cell shape linked to treatment response.
The paper is titled “CellSAM: A Foundation Model for Cell Segmentation.” Uriah Israel and Markus Marks are the study’s first authors, alongside a broader group of Caltech researchers and contributors. Funding was provided by a range of foundations, research programs, and institutes, including the National Institutes of Health, the Heritage Medical Research Institute, and the Howard Hughes Medical Institute Freeman Hrabowski Scholars program.
