Journal Article
. 2019 Feb; 8:23.
doi: 10.1038/s41377-019-0129-y.

PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning

Yair Rivenson 1 Tairan Liu 1 Zhensong Wei 1 Yibo Zhang 1 Kevin de Haan 1 Aydogan Ozcan 1 
Affiliations
  • PMID: 30728961
  •     34 References
  •     44 citations

Abstract

Using a deep neural network, we demonstrate a digital staining technique, which we term PhaseStain, to transform the quantitative phase images (QPI) of label-free tissue sections into images that are equivalent to the brightfield microscopy images of the same samples that are histologically stained. Through pairs of image data (QPI and the corresponding brightfield images, acquired after staining), we train a generative adversarial network and demonstrate the effectiveness of this virtual-staining approach using sections of human skin, kidney, and liver tissue, matching the brightfield microscopy images of the same samples stained with Hematoxylin and Eosin, Jones' stain, and Masson's trichrome stain, respectively. This digital-staining framework may further strengthen various uses of label-free QPI techniques in pathology applications and biomedical research in general, by eliminating the need for histological staining, reducing sample preparation related costs and saving time. Our results provide a powerful example of some of the unique opportunities created by data-driven image transformations enabled by deep learning.

Reflective interferometric chamber for quantitative phase imaging of biological sample dynamics.
Natan T Shaked, Yizheng Zhu, +2 authors, Adam Wax.
J Biomed Opt, 2010 Jul 10; 15(3). PMID: 20614989    Free PMC article.
A computational approach to edge detection.
J Canny.
IEEE Trans Pattern Anal Mach Intell, 1986 Jun 01; 8(6). PMID: 21869365
Highly Cited.
Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy.
Daniel A Orringer, Balaji Pandian, +21 authors, Sandra Camelo-Piragua.
Nat Biomed Eng, 2017 Sep 29; 1. PMID: 28955599    Free PMC article.
Highly Cited.
Deep learning massively accelerates super-resolution localization microscopy.
Wei Ouyang, Andrey Aristov, +2 authors, Christophe Zimmer.
Nat Biotechnol, 2018 Apr 17; 36(5). PMID: 29658943
Highly Cited.
Dual-interference-channel quantitative-phase microscopy of live cell dynamics.
Natan T Shaked, Matthew T Rinehart, Adam Wax.
Opt Lett, 2009 Mar 14; 34(6). PMID: 19282926    Free PMC article.
Phase recovery and holographic image reconstruction using deep learning in neural networks.
Yair Rivenson, Yibo Zhang, +2 authors, Aydogan Ozcan.
Light Sci Appl, 2019 Mar 07; 7. PMID: 30839514    Free PMC article.
Highly Cited.
Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning.
Yair Rivenson, Hongda Wang, +10 authors, Aydogan Ozcan.
Nat Biomed Eng, 2019 May 31; 3(6). PMID: 31142829
Highly Cited.
Preparation of Formalin-fixed Paraffin-embedded Tissue Cores for both RNA and DNA Extraction.
Palak G Patel, Shamini Selvarajah, +8 authors, David M Berman.
J Vis Exp, 2016 Sep 02; (114). PMID: 27583817    Free PMC article.
Wide-field computational imaging of pathology slides using lens-free on-chip microscopy.
Alon Greenbaum, Yibo Zhang, +4 authors, Aydogan Ozcan.
Sci Transl Med, 2014 Dec 19; 6(267). PMID: 25520396
Quantitative differential phase contrast imaging in an LED array microscope.
Lei Tian, Laura Waller.
Opt Express, 2015 May 15; 23(9). PMID: 25969234
Guidance for laboratories performing molecular pathology for cancer patients.
Ian A Cree, Zandra Deans, +14 authors, Royal College of Pathologists.
J Clin Pathol, 2014 Jul 12; 67(11). PMID: 25012948    Free PMC article.
Review.
Scattering-phase theorem.
Zhuo Wang, Huafeng Ding, Gabriel Popescu.
Opt Lett, 2011 Apr 12; 36(7). PMID: 21479034
Simultaneous amplitude-contrast and quantitative phase-contrast microscopy by numerical reconstruction of Fresnel off-axis holograms.
E Cuche, P Marquet, C Depeursinge.
Appl Opt, 2008 Mar 08; 38(34). PMID: 18324243
Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy.
Alon Greenbaum, Aydogan Ozcan.
Opt Express, 2012 Feb 15; 20(3). PMID: 22330550    Free PMC article.
Quantitative phase microscopy spatial signatures of cancer cells.
Darina Roitshtain, Lauren Wolbromsky, +3 authors, Natan T Shaked.
Cytometry A, 2017 Apr 21; 91(5). PMID: 28426133
Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy.
Alon Greenbaum, Wei Luo, +6 authors, Aydogan Ozcan.
Nat Methods, 2012 Sep 01; 9(9). PMID: 22936170    Free PMC article.
Highly Cited.
Deep Learning in Label-free Cell Classification.
Claire Lifan Chen, Ata Mahjoubfar, +4 authors, Bahram Jalali.
Sci Rep, 2016 Mar 16; 6. PMID: 26975219    Free PMC article.
Highly Cited.
Sickle cell disease diagnosis based on spatio-temporal cell dynamics analysis using 3D printed shearing digital holographic microscopy.
Bahram Javidi, Adam Markman, +3 authors, Biree Andemariam.
Opt Express, 2018 May 29; 26(10). PMID: 29801384
Propagation phasor approach for holographic image reconstruction.
Wei Luo, Yibo Zhang, +2 authors, Aydogan Ozcan.
Sci Rep, 2016 Mar 12; 6. PMID: 26964671    Free PMC article.
Holographic deep learning for rapid optical screening of anthrax spores.
YoungJu Jo, Sangjin Park, +7 authors, YongKeun Park.
Sci Adv, 2017 Aug 12; 3(8). PMID: 28798957    Free PMC article.
Wide-field, high-resolution Fourier ptychographic microscopy.
Guoan Zheng, Roarke Horstmeyer, Changhuei Yang.
Nat Photonics, 2014 Sep 23; 7(9). PMID: 25243016    Free PMC article.
Highly Cited.
Hilbert phase microscopy for investigating fast dynamics in transparent systems.
Takahiro Ikeda, Gabriel Popescu, Ramachandra R Dasari, Michael S Feld.
Opt Lett, 2005 Jun 11; 30(10). PMID: 15945142
Edge sparsity criterion for robust holographic autofocusing.
Yibo Zhang, Hongda Wang, +2 authors, Aydogan Ozcan.
Opt Lett, 2017 Sep 29; 42(19). PMID: 28957139
Spatial light interference microscopy (SLIM).
Zhuo Wang, Larry Millet, +5 authors, Gabriel Popescu.
Opt Express, 2011 Jan 26; 19(2). PMID: 21263640    Free PMC article.
Highly Cited.
Automated Detection of P. falciparum Using Machine Learning Algorithms with Quantitative Phase Images of Unstained Cells.
Han Sang Park, Matthew T Rinehart, +2 authors, Adam Wax.
PLoS One, 2016 Sep 17; 11(9). PMID: 27636719    Free PMC article.
High-precision microscopic phase imaging without phase unwrapping for cancer cell identification.
Eriko Watanabe, Takashi Hoshiba, Bahram Javidi.
Opt Lett, 2013 Apr 19; 38(8). PMID: 23595471
A survey on deep learning in medical image analysis.
Geert Litjens, Thijs Kooi, +6 authors, Clara I Sánchez.
Med Image Anal, 2017 Aug 05; 42. PMID: 28778026
Highly Cited. Review.
Deep learning enables cross-modality super-resolution in fluorescence microscopy.
Hongda Wang, Yair Rivenson, +6 authors, Aydogan Ozcan.
Nat Methods, 2018 Dec 19; 16(1). PMID: 30559434    Free PMC article.
Highly Cited.
Tissue refractive index as marker of disease.
Zhuo Wang, Krishnarao Tangella, Andre Balla, Gabriel Popescu.
J Biomed Opt, 2011 Nov 25; 16(11). PMID: 22112122    Free PMC article.
Fast and robust multiframe super resolution.
Sina Farsiu, M Dirk Robinson, Michael Elad, Peyman Milanfar.
IEEE Trans Image Process, 2004 Oct 07; 13(10). PMID: 15462143
Wide-field tissue polarimetry allows efficient localized mass spectrometry imaging of biological tissues.
Alessandra Tata, Adam Gribble, +7 authors, Arash Zarrine-Afsar.
Chem Sci, 2016 Mar 01; 7(3). PMID: 30155015    Free PMC article.
Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution.
Waheb Bishara, Ting-Wei Su, Ahmet F Coskun, Aydogan Ozcan.
Opt Express, 2010 Jul 01; 18(11). PMID: 20588977    Free PMC article.
Assessment of breast pathologies using nonlinear microscopy.
Yuankai K Tao, Dejun Shen, +8 authors, James G Fujimoto.
Proc Natl Acad Sci U S A, 2014 Oct 15; 111(43). PMID: 25313045    Free PMC article.
Highly Cited.
Virtual Hematoxylin and Eosin Transillumination Microscopy Using Epi-Fluorescence Imaging.
Michael G Giacomelli, Lennart Husvogt, +4 authors, James G Fujimoto.
PLoS One, 2016 Aug 09; 11(8). PMID: 27500636    Free PMC article.
Quantitative Histopathology of Stained Tissues using Color Spatial Light Interference Microscopy (cSLIM).
Hassaan Majeed, Adib Keikhosravi, +6 authors, Gabriel Popescu.
Sci Rep, 2019 Oct 13; 9(1). PMID: 31604963    Free PMC article.
Deep learning in holography and coherent imaging.
Yair Rivenson, Yichen Wu, Aydogan Ozcan.
Light Sci Appl, 2019 Oct 28; 8. PMID: 31645929    Free PMC article.
Computational cytometer based on magnetically modulated coherent imaging and deep learning.
Yibo Zhang, Mengxing Ouyang, +17 authors, Aydogan Ozcan.
Light Sci Appl, 2019 Oct 28; 8. PMID: 31645935    Free PMC article.
Design of task-specific optical systems using broadband diffractive neural networks.
Yi Luo, Deniz Mengu, +4 authors, Aydogan Ozcan.
Light Sci Appl, 2019 Dec 10; 8. PMID: 31814969    Free PMC article.
Quantitative scoring of epithelial and mesenchymal qualities of cancer cells using machine learning and quantitative phase imaging.
Van Lam, Thanh Nguyen, +4 authors, Christopher Raub.
J Biomed Opt, 2020 Feb 20; 25(2). PMID: 32072775    Free PMC article.
Fast stimulated Raman and second harmonic generation imaging for intraoperative gastro-intestinal cancer detection.
Barbara Sarri, Rafaël Canonge, +8 authors, Hervé Rigneault.
Sci Rep, 2019 Jul 13; 9(1). PMID: 31296917    Free PMC article.
Holographic virtual staining of individual biological cells.
Yoav N Nygate, Mattan Levi, +7 authors, Natan T Shaked.
Proc Natl Acad Sci U S A, 2020 Apr 15; 117(17). PMID: 32284403    Free PMC article.
Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue.
Yijie Zhang, Kevin de Haan, +3 authors, Aydogan Ozcan.
Light Sci Appl, 2020 May 16; 9. PMID: 32411363    Free PMC article.
Deep learning-enabled point-of-care sensing using multiplexed paper-based sensors.
Zachary S Ballard, Hyou-Arm Joung, +5 authors, Aydogan Ozcan.
NPJ Digit Med, 2020 May 16; 3. PMID: 32411827    Free PMC article.
A Local Neighborhood Robust Fuzzy Clustering Image Segmentation Algorithm Based on an Adaptive Feature Selection Gaussian Mixture Model.
Hang Ren, Taotao Hu.
Sensors (Basel), 2020 Apr 26; 20(8). PMID: 32331452    Free PMC article.
Label-free hematology analysis using deep-ultraviolet microscopy.
Ashkan Ojaghi, Gabriel Carrazana, +4 authors, Francisco E Robles.
Proc Natl Acad Sci U S A, 2020 Jun 21; 117(26). PMID: 32561645    Free PMC article.
An Adaptive Feature Selection Algorithm for Fuzzy Clustering Image Segmentation Based on Embedded Neighbourhood Information Constraints.
Hang Ren, Taotao Hu.
Sensors (Basel), 2020 Jul 09; 20(13). PMID: 32635283    Free PMC article.
High spatially sensitive quantitative phase imaging assisted with deep neural network for classification of human spermatozoa under stressed condition.
Ankit Butola, Daria Popova, +8 authors, Balpreet Singh Ahluwalia.
Sci Rep, 2020 Aug 06; 10(1). PMID: 32753627    Free PMC article.
Revealing architectural order with quantitative label-free imaging and deep learning.
Syuan-Ming Guo, Li-Hao Yeh, +11 authors, Shalin B Mehta.
Elife, 2020 Jul 28; 9. PMID: 32716843    Free PMC article.
Reproductive outcomes predicted by phase imaging with computational specificity of spermatozoon ultrastructure.
Mikhail E Kandel, Marcello Rubessa, +9 authors, Gabriel Popescu.
Proc Natl Acad Sci U S A, 2020 Jul 22; 117(31). PMID: 32690677    Free PMC article.
FalseColor-Python: A rapid intensity-leveling and digital-staining package for fluorescence-based slide-free digital pathology.
Robert Serafin, Weisi Xie, Adam K Glaser, Jonathan T C Liu.
PLoS One, 2020 Oct 02; 15(10). PMID: 33001995    Free PMC article.
Deep-learning-based whole-brain imaging at single-neuron resolution.
Kefu Ning, Xiaoyu Zhang, +5 authors, Jing Yuan.
Biomed Opt Express, 2020 Oct 06; 11(7). PMID: 33014552    Free PMC article.
Virtual organelle self-coding for fluorescence imaging via adversarial learning.
Thanh Nguyen, Vy Bui, +4 authors, Georges Nehmetallah.
J Biomed Opt, 2020 Oct 01; 25(9). PMID: 32996300    Free PMC article.
A Regulatory Science Initiative to Harmonize and Standardize Digital Pathology and Machine Learning Processes to Speed up Clinical Innovation to Patients.
Hetal Desai Marble, Richard Huang, +14 authors, Jochen K Lennerz.
J Pathol Inform, 2020 Oct 13; 11. PMID: 33042601    Free PMC article.
SHIFT: speedy histological-to-immunofluorescent translation of a tumor signature enabled by deep learning.
Erik A Burlingame, Mary McDonnell, +7 authors, Young Hwan Chang.
Sci Rep, 2020 Oct 17; 10(1). PMID: 33060677    Free PMC article.
Integrative quantitative-phase and airy light-sheet imaging.
N R Subedi, P S Jung, +4 authors, A E Vasdekis.
Sci Rep, 2020 Nov 21; 10(1). PMID: 33214600    Free PMC article.
Deep neural network models for computational histopathology: A survey.
Chetan L Srinidhi, Ozan Ciga, Anne L Martel.
Med Image Anal, 2020 Oct 14; 67. PMID: 33049577    Free PMC article.
Review.
VISTA: VIsual Semantic Tissue Analysis for pancreatic disease quantification in murine cohorts.
Luke Ternes, Ge Huang, +5 authors, Young Hwan Chang.
Sci Rep, 2020 Dec 03; 10(1). PMID: 33262400    Free PMC article.
Inference in artificial intelligence with deep optics and photonics.
Gordon Wetzstein, Aydogan Ozcan, +6 authors, Demetri Psaltis.
Nature, 2020 Dec 04; 588(7836). PMID: 33268862
Review.
Phase imaging with computational specificity (PICS) for measuring dry mass changes in sub-cellular compartments.
Mikhail E Kandel, Yuchen R He, +7 authors, Gabriel Popescu.
Nat Commun, 2020 Dec 09; 11(1). PMID: 33288761    Free PMC article.
Terahertz pulse shaping using diffractive surfaces.
Muhammed Veli, Deniz Mengu, +5 authors, Aydogan Ozcan.
Nat Commun, 2021 Jan 06; 12(1). PMID: 33397912    Free PMC article.
Deep learning extended depth-of-field microscope for fast and slide-free histology.
Lingbo Jin, Yubo Tang, +9 authors, Ashok Veeraraghavan.
Proc Natl Acad Sci U S A, 2020 Dec 16; 117(52). PMID: 33318169    Free PMC article.
DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning.
Elias Nehme, Daniel Freedman, +7 authors, Yoav Shechtman.
Nat Methods, 2020 Jun 17; 17(7). PMID: 32541853    Free PMC article.
Practical fluorescence reconstruction microscopy for large samples and low-magnification imaging.
Julienne LaChance, Daniel J Cohen.
PLoS Comput Biol, 2020 Dec 29; 16(12). PMID: 33362219    Free PMC article.
A large-scale internal validation study of unsupervised virtual trichrome staining technologies on nonalcoholic steatohepatitis liver biopsies.
Joshua J Levy, Nasim Azizgolshani, +7 authors, Louis J Vaickus.
Mod Pathol, 2020 Dec 11; 34(4). PMID: 33299110    Free PMC article.
Recurrent neural network-based volumetric fluorescence microscopy.
Luzhe Huang, Hanlong Chen, +2 authors, Aydogan Ozcan.
Light Sci Appl, 2021 Mar 24; 10(1). PMID: 33753716    Free PMC article.
Deep learning classification of lipid droplets in quantitative phase images.
Luke Sheneman, Gregory Stephanopoulos, Andreas E Vasdekis.
PLoS One, 2021 Apr 06; 16(4). PMID: 33819277    Free PMC article.
Spectroscopic and deep learning-based approaches to identify and quantify cerebral microhemorrhages.
Christian Crouzet, Gwangjin Jeong, +11 authors, Bernard Choi.
Sci Rep, 2021 May 23; 11(1). PMID: 34021170    Free PMC article.
Deep Learning in Biomedical Optics.
Lei Tian, Brady Hunt, +5 authors, Nicholas J Durr.
Lasers Surg Med, 2021 May 21; 53(6). PMID: 34015146    Free PMC article.
Review.
Synthetic polarization-sensitive optical coherence tomography by deep learning.
Yi Sun, Jianfeng Wang, Jindou Shi, Stephen A Boppart.
NPJ Digit Med, 2021 Jul 03; 4(1). PMID: 34211104    Free PMC article.
Label-free screening of brain tissue myelin content using phase imaging with computational specificity (PICS).
Michael Fanous, Chuqiao Shi, +6 authors, Gabriel Popescu.
APL Photonics, 2021 Jul 23; 6(7). PMID: 34291159    Free PMC article.
Deep Learning-Based Holographic Polarization Microscopy.
Tairan Liu, Kevin de Haan, +8 authors, Aydogan Ozcan.
ACS Photonics, 2021 Aug 10; 7(11). PMID: 34368395    Free PMC article.
Label-free colorectal cancer screening using deep learning and spatial light interference microscopy (SLIM).
Jingfang Kelly Zhang, Yuchen He, Nahil Sobh, Gabriel Popescu.
APL Photonics, 2020 Apr 01; 5(4). PMID: 34368439    Free PMC article.
Deep learning-based transformation of H&E stained tissues into special stains.
Kevin de Haan, Yijie Zhang, +11 authors, Aydogan Ozcan.
Nat Commun, 2021 Aug 14; 12(1). PMID: 34385460    Free PMC article.
Machine Learning Enhances the Performance of Bioreceptor-Free Biosensors.
Kenneth E Schackart, Jeong-Yeol Yoon.
Sensors (Basel), 2021 Aug 29; 21(16). PMID: 34450960    Free PMC article.
Review.
Learning to see colours: Biologically relevant virtual staining for adipocyte cell images.
Håkan Wieslander, Ankit Gupta, +2 authors, Philip John Harrison.
PLoS One, 2021 Oct 16; 16(10). PMID: 34653209    Free PMC article.
Machine Learning Assisted Classification of Cell Lines and Cell States on Quantitative Phase Images.
Andrey V Belashov, Anna A Zhikhoreva, +4 authors, Oleg S Vasyutinskii.
Cells, 2021 Oct 24; 10(10). PMID: 34685568    Free PMC article.
Deep-learning-assisted microscopy with ultraviolet surface excitation for rapid slide-free histological imaging.
Zhenghui Chen, Wentao Yu, Ivy H M Wong, Terence T W Wong.
Biomed Opt Express, 2021 Oct 26; 12(9). PMID: 34692225    Free PMC article.
Deep learning enables ultraviolet photoacoustic microscopy based histological imaging with near real-time virtual staining.
Lei Kang, Xiufeng Li, Yan Zhang, Terence T W Wong.
Photoacoustics, 2021 Oct 28; 25. PMID: 34703763    Free PMC article.