Deep learning algorithm and its application in optics

Zhou Hongqiang; Huang Lingling; Wang Yongtian

doi:10.3788/IRLA201948.1226004

Volume 48 Issue 12

Dec. 2019

Turn off MathJax

Article Contents

Article Navigation > Infrared and Laser Engineering > 2019 > 48(12): 1226004-1226004(20)

Zhou Hongqiang, Huang Lingling, Wang Yongtian. Deep learning algorithm and its application in optics[J]. Infrared and Laser Engineering, 2019, 48(12): 1226004-1226004(20). doi: 10.3788/IRLA201948.1226004

Citation:

Zhou Hongqiang, Huang Lingling, Wang Yongtian. Deep learning algorithm and its application in optics[J]. Infrared and Laser Engineering, 2019, 48(12): 1226004-1226004(20). doi: 10.3788/IRLA201948.1226004

Deep learning algorithm and its application in optics

doi: 10.3788/IRLA201948.1226004

1.
Key Laboratory of Photoelectronic Imaging Technology and System,Ministry of Education,School of Optics and Photonics,Beijing Institute of Technology,Beijing 100081,China

Received Date: 2019-10-11
Rev Recd Date: 2019-11-21
Publish Date: 2019-12-25

Abstract

As an important branch of machine learning, deep learning has reached another climax of machine learning since its inception. Deep learning has excellent performance in many fields such as image recognition and classification, semantic segmentation, and intelligent driving and so on. At the same time, deep learning algorithms are widely used in the field of optics such as computational hologram generation and imaging, non-parameter reconstruction of digital holography, and spectral resonance curves prediction due to their abstract feature recognition and extraction characteristics, strong model building and generalization capabilities. This article detailed the basic principles of deep learning and its typical application research in image classification, super-resolution imaging, computer generated hologram and digital holography, prediction of surface plasmonics resonance curves, and structural design of metasurfaces. And future development of deep learning in the physical optical field was worth exploring.
- deep learning,
- neural network,
- optics,
- imaging technology

References

[1]	Simon H. Machines, Neural Network and Learning[M]. 3rd ed. New Jersey:Pearson Education, Inc, 2009.
[2]	Simon H. A Comprehensive Foundation[M]. 2nd ed. New Jersey:Prentice Hall International, Inc, 1999.
[3]	Gardner M W, Dorling S R. Artificial neural networks(the multilayer perceptron)-a review of application in the atmospheric sciences[J]. Atmospheric Environment. 1998, 32(14):2627-2636.
[4]	Hornic K. Multilayer feedforward networks are universal approximators[J]. Neural Network, 1989, 2(5):359-366.
[5]	Arel I, Rose D C, Karnowski T P. Deep machine learning-a new frontier in artificial intelligence research[J]. IEEE Computational Intelligence Magazine, 2010, 5(4):13-18.
[6]	Qiao Fengjuan, Guo Hongli, Li Wei, et al. Research on deep learning classification based on SVM:a review[J]. Journal of Qilu University of Technology, 2018, 32(5):39-44. (in Chinese)
[7]	Cortes C, Vapnik V. Support-vector networks[J]. Machine Learning, 1995, 20(3):273-297.
[8]	Lecun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
[9]	Lecun Y, Boser B E, Denker J S, et al. Handwritten digit recognition with a back-propagation network[C]//Proc Advances in Neural Information Processing Systems, 1990:396-404.
[10]	Lecun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[C]//Proceedings of IEEE. 1998, 86(11):2278-2324.
[11]	Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[12]	Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[J]. arXiv Preprint, 2014, arXiv:1406.1078.
[13]	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems, 2014:2672-2680.
[14]	Yoav F, David H. Unsupervised learning of distributions of binary vectors using 2 layer networks[C]//Advances in Neural Information Processing Systems, 1991, 4:912-918.
[15]	Hinton G E, Osindero S, Teh Y. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1554.
[16]	Wang Bingrui, Lan Huiying, Chen Yunji. Programming frameworks for deep learning algorithms[J]. Big Data Research, 2018, 4(4):56-63. (in Chinese)
[17]	Jake B. Notes on convolutional neural networks[D]. Cambridge:MIT Press, 2006.
[18]	Lecun Y, Bengio Y. Convolutional Networks for Images, Speech, and Time Series[M]. Cambridge:MIT Press, 1995.
[19]	Waibel A, Hanazawa T, Hinton G, et al. Phoneme recognition using time-delay neural networks[J]. IEEE Transactions on Acoustics Speech and Signal Processing, 1989, 37(3):328-339.
[20]	Bottou L, Fogelman-Soubi F, Blanchet P, et al. Experiments with time delay networks and dynamic time warping for speaker independent isolated digit recognition[C]//Euro Speech:1989, 89:537-540.
[21]	Simard P Y, Steinkraus D, Platt J C. Best practices for convolutional neural networks applied to visual document analysis[C]//Seventh International Conference on Document Analysis and Recognition, 2003:958-962.
[22]	Vaillant R, Monrocq C, Lecun Y. Original approach for the localization of objects in images[J]. IEEE Proceedings-Vision Image and Signal Proceeding, 1994, 141(4):245-250.
[23]	Nowlan S, Platt J. A convolutional neural network hand tracker[C]//Advances Neural Information Processing Systems, 1995:901-908.
[24]	Lawrence S, Giles C L, Tsoi A C, et al. Face recognition:A convolutional neural-network approach[J]. IEEE Transactions on Neural Networks, 1997, 8(1):98-113.
[25]	Goodfellow Ian, Bengio Yoshua, Courville Aaron. Deep Learing[M]. Translated by Zhao S, Li Y, Fu T, et al. Beijing:Post and Telecom Press, 2017:274-298. (in Chinese)
[26]	Shin H, Roth H R, Gao M, et al. Deep convolutional neural networks for computer-aided detection:CNN architectures, dataset characteristics and transfer learning[J]. IEEE Transactions on Medical Imaging, 2016, 35(5SI):1285-1298.
[27]	He K, Zhang X, Ren S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1904-1916.
[28]	Litjens G, Kooi T, Bejnordi B E, et al. A survey on deep learning in medical image analysis[J]. Medical Image Analysis, 2017, 42:60-88.
[29]	Esteva A, Kuprel B, Novoa R A, et al. Dermatologist-level classification of skin cancer with deep neural networks[J]. Nature, 2017, 542(7639):115.
[30]	Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651.
[31]	Murtagh P, Tsoi A C. Implementation issues of sigmoid function and its derivative for VLSI digital neural networks[J]. IEEE Proceedings-E Computers and Digital Techniques, 1992, 139(3):207-214.
[32]	Xu G, Wu H, Shi Y. Structural design of convolutional neural networks for steganalysis[J]. IEEE Signal Processing Letters, 2016, 23(5):708-712.
[33]	Han J, Cho G, Kwak K. A design of convolutional neural network using ReLU-based ELM classifier and its application[C]//Proceedings of the 9th International Conference on Machine Learning and Computing, 2017.
[34]	Mirza M O S. Conditional generative adversarial nets[J]. arXiv preprint, 2014, arXiv:1411.1784.
[35]	Radford A M L C S. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv preprint, 2015, arXiv:1511. 06434.
[36]	Reed S, Akata Z, Yan X, et al. Generative adversarial text to image synthesis[J]. arXiv preprint, 2016, arXiv:1605.05396.
[37]	Wu J, Zhang C, Xue T, et al. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modelin[J]. arXiv preprint, 2016, arXiv:1610.07584.
[38]	Smith E, Meger D. Improved adversarial systems for 3D object generation and reconstruction[J]. arXiv preprint. 2017, arXiv:1707.09557.
[39]	Denton E, Gross S, Fergus R. Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks[J]. arXiv preprint, 2016, arXiv:1611.06430.
[40]	Yi Z, Zhang H, Tan P, et al. DualGAN:Unsupervised dual learning for image-to-image translation[J]. arXiv preprint, 2017, arXiv:1704. 02510.
[41]	Ledig C, Theis L, Huszar F, et al. Photo-realistic single image super-resolution using a generative adversarial network[J]. arXiv preprint, 2017, arXiv:1609. 04802v5.
[42]	Vondrick C, Pirsiavash H, Torralba A. Generating videos with scene dynamics[J]. arXiv preprint, 2016, arXiv:1609.02612v3.
[43]	Perarnau G, van de Weijer J, Raducanu B, et al. Invertible conditional GANs for image editing[J]. arXiv preprint, 2016, arXiv:1611.06355.
[44]	Antonia Creswell A A B. Inverting the generator of a generative adversarial network[J]. arXiv preprint, 2016, arXiv:1611.05644.
[45]	Zhou S, Xiao T, Yang Y, et al. GeneGAN:learning object transfiguration and attribute subspace from unpaired data[J]. arXiv preprint, arXiv:1705.04932.
[46]	Kim T, Cha M, Kim H, et al. Learning to discover cross-domain relations with generative adversarial networks[J]. arXiv preprint, 2017, arXiv:1703.05192.
[47]	Wang C, Wang C, Xu C, et al. Tag disentangled generative adversarial network for object image re-rendering[C]//The Twenty-Sixth International Joint Conference on Artificial Intelligence, 2017:2901-2907.
[48]	Antipov G, Dugelay M B J. Face aging with conditional generative adversarial networks[J]. arXiv preprint, 2017, arXiv:1702.01983.
[49]	Arjovsky M, Chintala S, Bottou L. Wasserstein GAN[J]. arXiv preprint, 2017, arXiv:1701.07875.
[50]	Ando T, Horisaki R, Tanida J. Speckle-learning-based object recognition through scattering media[J]. Optics Express, 2015, 23(26):33902-33910.
[51]	Horisaki R, Takagi R, Tanida J. Learning based imaging through scattering media[J]. Optics Express, 2016, 24(13):13738-13743.
[52]	Takagi R, Horisaki R, Tanida J. Object recognition through a multi-mode fiber[J]. Optical Review, 2017, 24(2):117-120.
[53]	Horisaki R, Takagi R, Tanida J. Learning based focusing through scattering media[J]. Applied Optics, 2017, 56(15):4358-4362.
[54]	Horisaki R, Takagi R, Tanida J. Learning based single-shot superresolution in diffractive imaging[J]. Applied Optics, 2017, 56(32):8896-8901.
[55]	Sinha A, Lee J, Li S, et al. Lensless computational imaging through deep learning[J]. Optica, 2017, 4(9):1117-1125.
[56]	Li R, Zeng T, Peng H, et al. Deep learning segmentation of optical microscopy images improves 3-D neuron reconstruction[J]. IEEE Transactions on Medical Imaging, 2017, 36(7):1533-1541.
[57]	Lyu M, Wang W, Wang H, et al. Deep-learning-based ghost imaging[J]. Scientific Reports, 2017, 7:17865.
[58]	Jo Y, Park S, Jung J, et al. Holographic deep learning for rapid optical screening of anthrax spores[J]. Science Advances, 2017, 3(8):e1700606.
[59]	Lin X, Rivenson Y, Yardimei N T, et al. All-optical machine learning using diffractive deep neural networks[J]. Science, 2018, 361(6406):1004.
[60]	Xiao H, Rasul K, Vollgraf R, et al. fashion-minist[EB/OL]. San Francisco:GitHub database site, 2017[2019-12-17]. https://github.com/zalandoresearch/fashion-mnist#fashion-mnist.htm.
[61]	Li Jingxi, Mengu Deniz, Luo Yi, et al. Class-specific differential detection in diffractive optical neural networks improves inference accuracy[J]. Advanced Photonics, 2019, 1(4):046001.
[62]	Huang B, Bates M, Zhuang X. Super resolution fluorescence microscopy[J]. Annual Review of Biochemistry, 2009, 78:993-1016.
[63]	Willig K I, Rizzoli S O, Westphal V, et al. STED microscopy reveals that synaptotagmin remains clustered after synaptic vesicle exocytosis[J]. Nature, 2006, 440(7086):935-939.
[64]	Rittweger E, Han K Y, Irvine S E, et al. STED microscopy reveals crystal colour centres with nanometric resolution[J]. Nature Photonics, 2009, 3(3):144-147.
[65]	Rust M J, Bates M, Zhuang X. Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)[J]. Nature Methods, 2006, 3(10):793-795.
[66]	Dong C, Loy C C, He K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307.
[67]	Rivenson Y, Gorocs Z, Gunaydin H, et al. Deep learning microscopy[J]. Optica, 2017, 4(11):1437-1443.
[68]	Wu Y, Rivenson Y, Zhang Y, et al. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery[J]. Optica, 2018, 5(6):704-710.
[69]	Rivenson Y, Zhang Y, Gnaydin H, et al. Phase recovery and holographic image reconstruction using deep learning in neural networks[J]. Light-Science Applications, 2018, 7:e17141.
[70]	Huang L, Muhlenbernd H, Li X, et al. Broadband hybrid holographic multiplexing with geometric metasurfaces[J]. Advanced Materials, 2015, 27(41):6444.
[71]	Huang L, Chen X, Muehlenbernd H, et al. Three-dimensional optical holography using a plasmonic metasurface[J]. Nature Communications, 2013, 4:2808.
[72]	Zijlstra P, Chon J W M, Gu M. Five dimensional optical recording mediated by surface plasmons in gold nanorods[J]. Nature, 2009, 459(7245):410-413.
[73]	Zhao R, Sain B, Wei Q, et al. Multichannel vectorial holographic display and encryption[J]. Light-Science Applications, 2018, 7:95.
[74]	Jiang Q, Jin G, Cao L. When metasurface meets hologram:principle and advances[J]. Advanced Optics and Photonics, 2019, 11(3):000518.
[75]	Goodman J W. Introduction to Fourier Optics[M]. 2nd ed. Singapore:McGraw-Hill, 1996:66-67.
[76]	Schnars U, Falldorf C, Watson J, et al. Digital Holography and Wavefront Sensing:Principles, Techniques and Applications[M]. Berlin:Springer, 2015.
[77]	Ren Z, Xu Z, Lam E Y. Learning-based nonparametric autofocusing or digital holography[J]. Optica, 2018, 5(4):337-344.
[78]	Slinger C, Cameron C, Stanley M. Computer-generated holography as a generic display technology[J]. Computer, 2005, 38(8):46-53.
[79]	Horisaki R, Takagi R, Tanida J. Deep-learning-generated holography[J]. Applied Optics, 2018, 57(14):3859-3863.
[80]	Pfeiffer C, Grbic A. Metamaterial huygens' surfaces:tailoring wave fronts with reflectionless sheets[J]. Physical Review Letters, 2013, 110(19):197401.
[81]	Arbabi A, Horie Y, Bagheri M, et al. Dielectric metasurfaces for complete control of phase and polarization with subwavelength spatial resolution and high transmission[J]. Nature Nanotechnology, 2015, 10(11):190-937.
[82]	Butet J, Brevet P, Martin O J F. Optical second harmonic generation in plasmonic nanostructures:from fundamental principles to advanced applications[J]. ACS Nano, 2015, 9(11):10545-10562.
[83]	Li G, Chen S, Pholchai N, et al. Continuous control of the nonlinearity phase for harmonic generations[J]. Nature Materials, 2015, 14(6):607-612.
[84]	Malkiel I, Mrejen M, Nagler A, et al. Plasmonic nanostructure design and characterization via deep learning[J]. Light-Science Applications, 2018, 7:60.
[85]	Liu D, Tan Y, Khoram E, et al. Training deep neural networks for the inverse design of nanophotonic structures[J]. ACS Photonics, 2018, 5(4):1365-1369.
[86]	Ma W, Cheng F, Liu Y. Deep learning enabled on-demand design of chiral metamaterials[J]. ACS Nano, 2018, 12(6):6326-6334.
[87]	Peurifoy J, Shen Y, Jing L, et al. Nanophotonic particle simulation and inverse design using artificial neural networks[J]. Science Advances, 2018, 4(6):aar4206.
[88]	Liu Z, Zhu D, Rodrigues S P, et al. Generative model for the inverse design of metasurfaces[J]. Nano Letters, 2018, 18(10):6570-6576.
[89]	Tahersima M H, Kojima K, Koike-Akino T, et al. Deep neural network inverse design of integrated photonic power splitters[J]. Scientific Reports, 2019, 9:1368.
[90]	Yao K, Unni R, Zheng Y. Intelligent nanophotonics:merging photonics and artificial intelligence at the nanoscale[J]. Nanophotonics, 2019, 8(3):339-366.
[91]	Su L, Piggott A Y, Sapra N V, et al. Inverse design and demonstration of a compact on-chip narrowband three-channel wavelength demultiplexer[J]. ACS Photonics, 2018, 5(2):301-305.
[92]	Piggott A Y, Lu J, Babinec T M, et al. Inverse design and implementation of a wavelength demultiplexing grating coupler[J]. Scientific Reports, 2014, 4:7210.
[93]	Molesky S, Lin Z, Piggott A Y, et al. Inverse design in nanophotonics[J]. Nature Photonics, 2018, 12(11):659-670.
[94]	Lan L, Sun F, Liu Y, et al. Experimentally demonstrated a unidirectional electromagnetic cloak designed by topology optimization[J]. Applied Physics Letters, 2013, 103(12):121113.
[95]	Fujii G, Watanabe H, Yamada T, et al. Level set based topology optimization for optical cloaks[J]. Applied Physics Letters, 2013, 102(25):251106.
[96]	Sell D, Yang J, Doshay S, et al. Periodic dielectric metasurfaces with high-efficiency, multiwavelength functionalities[J]. Advanced Optical Materials, 2017, 5(23):00645.
[97]	Deng Y, Korvink J G. Topology optimization for three-dimensional electromagnetic waves using an edge element-based finite-element method[C]//Proceedings of the Royal Society A-mathematical physical and engineering sciences, 2016, 472(2189):20150835.
[98]	Andkjaer J, Sigmund O. Topology optimized low-contrast all-dielectric optical cloak[J]. Applied Physics Letters, 2011, 98(2):021112.
[99]	Lin Z, Groever B, Capasso F, et al. Topology-optimized multilayered metaoptics[J]. Physical Review Applied, 2018, 9(4):044030.
[100]	Yin Baocai, Wang Wentong, Wang Litong. Review of deep learning[J]. Journal of Beijing University of Technology, 2015, 41(1):48-59. (in Chinese)
[101]	Barbastathis George, Ozcan Aydogan, Situ Guohai, On the use of deep learning for computational imaging[J]. Optica, 2019, 6(8):921-943.

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Get Citation

PDF

XML

Article Metrics

Article views(2619) PDF downloads(357) Cited by()

Proportional views

Deep learning algorithm and its application in optics

1. Key Laboratory of Photoelectronic Imaging Technology and System,Ministry of Education,School of Optics and Photonics,Beijing Institute of Technology,Beijing 100081,China

Received Date: 2019-10-11

Rev Recd Date: 2019-11-21

Publish Date: 2019-12-25

Key words:

Abstract: As an important branch of machine learning, deep learning has reached another climax of machine learning since its inception. Deep learning has excellent performance in many fields such as image recognition and classification, semantic segmentation, and intelligent driving and so on. At the same time, deep learning algorithms are widely used in the field of optics such as computational hologram generation and imaging, non-parameter reconstruction of digital holography, and spectral resonance curves prediction due to their abstract feature recognition and extraction characteristics, strong model building and generalization capabilities. This article detailed the basic principles of deep learning and its typical application research in image classification, super-resolution imaging, computer generated hologram and digital holography, prediction of surface plasmonics resonance curves, and structural design of metasurfaces. And future development of deep learning in the physical optical field was worth exploring.

HTML

Reference (101)

[1]	Simon H. Machines, Neural Network and Learning[M]. 3rd ed. New Jersey:Pearson Education, Inc, 2009.
[2]	Simon H. A Comprehensive Foundation[M]. 2nd ed. New Jersey:Prentice Hall International, Inc, 1999.
[3]	Gardner M W, Dorling S R. Artificial neural networks(the multilayer perceptron)-a review of application in the atmospheric sciences[J]. Atmospheric Environment. 1998, 32(14):2627-2636.
[4]	Hornic K. Multilayer feedforward networks are universal approximators[J]. Neural Network, 1989, 2(5):359-366.
[5]	Arel I, Rose D C, Karnowski T P. Deep machine learning-a new frontier in artificial intelligence research[J]. IEEE Computational Intelligence Magazine, 2010, 5(4):13-18.
[6]	Qiao Fengjuan, Guo Hongli, Li Wei, et al. Research on deep learning classification based on SVM:a review[J]. Journal of Qilu University of Technology, 2018, 32(5):39-44. (in Chinese)
[7]	Cortes C, Vapnik V. Support-vector networks[J]. Machine Learning, 1995, 20(3):273-297.
[8]	Lecun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
[9]	Lecun Y, Boser B E, Denker J S, et al. Handwritten digit recognition with a back-propagation network[C]//Proc Advances in Neural Information Processing Systems, 1990:396-404.
[10]	Lecun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[C]//Proceedings of IEEE. 1998, 86(11):2278-2324.
[11]	Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[12]	Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[J]. arXiv Preprint, 2014, arXiv:1406.1078.
[13]	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems, 2014:2672-2680.
[14]	Yoav F, David H. Unsupervised learning of distributions of binary vectors using 2 layer networks[C]//Advances in Neural Information Processing Systems, 1991, 4:912-918.
[15]	Hinton G E, Osindero S, Teh Y. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1554.
[16]	Wang Bingrui, Lan Huiying, Chen Yunji. Programming frameworks for deep learning algorithms[J]. Big Data Research, 2018, 4(4):56-63. (in Chinese)
[17]	Jake B. Notes on convolutional neural networks[D]. Cambridge:MIT Press, 2006.
[18]	Lecun Y, Bengio Y. Convolutional Networks for Images, Speech, and Time Series[M]. Cambridge:MIT Press, 1995.
[19]	Waibel A, Hanazawa T, Hinton G, et al. Phoneme recognition using time-delay neural networks[J]. IEEE Transactions on Acoustics Speech and Signal Processing, 1989, 37(3):328-339.
[20]	Bottou L, Fogelman-Soubi F, Blanchet P, et al. Experiments with time delay networks and dynamic time warping for speaker independent isolated digit recognition[C]//Euro Speech:1989, 89:537-540.
[21]	Simard P Y, Steinkraus D, Platt J C. Best practices for convolutional neural networks applied to visual document analysis[C]//Seventh International Conference on Document Analysis and Recognition, 2003:958-962.
[22]	Vaillant R, Monrocq C, Lecun Y. Original approach for the localization of objects in images[J]. IEEE Proceedings-Vision Image and Signal Proceeding, 1994, 141(4):245-250.
[23]	Nowlan S, Platt J. A convolutional neural network hand tracker[C]//Advances Neural Information Processing Systems, 1995:901-908.
[24]	Lawrence S, Giles C L, Tsoi A C, et al. Face recognition:A convolutional neural-network approach[J]. IEEE Transactions on Neural Networks, 1997, 8(1):98-113.
[25]	Goodfellow Ian, Bengio Yoshua, Courville Aaron. Deep Learing[M]. Translated by Zhao S, Li Y, Fu T, et al. Beijing:Post and Telecom Press, 2017:274-298. (in Chinese)
[26]	Shin H, Roth H R, Gao M, et al. Deep convolutional neural networks for computer-aided detection:CNN architectures, dataset characteristics and transfer learning[J]. IEEE Transactions on Medical Imaging, 2016, 35(5SI):1285-1298.
[27]	He K, Zhang X, Ren S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1904-1916.
[28]	Litjens G, Kooi T, Bejnordi B E, et al. A survey on deep learning in medical image analysis[J]. Medical Image Analysis, 2017, 42:60-88.
[29]	Esteva A, Kuprel B, Novoa R A, et al. Dermatologist-level classification of skin cancer with deep neural networks[J]. Nature, 2017, 542(7639):115.
[30]	Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651.
[31]	Murtagh P, Tsoi A C. Implementation issues of sigmoid function and its derivative for VLSI digital neural networks[J]. IEEE Proceedings-E Computers and Digital Techniques, 1992, 139(3):207-214.
[32]	Xu G, Wu H, Shi Y. Structural design of convolutional neural networks for steganalysis[J]. IEEE Signal Processing Letters, 2016, 23(5):708-712.
[33]	Han J, Cho G, Kwak K. A design of convolutional neural network using ReLU-based ELM classifier and its application[C]//Proceedings of the 9th International Conference on Machine Learning and Computing, 2017.
[34]	Mirza M O S. Conditional generative adversarial nets[J]. arXiv preprint, 2014, arXiv:1411.1784.
[35]	Radford A M L C S. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv preprint, 2015, arXiv:1511. 06434.
[36]	Reed S, Akata Z, Yan X, et al. Generative adversarial text to image synthesis[J]. arXiv preprint, 2016, arXiv:1605.05396.
[37]	Wu J, Zhang C, Xue T, et al. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modelin[J]. arXiv preprint, 2016, arXiv:1610.07584.
[38]	Smith E, Meger D. Improved adversarial systems for 3D object generation and reconstruction[J]. arXiv preprint. 2017, arXiv:1707.09557.
[39]	Denton E, Gross S, Fergus R. Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks[J]. arXiv preprint, 2016, arXiv:1611.06430.
[40]	Yi Z, Zhang H, Tan P, et al. DualGAN:Unsupervised dual learning for image-to-image translation[J]. arXiv preprint, 2017, arXiv:1704. 02510.
[41]	Ledig C, Theis L, Huszar F, et al. Photo-realistic single image super-resolution using a generative adversarial network[J]. arXiv preprint, 2017, arXiv:1609. 04802v5.
[42]	Vondrick C, Pirsiavash H, Torralba A. Generating videos with scene dynamics[J]. arXiv preprint, 2016, arXiv:1609.02612v3.
[43]	Perarnau G, van de Weijer J, Raducanu B, et al. Invertible conditional GANs for image editing[J]. arXiv preprint, 2016, arXiv:1611.06355.
[44]	Antonia Creswell A A B. Inverting the generator of a generative adversarial network[J]. arXiv preprint, 2016, arXiv:1611.05644.
[45]	Zhou S, Xiao T, Yang Y, et al. GeneGAN:learning object transfiguration and attribute subspace from unpaired data[J]. arXiv preprint, arXiv:1705.04932.
[46]	Kim T, Cha M, Kim H, et al. Learning to discover cross-domain relations with generative adversarial networks[J]. arXiv preprint, 2017, arXiv:1703.05192.
[47]	Wang C, Wang C, Xu C, et al. Tag disentangled generative adversarial network for object image re-rendering[C]//The Twenty-Sixth International Joint Conference on Artificial Intelligence, 2017:2901-2907.
[48]	Antipov G, Dugelay M B J. Face aging with conditional generative adversarial networks[J]. arXiv preprint, 2017, arXiv:1702.01983.
[49]	Arjovsky M, Chintala S, Bottou L. Wasserstein GAN[J]. arXiv preprint, 2017, arXiv:1701.07875.
[50]	Ando T, Horisaki R, Tanida J. Speckle-learning-based object recognition through scattering media[J]. Optics Express, 2015, 23(26):33902-33910.
[51]	Horisaki R, Takagi R, Tanida J. Learning based imaging through scattering media[J]. Optics Express, 2016, 24(13):13738-13743.
[52]	Takagi R, Horisaki R, Tanida J. Object recognition through a multi-mode fiber[J]. Optical Review, 2017, 24(2):117-120.
[53]	Horisaki R, Takagi R, Tanida J. Learning based focusing through scattering media[J]. Applied Optics, 2017, 56(15):4358-4362.
[54]	Horisaki R, Takagi R, Tanida J. Learning based single-shot superresolution in diffractive imaging[J]. Applied Optics, 2017, 56(32):8896-8901.
[55]	Sinha A, Lee J, Li S, et al. Lensless computational imaging through deep learning[J]. Optica, 2017, 4(9):1117-1125.
[56]	Li R, Zeng T, Peng H, et al. Deep learning segmentation of optical microscopy images improves 3-D neuron reconstruction[J]. IEEE Transactions on Medical Imaging, 2017, 36(7):1533-1541.
[57]	Lyu M, Wang W, Wang H, et al. Deep-learning-based ghost imaging[J]. Scientific Reports, 2017, 7:17865.
[58]	Jo Y, Park S, Jung J, et al. Holographic deep learning for rapid optical screening of anthrax spores[J]. Science Advances, 2017, 3(8):e1700606.
[59]	Lin X, Rivenson Y, Yardimei N T, et al. All-optical machine learning using diffractive deep neural networks[J]. Science, 2018, 361(6406):1004.
[60]	Xiao H, Rasul K, Vollgraf R, et al. fashion-minist[EB/OL]. San Francisco:GitHub database site, 2017[2019-12-17]. https://github.com/zalandoresearch/fashion-mnist#fashion-mnist.htm.
[61]	Li Jingxi, Mengu Deniz, Luo Yi, et al. Class-specific differential detection in diffractive optical neural networks improves inference accuracy[J]. Advanced Photonics, 2019, 1(4):046001.
[62]	Huang B, Bates M, Zhuang X. Super resolution fluorescence microscopy[J]. Annual Review of Biochemistry, 2009, 78:993-1016.
[63]	Willig K I, Rizzoli S O, Westphal V, et al. STED microscopy reveals that synaptotagmin remains clustered after synaptic vesicle exocytosis[J]. Nature, 2006, 440(7086):935-939.
[64]	Rittweger E, Han K Y, Irvine S E, et al. STED microscopy reveals crystal colour centres with nanometric resolution[J]. Nature Photonics, 2009, 3(3):144-147.
[65]	Rust M J, Bates M, Zhuang X. Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)[J]. Nature Methods, 2006, 3(10):793-795.
[66]	Dong C, Loy C C, He K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307.
[67]	Rivenson Y, Gorocs Z, Gunaydin H, et al. Deep learning microscopy[J]. Optica, 2017, 4(11):1437-1443.
[68]	Wu Y, Rivenson Y, Zhang Y, et al. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery[J]. Optica, 2018, 5(6):704-710.
[69]	Rivenson Y, Zhang Y, Gnaydin H, et al. Phase recovery and holographic image reconstruction using deep learning in neural networks[J]. Light-Science Applications, 2018, 7:e17141.
[70]	Huang L, Muhlenbernd H, Li X, et al. Broadband hybrid holographic multiplexing with geometric metasurfaces[J]. Advanced Materials, 2015, 27(41):6444.
[71]	Huang L, Chen X, Muehlenbernd H, et al. Three-dimensional optical holography using a plasmonic metasurface[J]. Nature Communications, 2013, 4:2808.
[72]	Zijlstra P, Chon J W M, Gu M. Five dimensional optical recording mediated by surface plasmons in gold nanorods[J]. Nature, 2009, 459(7245):410-413.
[73]	Zhao R, Sain B, Wei Q, et al. Multichannel vectorial holographic display and encryption[J]. Light-Science Applications, 2018, 7:95.
[74]	Jiang Q, Jin G, Cao L. When metasurface meets hologram:principle and advances[J]. Advanced Optics and Photonics, 2019, 11(3):000518.
[75]	Goodman J W. Introduction to Fourier Optics[M]. 2nd ed. Singapore:McGraw-Hill, 1996:66-67.
[76]	Schnars U, Falldorf C, Watson J, et al. Digital Holography and Wavefront Sensing:Principles, Techniques and Applications[M]. Berlin:Springer, 2015.
[77]	Ren Z, Xu Z, Lam E Y. Learning-based nonparametric autofocusing or digital holography[J]. Optica, 2018, 5(4):337-344.
[78]	Slinger C, Cameron C, Stanley M. Computer-generated holography as a generic display technology[J]. Computer, 2005, 38(8):46-53.
[79]	Horisaki R, Takagi R, Tanida J. Deep-learning-generated holography[J]. Applied Optics, 2018, 57(14):3859-3863.
[80]	Pfeiffer C, Grbic A. Metamaterial huygens' surfaces:tailoring wave fronts with reflectionless sheets[J]. Physical Review Letters, 2013, 110(19):197401.
[81]	Arbabi A, Horie Y, Bagheri M, et al. Dielectric metasurfaces for complete control of phase and polarization with subwavelength spatial resolution and high transmission[J]. Nature Nanotechnology, 2015, 10(11):190-937.
[82]	Butet J, Brevet P, Martin O J F. Optical second harmonic generation in plasmonic nanostructures:from fundamental principles to advanced applications[J]. ACS Nano, 2015, 9(11):10545-10562.
[83]	Li G, Chen S, Pholchai N, et al. Continuous control of the nonlinearity phase for harmonic generations[J]. Nature Materials, 2015, 14(6):607-612.
[84]	Malkiel I, Mrejen M, Nagler A, et al. Plasmonic nanostructure design and characterization via deep learning[J]. Light-Science Applications, 2018, 7:60.
[85]	Liu D, Tan Y, Khoram E, et al. Training deep neural networks for the inverse design of nanophotonic structures[J]. ACS Photonics, 2018, 5(4):1365-1369.
[86]	Ma W, Cheng F, Liu Y. Deep learning enabled on-demand design of chiral metamaterials[J]. ACS Nano, 2018, 12(6):6326-6334.
[87]	Peurifoy J, Shen Y, Jing L, et al. Nanophotonic particle simulation and inverse design using artificial neural networks[J]. Science Advances, 2018, 4(6):aar4206.
[88]	Liu Z, Zhu D, Rodrigues S P, et al. Generative model for the inverse design of metasurfaces[J]. Nano Letters, 2018, 18(10):6570-6576.
[89]	Tahersima M H, Kojima K, Koike-Akino T, et al. Deep neural network inverse design of integrated photonic power splitters[J]. Scientific Reports, 2019, 9:1368.
[90]	Yao K, Unni R, Zheng Y. Intelligent nanophotonics:merging photonics and artificial intelligence at the nanoscale[J]. Nanophotonics, 2019, 8(3):339-366.
[91]	Su L, Piggott A Y, Sapra N V, et al. Inverse design and demonstration of a compact on-chip narrowband three-channel wavelength demultiplexer[J]. ACS Photonics, 2018, 5(2):301-305.
[92]	Piggott A Y, Lu J, Babinec T M, et al. Inverse design and implementation of a wavelength demultiplexing grating coupler[J]. Scientific Reports, 2014, 4:7210.
[93]	Molesky S, Lin Z, Piggott A Y, et al. Inverse design in nanophotonics[J]. Nature Photonics, 2018, 12(11):659-670.
[94]	Lan L, Sun F, Liu Y, et al. Experimentally demonstrated a unidirectional electromagnetic cloak designed by topology optimization[J]. Applied Physics Letters, 2013, 103(12):121113.
[95]	Fujii G, Watanabe H, Yamada T, et al. Level set based topology optimization for optical cloaks[J]. Applied Physics Letters, 2013, 102(25):251106.
[96]	Sell D, Yang J, Doshay S, et al. Periodic dielectric metasurfaces with high-efficiency, multiwavelength functionalities[J]. Advanced Optical Materials, 2017, 5(23):00645.
[97]	Deng Y, Korvink J G. Topology optimization for three-dimensional electromagnetic waves using an edge element-based finite-element method[C]//Proceedings of the Royal Society A-mathematical physical and engineering sciences, 2016, 472(2189):20150835.
[98]	Andkjaer J, Sigmund O. Topology optimized low-contrast all-dielectric optical cloak[J]. Applied Physics Letters, 2011, 98(2):021112.
[99]	Lin Z, Groever B, Capasso F, et al. Topology-optimized multilayered metaoptics[J]. Physical Review Applied, 2018, 9(4):044030.
[100]	Yin Baocai, Wang Wentong, Wang Litong. Review of deep learning[J]. Journal of Beijing University of Technology, 2015, 41(1):48-59. (in Chinese)
[101]	Barbastathis George, Ozcan Aydogan, Situ Guohai, On the use of deep learning for computational imaging[J]. Optica, 2019, 6(8):921-943.

Deep learning algorithm and its application in optics

doi: 10.3788/IRLA201948.1226004

Abstract

References

Proportional views

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Related

Proportional views