Articleshttps://doi.org/10.1038/s41566-019-0474-7

Video-rate imaging of biological dynamics at centimetre scale and micrometre resolutionJingtao Fan   1,7, Jinli Suo1,7, Jiamin Wu   1,7, Hao Xie1,7, Yibing Shen2, Feng Chen1, Guijin Wang3, Liangcai Cao4, Guofan Jin4, Quansheng He5, Tianfu Li6, Guoming Luan6, Lingjie Kong   4*, Zhenrong Zheng   2* and Qionghai Dai1*

1Department of Automation, Tsinghua University, Beijing, China. 2State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, Hangzhou, China. 3Department of Electronic Engineering, Tsinghua University, Beijing, China. 4State Key Laboratory of Precision Measurement Technology and Instruments, Department of Precision Instrument, Tsinghua University, Beijing, China. 5State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China. 6Department of Neurosurgery, Brain Institute, and Department of Neurology, Epilepsy Centre, Sanbo Brain Hospital, Capital Medical University, Beijing, China. 7These authors contributed equally: Jingtao Fan, Jinli Suo, Jiamin Wu, Hao Xie. *e-mail: konglj@tsinghua.edu.cn; zzr@zju.edu.cn; qhdai@tsinghua.edu.cn

SUPPLEMENTARY INFORMATION

In the format provided by the authors and unedited.

NaTure PHoToNiCS | www.nature.com/naturephotonics

Supplementary Information

Video-rate imaging of biological dynamics at centimetre-scale

and micrometre-resolution

Jingtao FAN*, Jinli SUO*, Jiamin WU*, Hao XIE*, Yibing SHEN, Feng CHEN, Guijin

WANG, Liangcai CAO, Guofan JIN, Quansheng HE, Tianfu LI, Guoming LUAN,

Lingjie KONG†, Zhenrong ZHENG†, and Qionghai DAI†

Supplementary Figure S1 Full prescription data for the customized objective lens.

Supplementary Figure S2 Full prescription data for one collecting unit. Supplementary Figure S3 The ray trace of RUSH. Supplementary Figure S4 Curves of designed average Optical Transfer

Function (OTF) at different sub-FOVs. Supplementary Figure S5 Curves of designed average OTF at edge sub-

FOVs. Supplementary Figure S6 Measured Modulation Transfer Function (MTF). Supplementary Figure S7 The CAD sectional views of the optical and

mechanical elements. Supplementary Figure S8 Photographs of RUSH and the major components. Supplementary Figure S9 The calibration steps of the local-to-global map

table for seamless sub-FOV stitching. Supplementary Figure S10 Diagram for the data transmission, storage and

image processing. Supplementary Figure S11 The graphical user interface of RUSH. Supplementary Figure S12 Cross-sections of the RUSH’s PSF. Supplementary Figure S13 Resolution enhancement by multiple frames with

sub-pixel shifts. Supplementary Figure S14 Illustration of the overlap area between adjacent

sub-FOVs. Supplementary Figure S15 High-resolution wide-field imaging of a porcine

uterus section. Supplementary Figure S16 High-throughput calcium imaging of the non-

periodical dynamics in a cardiac cellular ensemble. Supplementary Figure S17 Large-FOV, high-resolution imaging of

mitochondria dynamics in an ensemble of neurons. Supplementary Figure S18 In vitro calcium imaging in an acute human cortical

slice at single-cell resolution showing the evoked epileptiform activity.

Supplementary Figure S19 Large-FOV, high-resolution, structural imaging of Thy1-YFP mouse brains in vivo.

Supplementary Figure S20 Large-FOV, high-resolution, calcium imaging of virus-infected adult C57BL/6 mouse brains in vivo.

Supplementary Figure S21 Large-FOV, high-resolution, calcium imaging of adult Thy1-GCaMP6s mouse brains in vivo.

Supplementary Figure S22 Observation of the calcium signal in cultured rat’s cardiomyocyte placed in a microfluidic device at room temperature for high-content drug screening.

Supplementary Figure S23 Implementations of HiLo illumination on our RUSH platform

Supplementary Figure S24 Large-FOV, high-resolution imaging of a clarified brain slice from Thy1-YFP mice, with HiLo illumination (left column) and uniform illumination (right column).

Supplementary Figure S25 Large-FOV, high-resolution imaging of a brain slice from Thy1-YFP mice with HiLo illumination (left column) and uniform illumination (right column).

Supplementary Video 1 High-throughput calcium imaging of cardiac cellular ensembles with periodical dynamics.

Supplementary Video 2 High-throughput calcium imaging of cardiac cellular ensembles with non-periodical dynamics.

Supplementary Video 3 High-throughput calcium imaging of neuron cellular ensembles.

Supplementary Video 4 In vitro calcium imaging of spontaneous epileptiform activities in acute human cortical slices.

Supplementary Video 5 In vitro calcium imaging in a human cortical slice at single-cell resolution showing the evoked epileptiform activity.

Supplementary Video 6 Tracking of immune cells. Supplementary Video 7 Simultaneous imaging of vascular morphological

dynamics and neural calcium signals in awake, behaving Thy1-GCaMP6s mice.

Supplementary Video 8 In vivo brain-wide calcium imaging of neural activities in virus-infected adult C57BL/6 mice.

Supplementary Video 9 High-throughput calcium imaging of the cardiac cellular ensemble in a microfluidic device.

Supplementary Figure S1 Full prescription data for the customized objective lens. (a) The ray trace of the objective lens with the labels of several surfaces. The designed wavelength ranges from 420 nm to 680 nm. (b) The list for the prescription data.

Supplementary Figure S2 Full prescription data for one collecting unit. (a) The ray trace of the collecting unit with the labels of several surfaces. The designed wavelength ranges from 420 nm to 680 nm. (b) The list for the prescription data.

Supplementary Figure S3 The ray trace of RUSH. (a) The ray trace of the detection path. We design an objective with a spherical intermediate image surface, followed by an array of collecting units. The objective includes 14 spherical lenses in 11 groups, 3 of which are achromatic doublet lenses. Each collecting unit is composed of a field lens for FOV division, 8 lenses for aberration correction and a sCMOS camera for image acquisition. The field lenses are designed to match the chief ray of the objective to that of the collecting units, and further reduce the vignetting effect. The position of the spherical intermediate image is illustrated with brown solid curve. (b) The magnified illustration for FOV division with a typical example of a point P at the overlap of two adjacent sub-FOVs. Its image point P’ on the image plane formed by the objective without the field lens is shown by the dashed red line. After going through the field lenses and two collecting units, the light rays from P arrive at P1 and P2 at two different sensors. This beam splitting divides the numerical aperture and reduces the spatial resolution, so we set 3.6% overlap between adjacent sub-FOVs to minimize the resolution reduction within optical alignment precision for seamless stitching.

Supplementary Figure S4 Curves of designed average Optical Transfer Function (OTF) at different sub-FOVs. (a) The indexes of different sub-FOVs. Each sub-FOV corresponds to one sensor. (b) The average OTF curves of different regions in sub-FOV #0, which is in the middle of the whole FOV. (c-f) The average OTF curves of different regions in sub-FOV #16, #26, #21, #19, respectively, which are located at the edge of the whole FOV.

Supplementary Figure S5 Curves of designed average OTF at edge sub-FOVs. (a-d) The average OTF curves of different regions in sub-FOV #34, #32, #33, #29, respectively, which are located at the margin of the whole FOV (as labelled in Supplementary Fig. S4a).

Supplementary Figure S6 Measured Modulation Transfer Function (MTF). (a) The image of a customized resolution chart demonstrating the FOV and lateral resolution of RUSH. The chart is manufactured via high-precision focused ion beam direct writing, including 5×7 groups of repetitive patterns, each of which consists of at four resolution levels (8 μm, 4 μm, 2 μm, and 1 μm, respectively). (b) Zoom-ins at highlighted central and marginal sub-FOVs. (c) The average MTF with error bars, denoting the standard deviation, over the whole FOV in horizontal and vertical directions. (d) The profiles along the selected lines in (b). Scale bars, 1000 μm (a), 100 μm (b).

Supplementary Figure S7 The CAD sectional views of the optical and mechanical elements. (a) The objective lens with a maximum diameter of 161 mm and a maximum length of 202 mm. (b) The collection unit of a sub FOV, including a field lens and a camera system. The size of each collection unit is around 65 mm×70 mm×484 mm.

Supplementary Figure S8 Photographs of RUSH and the major components. (a) The major structure is placed on a 1.2 m×2.4 m optical table. The height of the system is around 2.38 m. (b) A 5×7 field lens array is mounted in a spherical surface with a radius of 1900mm and an area of 325 mm×385 mm. (c) The customized objective covering 10 mm×12 mm FOV with 0.35 NA. (d) A 5×7 camera array, each of which is composed of 9 spherical lenses (in 3 groups), to record the sub-FOV images. All the cameras are equipped with water-cooling systems to maintain the working temperature at 10°C.

Supplementary Figure S9 The calibration steps of the local-to-global map table for seamless sub-FOV stitching. STEP 1: We set the global physical coordinates using the temporal-division coding of different cells on a high-density LCD module (~500 PPI). Specifically, 12-bit temporal codes are used for x and y coordinates, respectively. STEP 2: Each sub-FOV camera captures an image sequence of one specific small LCD region. Then the corresponding global position of each LCD cell can be decoded from its temporal pattern. STEP 3: For each cell in the sub-FOV image, we further extract its precise local image coordinates using centre-of-area method, including binarization and weighted averaging. STEP 4: After retrieving the global and local coordinates of all the sub-FOVs, we build a set of homographic matrices {H#} for successive stitching.

Supplementary Figure S10 Diagram for the data transmission, storage and image processing. We use a root server to control the cameras, the computing cluster, the user client and other external devices including the synchronizer, light source, 3-axis stage, calibrator, and the monitor. Firstly, 35 cameras are synchronized to capture the sub-FOV images based on our customized protocol. The high-bandwidth data is then transmitted through a DFS for data processing. Parallel stitching algorithm is conducted based on the calibrated homographic matrices for real-time display. Users can specify the parameters such as exposure time and illumination intensity by the user control client.

Supplementary Figure S11 The graphical user interface of RUSH. The developed software enables parameter configuration and flexible visualization of the multi-scale data. Both the global view (a, online seamlessly stitched) and user-defined zoom-ins (b) are shown on two high-definition displays simultaneously.

Supplementary Figure S12 Cross-sections of the RUSH’s PSF. (a) The x-y sections of 35 sub-FOVs. (b) The statistical distribution of lateral FWHMs from 500 nm-diameter fluorescence beads across the whole FOV. (c) The x-z sections of 35 sub-FOVs. (d) The statistical distribution of axial FWHMs. Scale bars, 5 μm (a), 20 μm (c).

Supplementary Figure S13 Resolution enhancement by multiple frames with sub-pixel shifts. (a) The whole-FOV image of a USAF 1951 chart. (b) The comparison of the zoom-in marked in a. Left panel: the image from a single frame. Right panel: the reconstructed super-resolution image by sub-pixel shifts. The resolution can be enhanced to resolve the last element of the resolution chart (group 9 element 1), which corresponds to 0.98 μm line width. (c) The profiles of the selected lines marked in b to show the resolution improvement. (d) The whole-FOV image of a slide of sieve elements. (e) The comparison of the magnified region marked in d. Left panel: the image from a single frame. Right panel: the reconstructed super-resolution image by sub-pixel shifts. Scale bars, 1000 μm (a, d), 20 μm (b, e).

Supplementary Figure S14 Illustration of the overlap area between adjacent sub-FOVs. (a) The stitched image of the whole FOV, which has 170,081,961 Effective Pixels (EP) in one frame, including 163,813,561 pixels captured by single cameras (96.4% of EP, shown in light green), 6,268,400 pixels captured by two or more cameras (3.6% of EP, shown in dark green). (b) The zoom-in of one stitching area marked in a. There are 54,519 Non-Effective Pixels (NEP) without information (0.032% of EP and NEP, shown in black), which exist at the crossing point of the 4 adjacent stitching seams, among each four adjacent sub-FOVs. (c) A typical example of fluorescence imaging in the area corresponding to b verifying that there is no information lost at the stitching area except for the crossing point. Scale bars, 200 μm.

Supplementary Figure S15 High-resolution wide-field imaging of a porcine uterus section. (a) The field of view for RUSH, in comparison with conventional 2.5× (Zeiss, EC Plan-Neofluar 2.5×/0.075 NA) and 10× (Zeiss, EC Plan-Neofluar 10×/0.30 NA Ph1) objectives. (b) The snapshot giga-pixel image acquired by the RUSH system. (c) The zoomed-in view of a highlighted region in (b). (d, e) The images from conventional objectives of the same highlighted region, which were captured with a commercial microscope (Zeiss Observer Z1) and a commercial EMCCD (Andor iXon Ultra 888). Scale bars, 1000 μm (b), and 20 μm (c-e).

Supplementary Figure S16 High-throughput calcium imaging of the non-periodical dynamics in a cardiac cellular ensemble. (a) The colour-coded spatio-temporal projection of the calcium signals from cultured rat cardiomyocytes. Different

colours visualize the peak instants of the calcium signal in the selected time window (from 20 s to 25 s), and the intensity corresponds to the standard deviation of the fluorescence signal. (b) The magnified view of a highlighted region in a and the cell segmentation result. (c) The calcium signal of 12 labelled exemplar cells in (b). (d) The calcium intensity traces (ΔF/F0) of all segmented cell groups, indexed by their spatial positions. (e, g) The calcium signal of a small region including multiple cells. Left: the intensity map with several points with strong responses. Right: the temporal traces of the labelled points over a small time-window. (f) The peak instants (with polynomial fitting to the temporal traces) across the region in e. Here the intensity of each pixel denotes the standard deviation of the temporal response over the selected time window. (h) The colour-coded peak instants of the signal in g. Scale bars, 1000 μm (a), 100 μm (b), 20 μm (e, f, g, h).

Supplementary Figure S17 Large-FOV, high-resolution imaging of mitochondria dynamics in an ensemble of neurons. Whole FOV view (a) and magnified views of the dashed rectangles (b-e), with the traces visualizing the movements of segmented mitochondria and the colour encoding the velocity. Inset in a: statistical distribution of the movement velocity. (f-i) Further magnified views for each solid rectangle labelled in b-e, respectively. In each sequence, the localization of a target mitochondria is labelled with a red solid dot. (j) Further magnified views of h, showing the mitochondrial movement under tracking illustrated by a red arrow. With RUSH platform, the individual mitochondria behaviour is now be distinctly resolved regarding to the morphological complex geometry of neurons in the network scale, which possibly allows people to delineate how mitochondria behaviours are coordinated at pre- and post-synapse and cell body in response to a local stimulus in the neural network. Scale bars, 1000 μm (a), 200 μm (b, c, d, e), 30 μm (f, g, h, i, j).

Supplementary Figure S18 In vitro calcium imaging in an acute human cortical slice at single-cell resolution showing the evoked epileptiform activity. (a) The temporal intensity projection of the whole brain slice with two zoom-ins shown in (b) and (c). (d) Temporal traces of the extracted neurons visualizing the propagation of the calcium waves. The neurons are all indexed by their locations from left to right. The blue arrows indicate the propagation of the evoked wave. Note those echoes after the evoked activities at the time points of 32, 52 and 125 second. Flash: electrode position in a and trigger moments in d. Scale bars, 1000 μm (a), 200 μm (b, c).

Supplementary Figure S19 Large-FOV, high-resolution, structural imaging of Thy1-YFP mouse brains in vivo. A global view is shown in (a) and different zoom-ins are selected out for the observation of the fine neural networks in (b-d), after background subtractions. Scale bars, 1000 μm (a), 200 μm (b), 100 μm (c), 20 μm (d).

Supplementary Figure S20 Large-FOV, high-resolution, calcium imaging of virus-infected adult C57BL/6 mouse brains in vivo. (a) The large-FOV, high-resolution image (maximum intensity projection) of a mouse brain cortex. (b) A zoomed-in view (standard deviation projection) of a small region as labelled in a. (c) Fluorescence dynamics of segmented neurons as labelled in b. (d) A zoomed-in view of the region highlighted in b, showing clear structures of dendrites. (e) The calcium signals of labelled ROIs along dendrites. (f) The colour coded peak instants of the signal in d, illustrating the propagation of calcium signal within a single neuron. Such experiments can provide multi-scale data required in investigating the brain-wide correlation among the neuron responses, including the response delay of the calcium signals at different branches of single neurons. Scale bars, 1000 μm (a), 100 μm (b), 20 μm (d, f).

Supplementary Figure S21 Large-FOV, high-resolution, calcium imaging of adult Thy1-GCaMP6s mouse brains in vivo. (a) The large-FOV, high-resolution image (maximum intensity projection) of a mouse brain cortex. (b and c) Zoom-ins of the regions labelled in a, with calcium signals from corresponding segmented neurons in the right panels. (d) Temporal traces of calcium signals from 914 neurons across the whole FOV. Scale bars, 1000 μm (a), 100 μm (b, c).

Supplementary Figure S22 Observation of the calcium signal in cultured rat’s cardiomyocyte placed in a microfluidic device at room temperature for high-content drug screening. (a) The maximum intensity projection image of the whole FOV with an inset showing the photo of the microfluidic device. (b) Selected zoom-in in a. (c) Selected zoom-in in b. (d) The temporal traces of the selected ROIs shown in (c). The cells are stained with Fluo-4 AM. Scale bars, 1000 μm (a), 100 μm (b, c).

Supplementary Figure S23 Implementations of HiLo illumination on our RUSH platform. (a) The schematic diagram of the HiLo illumination implemented on the RUSH platform. We replace the original light source with a laser beam passing a diffuser to produce speckle illumination. A one-dimensional galvo is used to reflect the laser beam and the illumination shifts between the speckled and uniform patterns at high speed, which is commonly used for HiLo implementation1. (b) A small region of a thin Fluorescein isothiocyanate (FITC) layer on a glass slide imaged by the RUSH platform, with the galvo static. When the galvo is static, we can obtain the speckle illumination to estimate the background fluorescence. When the galvo is scanning, we can get the uniform illumination for the in-focus information. (c, d) The image of a fixed brain slice from Thy1-YFP mice with uniform illumination by the RUSH platform. (e, f) The same slice as in c, d, while imaged with speckle illumination by the RUSH platform. (g, h) The HiLo reconstruction results obtained by the open-source HiLo software1 utilizing the images under uniform illumination and speckle illumination. The whole pipeline follows the previous study1. Scale bars, 50 μm (b), 1000 μm (c, e, g), 200 μm (d, f, h).

Supplementary Figure S24 Large-FOV, high-resolution imaging of a clarified brain slice from Thy1-YFP mice, with HiLo illumination (left column) and uniform illumination (right column). (a, c, e and b, d, f) The global view and two zoomed-in views of highlighted regions, in HiLo illumination and uniform illumination, respectively. Scale bars, 1000 μm (a, b), 100 μm (c, d, e, f).

Supplementary Figure S25 Large-FOV, high-resolution imaging of a fixed brain slice from Thy1-YFP mice with HiLo illumination (left column) and uniform illumination (right column). (a, c) The global view and a zoomed-in view of the highlighted region in HiLo mode, with colour encoding the depth. (b, d) The counterparts (maximum intensity projection of the focal stack) under uniform illumination. Scale bars, 1000 μm (a, b), 100 μm (c, d).

References 1. Lim, D., Chu, K. K. & Mertz, J. Wide-field fluorescence sectioning with hybrid speckle and

uniform-illumination microscopy." Opt. Lett. 33, 1819-1821 (2008).