three-dimensional board-to-board free-space optical interconnects and their application to the...

8
Three-dimensional board-to-board free-space optical interconnects and their application to the prototype multiprocessor system: COSINE-III Toshikazu Sakano,Takao Matsumoto, and Kazuhiro Noguchi A prototype multiprocessor system using three-dimensional board-to-board free-space optical intercon- nects is constructed for the first time to our knowledge. In the system, 64 processing units form a three-dimensional mesh processor network with the help of bidirectional board-to-board free-space optical interconnects. A theoretical analysis shows that the three-dimensional board-to-board free- space optical interconnects effectively solve common interconnection problems such as wiring congestion, signal delay, and clock skew. The prototype system, COSINE-III, is confirmed to work well as a multiprocessor system. The system is also shown to be easy to extend to a larger and more flexible system. Key words: Free-space optical interconnects, interconnection network, parallel processing, three- dimensional mesh, differential signal transmission. 1. Introduction The performance bottleneck of advanced multiproces- sor systems is usually the conventional electrical interconnects. The problem is being recognized as the interconnection bottleneck. Innovative intercon- nection technology to overcome the bottleneck is urgently needed. 1 Optical interconnects are ex- pected to be the solution because of their excellent potential. 2 Their advantages include high speed, wide bandwidth, freedom from electromagnetic inter- ference, and configuration flexibility. In addition to these advantages, free-space optical interconnects offer the unique feature that the interconnects can be established without any physical contact. This feature enables the direct implementation of three- dimensional 13-D2 interconnects in all the interconnec- tion layers in the system, from chip-to-chip to frame- to-frame. Multiprocessor systems are usually constructed by stacking of processor boards in a machine frame, each of which accommodates processors, memory, and other electronic devices. As the number of proces- sors and the clock speed are increased in such sys- tems, board-to-board interconnects, usually set up on a backplane, create critical drawbacks such as wiring congestion, increased signal delay, and clock skew. To overcome these drawbacks, some multiprocessor systems that apply free-space optical interconnects to the backplane have been reported. 3–5 Although these systems could overcome the drawbacks fairly well, the unique feature of free-space optical interconnects mentioned above was not fully utilized. The feature enables one to establish interconnects anywhere on the boards to connect the adjacent nodes on neighbor- ing boards with the shortest path length and without physical contact. 6 A system using direct board-to- board interconnects will have smaller delay and clock skew and be more flexible in terms of system construction than systems that use backplane inter- connects. In this paper a prototype multiprocessor system that uses 3-D board-to-board free-space optical inter- connects is reported for the first time. The system, which we call COSINE-III 1computer system employ- ing optical spatial interconnections for experimenta- tion III2 is made of 64 processing units interconnected in a 3-D mesh network through direct board-to-board free-space optical interconnects and intraboard elec- trical links. In Section 2, the features of the 3-D mesh network based on the direct board-to-board free-space optical interconnects are analyzed theoreti- cally. In Section 3 a signal-transmission scheme suitable for direct free-space optical interconnects is introduced, and a preliminary experiment is reported. The authors are with NTT Optical Network Systems Laborato- ries, 1-2356 Take, Yokosuka-Shi, Kanagawa-Ken 238-03, Japan. Received 27 July 1994; revised manuscript received 14 November 1994. 0003-6935@95@111815-08$06.00@0. r 1995 Optical Society of America. 10 April 1995 @ Vol. 34, No. 11 @ APPLIED OPTICS 1815

Upload: kazuhiro

Post on 05-Oct-2016

212 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Three-dimensional board-to-board free-space optical interconnects and their application to the prototype multiprocessor system: COSINE-III

Three-dimensional board-to-board free-spaceoptical interconnects and their application to theprototype multiprocessor system: COSINE-III

Toshikazu Sakano, Takao Matsumoto, and Kazuhiro Noguchi

A prototype multiprocessor system using three-dimensional board-to-board free-space optical intercon-nects is constructed for the first time to our knowledge. In the system, 64 processing units form athree-dimensional mesh processor network with the help of bidirectional board-to-board free-spaceoptical interconnects. A theoretical analysis shows that the three-dimensional board-to-board free-space optical interconnects effectively solve common interconnection problems such as wiring congestion,signal delay, and clock skew. The prototype system, COSINE-III, is confirmed to work well as amultiprocessor system. The system is also shown to be easy to extend to a larger andmore flexible system.Key words: Free-space optical interconnects, interconnection network, parallel processing, three-

dimensional mesh, differential signal transmission.

1. Introduction

The performance bottleneck of advanced multiproces-sor systems is usually the conventional electricalinterconnects. The problem is being recognized asthe interconnection bottleneck. Innovative intercon-nection technology to overcome the bottleneck isurgently needed.1 Optical interconnects are ex-pected to be the solution because of their excellentpotential.2 Their advantages include high speed,wide bandwidth, freedom from electromagnetic inter-ference, and configuration flexibility. In addition tothese advantages, free-space optical interconnectsoffer the unique feature that the interconnects canbe established without any physical contact. Thisfeature enables the direct implementation of three-dimensional 13-D2 interconnects in all the interconnec-tion layers in the system, from chip-to-chip to frame-to-frame.Multiprocessor systems are usually constructed by

stacking of processor boards in a machine frame, eachof which accommodates processors, memory, andother electronic devices. As the number of proces-sors and the clock speed are increased in such sys-

The authors are with NTT Optical Network Systems Laborato-ries, 1-2356 Take, Yokosuka-Shi, Kanagawa-Ken 238-03, Japan.Received 27 July 1994; revised manuscript received 14 November

1994.0003-6935@95@111815-08$06.00@0.

r 1995 Optical Society of America.

tems, board-to-board interconnects, usually set up ona backplane, create critical drawbacks such as wiringcongestion, increased signal delay, and clock skew.To overcome these drawbacks, some multiprocessorsystems that apply free-space optical interconnects tothe backplane have been reported.3–5 Although thesesystems could overcome the drawbacks fairly well,the unique feature of free-space optical interconnectsmentioned above was not fully utilized. The featureenables one to establish interconnects anywhere onthe boards to connect the adjacent nodes on neighbor-ing boards with the shortest path length and withoutphysical contact.6 A system using direct board-to-board interconnects will have smaller delay and clockskew and be more flexible in terms of systemconstruction than systems that use backplane inter-connects.In this paper a prototype multiprocessor system

that uses 3-D board-to-board free-space optical inter-connects is reported for the first time. The system,which we call COSINE-III 1computer system employ-ing optical spatial interconnections for experimenta-tion III2 is made of 64 processing units interconnectedin a 3-D mesh network through direct board-to-boardfree-space optical interconnects and intraboard elec-trical links. In Section 2, the features of the 3-Dmesh network based on the direct board-to-boardfree-space optical interconnects are analyzed theoreti-cally. In Section 3 a signal-transmission schemesuitable for direct free-space optical interconnects isintroduced, and a preliminary experiment is reported.

10 April 1995 @ Vol. 34, No. 11 @ APPLIED OPTICS 1815

Page 2: Three-dimensional board-to-board free-space optical interconnects and their application to the prototype multiprocessor system: COSINE-III

The configuration of the fabricated prototype systemand the results of preliminary tests are then de-scribed in Section 4. The extension of the system isdiscussed in Section 5.

2. Features of Three-Dimensional Interconnects

A. Reduction of Signal Delay and Skew

Three-dimensional board-to-board free-space opticalinterconnects offer unique features. Figures 11a2 and11b2 show the conceptual structures of conventionalbackplane interconnects and 3-D free-space opticalinterconnects, respectively. Board-to-board intercon-nects in multiprocessor systems are usually estab-lished on a backplane that is perpendicular to theprocessor boards and touches an edge of each proces-sor board, as shown in Fig. 11a2. In such systemsbased on the backplane interconnects, all intercon-nects across the processor boards have to pass throughthe backplane. The backplane interconnects thuscause wiring congestion, signal delay, and clock skew.On the other hand, the 3-D free-space optical intercon-nects shown in Fig. 11b2 can reduce those problemsbecause the processors on a board can be intercon-nected directly with the adjacent processors on theneighboring boards. In this section the signal-delaycharacteristics of backplane interconnects and free-space interconnects are compared theoretically.Figure 2 shows the system model used for the

comparison. Themodel is a 3-D cubicmeshmultipro-cessor system that is composed of n stacked processorboards. Each board accommodates n2 processors1nodes2 arranged and interconnected in a two-di-mensional 12-D2 square mesh. The total number ofthe nodes in the system, N, is n3. In the system

Fig. 1. Conceptual structures of 1a2 conventional backplane inter-connects and 1b2 3-D free-space optical interconnects.

1816 APPLIED OPTICS @ Vol. 34, No. 11 @ 10 April 1995

using backplane interconnects a backplane is placedon the x–z plane and all board-to-board communica-tion is performed only through the backplane. In thenetwork using the 3-D board-to-board free-space opti-cal interconnects, all pairs of adjacent nodes onneighboring processor boards are connected directlywith free-space optical interconnects.The signal delay between two adjacent nodes for an

intraboard interconnect and that for a board-to-boardinterconnect are assumed to beT and aT, respectively.The constant a is introduced because the signal delayof board-to-board interconnects depends on the spac-ing between the boards. We ignore the difference ofpropagation delay between optical and electrical inter-connects. Let us consider the signal delay for commu-nication between two nodes,Pi1xi, yi, zi2 andPj1xj, yj, zj210 # xi, yi, zi, xj, yj, zj , n, 0 # i, j , N 2. The mini-mum delay of the network using the direct free-spaceoptical interconnect, Dopt1i, j2, and that using thebackplane interconnect, Dbkp1i, j2, are then expressedas

Dopt1i, j2 5 T50xj 2 xi 0 1 0yj 2 yi 0 1 a0zj 2 zi 06, 112

Dbkp1i, j2

5 5T50xj 2 xi 0 1 0yj 2 yi 06 1zj 5 zi2

T50xj 2 xi 0 1 0yj 1 yi 0 1 a0zj 2 zi 06 1zj ∞ zi2,

122

where the routing time at each node to relay a signalis ignored.Equations 112 and 122 were calculated for all pos-

sible node pairs with N 5 1000 for a 5 1 and a 5 0.2.The histogram of the minimum signal delay normal-ized by T is shown in Fig 3. The center values andthe widths of the curve for the free-space opticalinterconnects are smaller than those for the back-plane interconnects for both a values. The meansthat the average delay and the clock skew of theformer are smaller than those of the latter. Figure 4

Fig. 2. System model used for signal-delay analysis of backplaneinterconnects and free-space optical interconnects.

Page 3: Three-dimensional board-to-board free-space optical interconnects and their application to the prototype multiprocessor system: COSINE-III

shows calculated relationships between node numberN and average delay normalized by T. A curve for a2-D square mesh on a board 12-D square mesh2 andthat for a network completely connected with directinterconnects without relays 1physical limit2 are alsoplotted for reference. The normalized average de-lays of the free-space optical interconnects are alwayssmaller than those of both the 2-D square mesh andbackboard interconnects. The direct board-to-boardfree-space optical interconnects are thus effective inreducing signal delay.Recently the processing speeds of newly developed

processors have been getting faster and faster. Inconstructing high-speed systems using such proces-sors the delay and the clock-skew problems arebecoming critical issues that must be overcome.Three-dimensional board-to-board free-space opticalinterconnection technology appears to be the only

Fig. 3. Histogram of the calculatedminimum signal delay normal-ized by T.

Fig. 4. Calculated relationships between node number N andaverage delay normalized by T.

way to fully extract the processors’ potential in thenear future.

B. Reduction of Backplane Wiring Congestion

Reduction of wiring congestion is another feature thatfree-space interconnects offers. In this section, wir-ing congestion in the systemusing backplane intercon-nects is compared with that of a free-space opticalinterconnect-based system.Here the 3-D mesh multiprocessor system based on

themodel shown in Fig. 2 is considered for comparison.In the system the nodes on edges of a 2-D meshprocessor network on each board have one or twounused interconnects. Each processor board has 8nunused interconnects in all. When the system isextended to a larger system by incorporation withother systems, the unused interconnects are con-nected to the systems through a backplane. So weassume that each processor board uses 8n backplaneinterconnects for system extension, no matter whattype of board-to-board interconnects are employed inthe system.In the case of backplane interconnects, board-to-

board connection is performed only through the back-plane. Each board needs 2n2 bidirectional back-plane interconnects to form the 3-D mesh network.Therefore the number of unidirectional interconnectsconnected through the backplane to a board, MB, isexpressed as

MB 5 4n2 1 8n. 132

In the case of free-space optical interconnects, on theother hand, board-to-board interconnects directly con-nect each processor to the adjacent processors on theneighboring boards. Backplane interconnects areused only when the system is connected to othersystems. So the number of backplane interconnectsin the system based on free-space optical intercon-nectsMO becomes

MO 5 8n. 142

Figure 5 shows the relationship between Eqs. 132 and142. By application of board-to-board free-space opti-cal interconnects the number of backplane intercon-nects in the system can be reduced dramatically.It is concluded that free-space optical interconnectsare very effective to relax the wiring congestionproblem.

3. Free-Space Optical Interconnects

Three-dimensional free-space optical interconnectsdistributed over the processor boards can be affectedby several environmental factors. The environmen-tal factors are 112 received optical power fluctuationscaused by board displacement and@or vibration, 122leakage of environment light, 132 variation in couplingefficiency between the optical sources and the detec-tors caused by device fabrication inaccuracy, and 142dc drift of electrical and optical device characteristics

10 April 1995 @ Vol. 34, No. 11 @ APPLIED OPTICS 1817

Page 4: Three-dimensional board-to-board free-space optical interconnects and their application to the prototype multiprocessor system: COSINE-III

caused by heat. Signal transmission must be keptstable despite these environmental factors. More-over, the circuits for the interconnects must be simpleso as not to waste the board area.The differential-signal-transmissionmethod7meets

these requirements and is suitable for realizing 3-Dfree-space optical interconnects. With the methodthe environmental factors, which are common to thetwo optical channels, are canceled in the process ofmaking the differential signal. This ensures thethreshold level is kept optimum in the decision pro-cess regardless of the environmental factors, and noadditional control circuits are needed.Before constructing a prototype multiprocessor

system, we fabricated a free-space optical intercon-nect based on the differential-signal-transmissionmethod, and its characteristics were evaluated.Figure 6 shows the configuration of the fabricatedinterconnect. The driver on board S modulates theoptical sources LED-1 and -2 1light-emitting diodes2with the input transistor-transistor-logic signal andits inverted equivalent, respectively. At the receiveron board R the optical beams are detected by detec-tors PD-1 and -2 1photodiodes2, respectively. Theirdifferential signal is amplified and reshaped by thecomparator. The reshaped signal is converted to atransistor-translator-logic signal by the level con-

Fig. 5. Reduction of wiring congestion by application of free-spaceoptical interconnects to a 3-D mesh multiprocessor system.

Fig. 6. Configuration of the fabricated free-space optical intercon-nect based on the differential-signal-transmission method.

1818 APPLIED OPTICS @ Vol. 34, No. 11 @ 10 April 1995

verter and output from the receiver. With the configu-ration described in Fig. 6 the environmental factorsthat are common to the two optical channels arecanceled at the comparator. That is, the optimumthreshold level in the decision process is always thesame regardless of the environmental factors as longas they are common to the two optical channels.Furthermore, because the method is equivalent tobipolar signal transmission, the optical power at eachdetector necessary to achieve a specified bit error rate1BER2 is ,3 dB less than is needed in the conventionalunipolar optical signal-transmission method.Table 1 summarizes the specifications of the fabri-

cated free-space optical interconnect. A surface-emitting-type LED 1l 5 0.82 µm2 and a collimatinglens whose diameter was 5 mm were combined toform an LED module. LED’s were used rather thanlaser diodes because of their stability and strengthagainst electrical noise. We found a PD module byattaching a focusing lens whose diameter was 5 mmto a PIN PD. Two LED modules were mounted on aboard as closely as possible 18-mm spacing2. Two PDmodules were fixed to another board with the samespacing. These two boards were placed face to face.With the setup, coupling characteristics between

the LEDmodules and the PDmodules weremeasured.The peak optical-power output from each LED mod-ule was 13.8 dBm at a drive current of 20 mA. Inmost cases, interconnection length for board-to-boardconnections was 1–5 cm. The average received opti-cal power at the facing PD module was 21.5 dBmwhen the board-to-board separation was 5 cm.Figure 7 shows the measured coupling characteristicsbetween LED-1 and PD-1. The horizontal and thevertical axis show the board displacement and thephotocurrent at PD-1, respectively. Because the op-tical beam output from the LED module is incoherentlight, the optical beam broadens dramatically as itpropagates. The original beam diameter of 4.2 mmat the output of the module increases to 6.3 mm afterpropagating 5 cm. This beam boardening causescross talk between the two LED–PD pairs. Thecurve for L 5 1 cm is asymmetric against the opticalaxis, whereas other curves are symmetric. This maybe because the LED is positioned correctly but is

Table 1. Specifications of Free-space Optical Interconnects

Optical source Surface-emitting LED 1wavelength 0.82 µm2Collimating lens Planar convex 1D 5 5 min and f 5 3.1 mm2a

Output power 13.8 dBm at a drive current of 20 mA 1typi-cal2

Detector Si-PIN photodiodeDetection region is 0.9 mm square

Focusing lens Planar convex 1D 5 5 mm and f 5 3.1 mm2a

Received power 21.5 dBm at a board spacing of 5 cm 1typi-cal2

Module spacing 8 mm 1in preliminary experiment210 mm 1in COSINE-III2

Transmissionspeed

20 Mbits@s

aD, diameter, f, focal length.

Page 5: Three-dimensional board-to-board free-space optical interconnects and their application to the prototype multiprocessor system: COSINE-III

slightly tilted against the optical axis. The tilt causesthe asymmetrical field pattern just outside the lens,owing to the diffraction at the edge of the lens andaberration. Figure 8 shows an example of measuredcross talk. The cross talk is defined as the ratio ofthe optical power coupled to PD-1 from LED-1 to thatfrom LED-2. Cross talk increased with the boarddisplacement and reached 0 dB at the displacement of23.7 mmwhen L 5 5 cm.Next, BER characteristics at 20 Mbits@s were mea-

sured. The results are shown in Fig. 9. Theresults for the unipolar-signal-transmission methodare also plotted for comparison. In the differential-signal-transmission method the average received op-tical power per detector to achieve a BER of 1029 was226.7 dBm. This value is 2.3 dB lower than that ofthe unipolar-signal-transmission method and 25.2 dBlower than the typical received power of the setup.Figure 10 shows the measured relationship between

Fig. 7. Measured coupling characteristics between LED-1 andPD-1.

Fig. 8. Measured cross-talk characteristics for the LED-1 andPD-1 pair.

BER, board-to-board separation L, and lateral boarddisplacement d. With 22.5 mm , d , 2.5 mm theBER was less than 10211 for L equal to 1–7 cm.Because tolerance for lateral board displacement is62.5 mm, it is concluded that the existing board-fabrication technologies can be used for implementingstable board-to-board interconnects.

4. Prototype Multiprocessor System: COSINE-III

A. System Configuration

The prototype multiprocessor system COSINE-IIIuses a 3-D mesh processor network. This networkconfiguration was chosen because the interconnectstherein, which are static and are short, are suitablefor direct free-space optical interconnects. More-over, the network is simple and more suitable forvarious scientific applications than other static-processor networks. Figure 11 shows the system

Fig. 9. BER characteristics of the fabricated interconnect at 20Mbits@s.

Fig. 10. Measured relationship between BER, board-to-boardseparation L, and lateral board displacement d.

10 April 1995 @ Vol. 34, No. 11 @ APPLIED OPTICS 1819

Page 6: Three-dimensional board-to-board free-space optical interconnects and their application to the prototype multiprocessor system: COSINE-III

configuration of COSINE-III. The system is com-posed of four stacked processor boards, each of whichhas 16 processing units arranged in a 2-D 4 3 4 grid.Each processing unit has localmemory and six bidirec-tional communication links to establish connectionswith its neighbors. Four of the links are electricalinterconnects that connect the processing unit withthe four adjacent ones on the same board. The tworemaining links are free-space optical interconnectsthat establish connections to the two facing process-ing units on neighboring processor boards. Eachprocessing unit can send 1receive2 data packets to1from2 the adjacent units and can relay data packetsfrom an adjacent unit to another one, all through thecommunication links. A host computer is connectedto the system through one processing-unit 1PU2 commu-nication link. The programs for each of processingunits are installed from the host computer and areexecuted in parallel.The free-space optical interconnects offer not only

small delay and small clock-skew characteristics butalso easy maintenance. It is possible to remove aprocessor board and create a new processor networkwithout physical wiring reconfiguration because theboard-to-board interconnects associated with the re-moved board are extended automatically to the nextboard. This feature simplifies system maintenance.When a processor boardmust be replaced, no intercon-nect reconfiguration is necessary except to remove theboard. If a processor board is removed fromCOSINE-III, the interconnection length will become 5 cm.A preliminary experiment on free-space optical inter-connects suggests that this interconnection lengthdoes not affect the signal-transmission characteris-tics.

B. Fabrication and Tests

COSINE-III was fabricated based on the results ofthe free-space optical interconnect preliminary test.The specifications of the fabricated system are summa-rized in Table 2. Each processing unit was formedby cascading of two Transputers 1T800’s2, each ofwhich has four 20-Mbits@s bidirectional communica-tion links and a local memory of 4 Mbytes. One

Fig. 11. System configuration of COSINE-III.

1820 APPLIED OPTICS @ Vol. 34, No. 11 @ 10 April 1995

processor board accommodates 32 bidirectional free-space optical interconnects 116 interconnects on eachside of the board2. Conventional epoxy-glass boards1438 mm 3 450 mm 3 1.6 mm2 were used as theprocessor boards. The four processor boards wereaccommodated in a machine frame. Figure 12 is anoutside view of the fabricated system with a sidepanel removed. The outside dimensions of the sys-tem are 500 mm wide, 760 mm high, and 460 mmdeep. The spacing between boards in the frame is 25mm. Figure 13 is a close-up of one processor board.The Transputers and associated memories, which arethe square chips and the densely stacked chips,respectively, are arranged two dimensionally on bothsides of the board. LED modules and PD modulesare attached to the board with stiffeners to avoidmisalignment due to board warp. In the photographthe modules are the rows of small dots on the stiffen-ers fixed between the processors.Signal transmission at 20 Mbits@s was performed

for all the 96 free-space optical links used for board-to-board interconnects in the fabricated system. Allsignal transmission was established successfully even

Table 2. Specifications of COSINE-III

Processor unit 1PU2 A pair of Transputers 1T8002Local memory@processor 4 MbytesNumber of PU’s@board 16Number of boards 4Network configuration 3-D meshNetwork scale 4 3 4 3 4Board spacing 25 mmBoard dimension 438 mm 3 450 mm 3 1.6 mmFrame dimension 500 m 3 760 mm 3 450 mmNumber of bidirectionaloptical links

48

Fig. 12 Outside view of COSINE-III with a side panel removed.

Page 7: Three-dimensional board-to-board free-space optical interconnects and their application to the prototype multiprocessor system: COSINE-III

after repeated extraction and insertion of the boards.The average delay time was 18 ns.Next, a simple programwas loaded and executed on

the fabricated system. Figure 14 shows the networkconfiguration used. The network is a cube networkcomposed of eight processing units distributed overtwo processor boards, eight intraboard interconnects1electrical2, and four board-to-board interconnects 1free-space optical2. A random data packet was continu-ously transferred from PU to PU in the upper-halfloop of the network. Another random data packetwas transferred in the lower-half loop in parallel.A PU in each loop checks for errors during datatransfer and sends the results to the host computer.This program was executed for 100 h, and we con-firmed that data transfer was error free. It can beconcluded that the fabricated system is stable enoughto use as a multiprocessor system.

5. Discussion

For application of the free-space optical interconnectsto a very-high-speed computer system whose clock

Fig. 13. Close-up of one processor board.

Fig. 14. Network configuration used for the program execution.

rate is over a few hundred megahertz, the realizationof very-high-speed and dense free-space optical inter-connects will be the key. The signal-transmissionspeed of the fabricated free-space optical intercon-nects using LED’s was as high as 50 Mbits@s, which isenough for interprocessor communication in COSINE-III. As for the interconnection density, the opticalsources, detectors, and related electrical circuits weredistributed over the processor board in COSINE-III.To increase both the transmission speed and theinterconnection density, one could use an interconnec-tion module that includes a laser diode or arraydevices.In implementation of free-space optical intercon-

nects as board-to-board connections, board warp andmisalignment of optical components seriously affectthe interconnection characteristics. One effectiveway to avoid these problems is to use the stiffeners tofix the optical components on the processor boards asin COSINE-III. Figure 15 shows an image of thestiffener that overcomes these problems. All opticalcomponents, including optical sources, detectors, driv-ers, and receivers, are embedded in the stiffener.Once these components are precisely arranged andfixed in the stiffener, pricise arrangement of thecomponents is accomplished simply by the stiffenerbeing fixed precisely on the board. Moreover, thestiffener can be used as a cooling fin.The free-space optical interconnects implemented

in COSINE-III enable board-to-board interconnectsto be established anywhere on the boards withoutphysical contact. This feature makes it easy notonly to construct a 3-D processor network in a systembut also to extend the system. Figure 16 schemati-cally shows one example of system extension. Be-cause COSINE-III is constructed by stacking of pro-cessor boards, each of which has optical interconnectports on both sides, the system also has opticalinterconnect ports on its external sides. These portscan be used to interconnect the system with othersystems. A system therefore can be extended simplyby arrangement of several system units in a row, asshown in Fig. 15. By use of reflection mirrors therows can be folded, thus compacting the layout of thesystem. Even with an extended system, board inser-tion and extraction can be performed easily.

Fig. 15. Image of the stiffener.

10 April 1995 @ Vol. 34, No. 11 @ APPLIED OPTICS 1821

Page 8: Three-dimensional board-to-board free-space optical interconnects and their application to the prototype multiprocessor system: COSINE-III

Figure 17 shows another extended system thatenables us to reconfigure the interconnects betweenthe rows of system units. Multiple rows of systemunits are arranged in parallel. At one side of eachrow a switching unit is placed that is made of apolarization beam splitter, a quarter-wave plate, areflection mirror, and two spatial light modulators.The set of polarizing beam splitters, quarter-waveplate, and reflection mirror selects the direction of theinput optical beams according to their polarizationstates.8 Spatial lightmodulators control the polariza-tion states of the passing optical beams.9 With theconfiguration shown in Fig. 17 the polarized paralleloptical beams 1S and P beams2 output from a row ofsystem units can be connected to any other row bycontrol of the spatial light modulators. In this way,by application of the polarization-control technique tothe direct free-space optical interconnects, a reconfigu-rable system can be created.As discussed above, a system based on free-space

optical interconnects can be easily extended in sizebut for which maintenance remains easy. If thetime-division and@or the frequency-division multi-plexing@demultiplexing techniques are combinedwith

Fig. 16. Example of system extension.

Fig. 17. Extended system that enables us to reconfigure theinterconnects between the rows of system units.

1822 APPLIED OPTICS @ Vol. 34, No. 11 @ 10 April 1995

direct free-space optical interconnects, throughputand flexibility of the processor network could beimproved even more.

6. Conclusion

A multiprocessor system using direct board-to-boardfree-space optical interconnects has been constructedfor the first time to our knowledge. The prototypesystem arranges 64 processing units into a 3-D meshnetwork with the help of bidirectional free-spaceoptical interconnects distributed on both sides of fourprocessor boards. An analysis of signal-delay charac-teristics of the 3-D processor network concludes thatfree-space optical interconnects are superior to conven-tional backplane technology. We confirmed that theprototype system worked well as a multiprocessorsystem. The system was also shown to be easy toextend to a larger, more flexible system. The chal-lenging technology described in this paper needscompact and reliable optoelectronic integrated cir-cuits to be developed before sophisticatedmultiproces-sor systems based on free-space optical interconnectscan become widely used.

The authors thank Tetsuya Miki and Hideki Ishiofor their guidance and encouragement.

References and Notes1. Special issue on supercomputing, IEEE Spectrum 29192, 69–75

119922.2. J. W. Goodman, F. I. Leonberger, S. Y. Kung, and R. A. Athale,

‘‘Optical interconnections for VLSI systems,’’ Proc. IEEE 72,850–866 119842.

3. Y. Okada, H. Tajima, Y. Hamazaki, and K. Tamura, ‘‘Dialog H:A highly parallel processor based on optical common bus,’’ inCOMPCON’83, 1983 OSA Technical Digest Series 1OpticalSociety of America, Washington, D.C., 19832, pp. 461–467.

4. T. Matsumoto, T. Sakano, K. Noguchi, and T. Sawabe, ‘‘Com-puter systems employing reconfigurable board-to-board free-space optical interconnections: COSINE-1 and -2,’’ Proceed-ings ICCD’90 1Institute of Electrical and Electronics Engineers,NewYork, 19902, pp. 426–429.

5. J. W. Parker, ‘‘Progress in optical interconnection technologiesand demonstrators under the ESPRIT II OLIVES programme,’’in Optical Computing, Vol. 6 of 1991 OSA Technical DigestSeries 1Optical Society of America, Washington, D.C., 19912, pp.304–307.

6. D. Z. Tsang, ‘‘Optical interconnects for digital systems,’’ IEEETrans. Aerosp. Electron. Syst. 7, 10–15 119922.

7. P. J. Ayliffe, J. W. Parker, and A. Robinson, ‘‘Comparison ofoptical and electrical data interconnections at the board andbackplane levels,’’ inOptical Interconnections and Networks,H.Bartelt, ed., Proc. Soc. Photo-Opt. Instrum. Eng. 1281, 2–15119902.

8. F. B. McCormick, F. A. P. Tooley, J. L. Brubaker, J. M. Sasian, T.J. Cloonan, A. L. Lentine, R. L. Morrison. R. J. Criscri, S. L.Walker, S. J. Hinterlong, andM. J. Herron, ‘‘Optomechanics of afree-space photonic switching fabric: the system,’’ in Optome-chanics andDimensional Stability,R.A. Paquin andD. Vukobra-tovich, eds., Proc. Soc. Photo-Opt. Instrum. Eng. 1533, 97–114119912.

9. K. Noguchi, T. Sakano, and T. Matsumoto, ‘‘A rearrangeablemultichannel free-space optical switch based on multistagenetwork configuration,’’ J. Lightwave Technol. 9, 1726–1732119912.