early days of message-passing computing: transputers ... · – ipsc1 had osf mach os on each node...
TRANSCRIPT
Early days of message-passing computing: transputers, occam
and all that
Tony Hey Chief Data Scientist
STFC Rutherford Appleton Laboratory Harwell Campus, UK
• In1981Iwasonsabba,calatCaltech–asatheore,calpar,clephysicist–andGeoffreyFoxandIwenttoacolloquiumbyCarverMead…
• CarverdemonstratedthattherewerenoengineeringobstaclestochipsgeDngsmallerandfasterforthenext20years
• IwentbacktotheUKandbuiltmessage-passingmachinesusingtheInmosTransputer
• GeoffreyFoxcollaboratedwithChuckSeitzinbuildingahypercubemessage-passingmachinethatwasusableforscien,ficapplica,ons
TheBeginnings
• Designedandbuiltintheearly1980’sbyGeoffreyFoxandChuckSeitzandtheirteamsinPhysicsandCS
• Processorsatnodesofhypercube;messagepassingbetweennodes
• Experimentedwithparallelizingawholesetofscien,ficapplica,ons
• Developed‘CrystallineOS’–CrOS–whichwasreallyalibraryofcommunica,onrou,nes
• Demonstratedadvantagesofvirtualaddresses,virtualcommunica,onchannelsandkernel-likesupportateachnode
TheCaltechCosmicCube
• Exploiteddataparallelismofregularproblemsby‘domaindecomposi,on’
• Forhighefficiency,needforlightweightkernelsonnodesthatallowedforlowlatencymessagestartup,mes
• Laidthefounda,onsforparallelprogrammingmethodologyandparallelperformanceanalysisthatares,llrelevanttoday
• Irregularproblemsweremoredifficult…
Lessonslearnt
• TheIntelPersonalSupercomputer– iPSC1hadOSFMachOSoneachnodewhichhadveryhighlatencyforini,a,ngcommunica,ons
– iPSC2releasedsoonaZerwardswithNX‘DistributedProcess’environmentbasedonCaltech’s‘Reac,veKernel’OS
• TheTransputerSupernodemachine– BasedontheInmosT800transputerthatcombinedCPU,FPU,memoryandcommunica,onchannelsonchip
– Na,veprogramminglanguagewas‘occam’,arealiza,onofasimplifiedversionofHoare’sCSP
– EU‘Supernode’project:MachinesmanufacturedbyTelMatandParsys
TheParallelCompu7ngLandscape(1)
• Manyothervendorsofparallelmessage-passingmachines:– nCUBE– MeikoCS-1andCS-2– Suprenum– Parsytec– IBMSPseries– …
Ø Eachvendorhadproprietarymessagepassingsystem
TheParallelCompu7ngLandscape(2)
• ThePARMACSmacrosfromtheArgonneteam– RustyLusketal.‘Mark1’
• Thep4parallelprogrammingsystem– RustyLusketal.‘Mark2’
• TheParallelVirtualMachinePVM– VaidySunderam,AlGeistandothers– Supportedmessagepassingacrossheterogeneousdistributedsystems
• ThePARMACSmessagepassinglibraries– DevelopedbyRolfHempelandothersintheEU‘PPPE’project
PortableMessagePassingInterfaces?
• In1991GeoffreyFoxandKenKennedystartedacommunityprocesstowardsadataparallelFortranstandard– ThisbecametheHighPerformanceFortraneffortandtypifiedthe‘heroic’compilerschoolofparallelprogramming
• However,whatwasclearlyneededwasalowerlevelstandardforportabilityofmessagepassingprogramsacrossdifferentparallelcomputers– TheUSwereusingp4andExpress– TheEUwereusingPARMACSinthePPPEandRAPSprojects– PVMwaswidelyusedforprogrammingnetworksofworksta,onsbutnotop,mizedformorecloselycoupledparallelmachines
TheOriginsofMPI(1)
• WorkshoponStandardsforMessagePassinginaDistributedMemoryEnvironment– Williamsburg,Virginia,April1992– OrganizedbyJackDongarraandDavidWalker– SponsoredbyCRPCandKenKennedyurgedac,on
• Insummerof1992,IcontactedJackDongarraaboutstar,ngsuchastandardiza,onac,vity– DidnotwantUSandEuropetodiverge– Co-wroteafirstdraZofanMPIstandardwithJackDongarra,RolfHempelandDavidWalkerinOctober1992,nowknownasMPI-0
TheOriginsofMPI(2)
OrganizedBOFsessionatSuperCompu,ng92inMinneapolis
• MPI-0documentservedasacatalyst• MarcSnirofIBMemailedmetosay‘hewashappytohavebeenplagiarized’
• IhavenoideawhyweleZtheobviouscollec,vecommunica,onsrou,nesoutofMPI-0
• RustyLuskandBillGroppfromArgonnevolunteeredtoproduceanopensourceimplementa,onoftheevolvingMPIstandard
• AndtheEUPPPEprojectpaidforthebeer…
TheOriginsofMPI(3)
• FollowedproceduresofHPFForum– Setambi,ousgoalofagreeingastandardwithinoneyear
• Metevery6weeksinDallasairporthotel– IsentIanGlendinningfrommygroupinSouthamptonfundedbyEUPPPEproject
• InmyopinionMPI-1succeededbecause:– Argonneproducedanopensourceimplementa,on– Excep,onaltechnicalleadershipfrompeoplelikeMarcSnirfromIBMandJimCowniefromMeiko
– Itwasneededandhadthesupportofthecommunity
TheMPIProcess
• AdventofMPImeantthatitwaspossibletoassemblesuiteofMessage-Passingbenchmarksforperformanceanalysisofmachinesandapplica,ons
• EUGenesisprojectdefined3levelsofbenchmarks– Low-level,KernelsandasetofCompactApplica,onsimplementedwithPARMACSlibraries
• Interna,onalParkbenchGroup– CombinedGenesismethodologywithLinearAlgebraandNASParallelBenchmarksimplementedwithMPI-1
Ø Butthemarke,ngcommunitypreferredJack’sTop500Benchmark…
Parkbench:PortableDMMessage-PassingKernelsandBenchmarks
• ManythankstoJackDongarra,RolfHempelandDavidWalker
• Auseful‘aidememoire’wasthear,clebyDongarra,Fagg,HempelandWalkerintheEncyclopediaofElectronicsandElectricalEngineering(Wiley)
Acknowledgements
JackDongarra,RolfHempel,TonyHey,DavidWalker‘ADraZStandardforMessagePassingonDistributedMemoryComputers’ProceedingsoftheFiZhECMWFWorkshopontheUseofParallelProcessorsinMeteorology:“ParallelSupercompu,nginAtmosphericScience”Editors:Geerd-RHoffmannandTuomoKaurannePublishedbyWorldScien,fic,1993
MPI-0Reference