leesa brieger, jason coposky, vijay dantuluri, kevin ... · • git – distributed revision...

iRODSUserMee,ngFebruary17‐18,2011

LeesaBrieger,JasonCoposky,VijayDantuluri,KevinGamiel,RayIdaszak,

OlegKapeljushnik,NassibNassar,JasonReilly,MichaelStealey,LisaStillwell

iRODS@RENCI

1


irods@renci

•  AnewinitiativeattheRenaissanceComputingInstitute(RENCI),aresearchunitofUNC

•  AninvestmentbyUNC•  Astep‐upofthecollaborationwithDICE,alreadyadministrativelytiedtoRENCI:

  DICE‐UCSD:InstituteofNeuralComputing(INC)  DICE‐UNC:RENCIandtheSchoolofInformationandLibraryScience(SILS)

•  Steppingtowardlong‐termsustainability

2


AgileDevelopmentApproach

•  Incrementalanditerativemethodology•  Shortdevelopmentcycles(1‐4weeks)•  Wholeteamworksthroughafulldevcycle:

o  planning‐stakeholderneedstakenintoaccounto  requirementsanalysiso  designo  codingo  unitandacceptancetestingo  productdemonstratedtostakeholders

•  Iteratefornextdevcycle,adaptingtonewrequirements,technologicalconstraints,etc

3


Agile:NotaWaterfallModel

•  WaterfallModel:goodformanufacturing,notforsoftwaredev

•  Agile:requirementsandsolutionsevolveinshort‐cyclecollaborationbetweenself‐organizing,cross‐functionalteams

•  iRODS:defactoAgiledevelopmentdrivenbycommunitystakeholders;couldneverhaveevolvedinawaterfallmodel

•  RENCI:formalizeanAgileapproachiniRODSdevelopmentforhardening,communityparticipation,andsustainability

4


CollaborativeDevelopmentEnvironment

•  Git–distributedrevisioncontrolsystem•  GForge–projectandsoftwaredevelopmentmanagementsystem:o  hosting&versioncontrolo  bug‐trackingo  messaging

•  Hudson–continuousintegrationenvironment:incrementalqualitycontrol

•  Nexus–Mavenrepositorythattracksdependenciesandbundlesforcheck‐out(Java)

5


InfrastructureOverviewSupportscommunity‐basedsoDwaredevelopment

6


CollaborativeDevEnvironment

•  StartingoutwithJargontotesttheinfrastructure

•  OtheriRODSclientshosted:PHP,Windows,Python,etc

•  AfulliRODSmirrorwillresideinthisenvironment

•  ProvideautomatedcontinuousbuildandtestforiRODSserverandclients

7


GForgeAcademiclicense:nolimiton#ofusers

Communitybasedplug‐ins:

https://code.renci.org/gf:

FileReleaseSystemMailinglistsWikiContinuousBuild

ForumsTrackersDocumentManagersNews

8


GForgeProjectsinRENCIgforge

https://code.renci.org/gf/project/

9


GForge

TrackingFeatureRequestsinJargonTrunk

10


GForgeTrackingProjectActivityinJargonTrunk

11


GForgeiDropTrackerReportActivity

12


GForgeProjectSummaryforiRODSPHP

13


Hudson

Buildautomation:

•  Continuousbuild/testofsoftwareprojectso  easierfordeveloperstointegrate

changestotheprojecto  easierforuserstoobtainafresh

build

•  Notificationoffailurestohelpkeepsystemshealthy

•  Monitorexecutionsofexternally‐runjobso  cronjobsandprocmailjobs,even

runningremotelyo  Hudsonkeepsjoboutputsand

makesiteasytonoticewhensomethingiswrong

hNps://ci‐dev.renci.org/hudson:

14


SonatypeNexus

Nexus‐Mavenartifact repository

•  Managessoftwareartifactsfordevelopment,deployment,andprovisioning

•  Allowssharingofartifactswithotherdevelopersandend‐users

•  Centralizedcontrolofaccessanddeploymentofartifacts

hNps://ci‐dev.renci.org/nexus/

15


TowardaUnifiedCross‐PlatformCode

•  Migrateplatform‐specificAPIsandsystemcallsawayfromserver‐levelcode

•  Providestrategyforfuturesupportonotherplatforms

•  Movetog++foraccesstolibrariessuchasBoostC++librariesTargetedAPIsandFunc,ons:•  Threading•  RegularExpressions•  CharacterEncoding•  Signals•  Fork

CurrentStatus:•  porttog++isdoneand

buildispassingdevtest•  Movetoboost::threadis

almostcompleteandalsopassingdevtest

16


WindowsSupport

•  Updateofnon‐iCATiRODSserver

•  ImplementationofiCAT‐enablediRODSserver:o  IntegrationofMSSQLServerwithiRODSo  BuildsofsupportediCATDBsforWindows

•  SupportofWindowsiExplorerclient

17


iRODS.NETClient•  ConnecttoiRODSserverfrom.NETplatform

o  iRods.NETClientwillperformmostoftheiRODSclientoperations

o  Nativeintegrationwith.NETFrameworko  Supports.NET3.5andup

•  UsagescenariosforiRODS.NETCliento  LinqToIRodsdevelopmenttoqueryiRODSservero  Powershellcommandstosimulateicommandso  DriveorfoldermountedtoiRODSo  .NETwebandwindowsapplicationdevelopment

18


DatabaseActivities•  DatabaseResourcesTesting

o  DBRinteractionwithdatabaseinstances:MySQL,Oracle,PostgresonUbuntu10.10  LocaltoiRODSserver  RemotetoiRODSserver

•  DevelopingMSSQLServerInterfacetosupportiCATWindowsImplementation

•  iCATdataredundancyandfailovermechanisms•  iCATSpecialQueryUsageandApplicabilitywith

examples•  iCATdatabaseperformancetuningand

recommendedenhancements

19


JavaRuleEngine

•  AnalyzedesignandresourcerequirementsforaJava‐basedruleengineo  Determinebestinter‐processcommunicationmethodbetween(next‐gen)CengineandJava

o  SemanticsynchronizationwithCengineo  IntegrationwithJargon

•  Analysiscompletebysummer2011

20


iRODSClients:PHPAPI&WebBrowser

iRODS@RENCIStart‐up:

•  AssistDICEwithongoingsupport•  MonitorChatdiscussion(alreadyfixed1issue!)•  Pullmostcomplete&up‐to‐dateversionsfromDICE/GoogleCode/

Communityandmerge•  PutinGForge,Gitrepository.•  IncorporatePHPUnitandHudsoncontinuousintegration

21


PHPAPI&WebBrowser

CommunityInvolvement:

•  WhoisusingPHPAPI/WebBrowser?•  Whoiscontributingtosource?•  Whowouldliketohelpmaintain?•  Whatnewfeaturesareneeded?

22


PHPAPI&WebBrowser

Long‐TermPlanning:

•  Determinenewfeatureset•  Investigateimplementationtechnologies(i.e.PHPinJAVA

VM)•  StandardizationamongclientAPIs(i.e.iRODSServer/Client

compatibilityversioning,andstandardizefunction&parameternaming

•  Hardening

23


SpecialProjects

•  NCDCusecases•  Hadoopdriver‐genomics

•  TUCASIproject–federatedenvironmentforthe3Triangleuniversities(Duke,NCSU,UNC)

•  ShibbolethAuthentication–mapexternalauthenticatedusertoiRODSuser(TUCASIproject)

•  NARA–CyberinfrastructureforBillionsofRecords•  NCBioPreparedness–distributeddataaccess

24


Sustainability–theCommunityArchitectureModel

•  SupportandformalizethecollaborativemodelforiRODSo  Collaborativedevelopmentenvironmento  Agiledevelopmentapproach

•  RedHatFedoramodel(FedoraLinux,notFedoraDigitalRepository)

•  Developservicelevelagreementsfortargetedusergroups

25


irods@renci BuildingaRedHatFedoraModel

3‐4monthreleaseschedule;

currentusersupportmodel

iRODScommunity

code

iRODShardenedcode

18‐monthreleaseschedule;

commercialsupportmodel

FundedDevelopment

DICE‐driven irods@renci‐driven

CommunityContribu,ons

andRequests

26


DifferencesBetweenCommunityandHardenedReleases

CommunityCode HardenedCode

Users TechnicalenthusiastsusingiRODSinnon‐criticalcomputingenvironments

Userslookingforstable,supported,andcertifiediRODS(business,government,etc)

PrimaryBenefits Bleeding‐edgetechnologyreleasedearlyandoften.

Stable,reliable,andbroadlysupported.Easytodeployandmanage.Manycertifiedapplicationsavailable.

FeatureSelection DICEanddevelopercommunity

DICEandirods@renci

DevelopmentModel OpenSource OpenSource

27


CommunityCode HardenedCode

Certifications None Hardware,software

SupportOptions None(communitysupported)

Many,including24x7with1hourresponse.Unlimitedincidents.Includeupgrades

Maintenance&Updates Community&3rdpartydriven

Completeupdateandmanagement

Testers DeveloperCommunity irods@renci,DICE,partners,betateam

Price Freedownload Annualsubscription,multipleofferings

CommunityReleasevs.HardenedRelease

28


ServiceLevelAgreements

Theextensionofthecommunitycode

• hardenedcodebase• certifications• specializedservicesfortargetgroups

willleadtoservicelevelagreementstailoredtotargetuserneeds.

29

leesa brieger, jason coposky, vijay dantuluri, kevin ... · • git – distributed revision...

Documents