Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
iRODSUserMee,ngFebruary17‐18,2011
LeesaBrieger,JasonCoposky,VijayDantuluri,KevinGamiel,RayIdaszak,
OlegKapeljushnik,NassibNassar,JasonReilly,MichaelStealey,LisaStillwell
iRODS@RENCI
1
iRODSUserMee,ngFebruary17‐18,2011
irods@renci
• AnewinitiativeattheRenaissanceComputingInstitute(RENCI),aresearchunitofUNC
• AninvestmentbyUNC• Astep‐upofthecollaborationwithDICE,alreadyadministrativelytiedtoRENCI:
DICE‐UCSD:InstituteofNeuralComputing(INC) DICE‐UNC:RENCIandtheSchoolofInformationandLibraryScience(SILS)
• Steppingtowardlong‐termsustainability
2
iRODSUserMee,ngFebruary17‐18,2011
AgileDevelopmentApproach
• Incrementalanditerativemethodology• Shortdevelopmentcycles(1‐4weeks)• Wholeteamworksthroughafulldevcycle:
o planning‐stakeholderneedstakenintoaccounto requirementsanalysiso designo codingo unitandacceptancetestingo productdemonstratedtostakeholders
• Iteratefornextdevcycle,adaptingtonewrequirements,technologicalconstraints,etc
3
iRODSUserMee,ngFebruary17‐18,2011
Agile:NotaWaterfallModel
• WaterfallModel:goodformanufacturing,notforsoftwaredev
• Agile:requirementsandsolutionsevolveinshort‐cyclecollaborationbetweenself‐organizing,cross‐functionalteams
• iRODS:defactoAgiledevelopmentdrivenbycommunitystakeholders;couldneverhaveevolvedinawaterfallmodel
• RENCI:formalizeanAgileapproachiniRODSdevelopmentforhardening,communityparticipation,andsustainability
4
iRODSUserMee,ngFebruary17‐18,2011
CollaborativeDevelopmentEnvironment
• Git–distributedrevisioncontrolsystem• GForge–projectandsoftwaredevelopmentmanagementsystem:o hosting&versioncontrolo bug‐trackingo messaging
• Hudson–continuousintegrationenvironment:incrementalqualitycontrol
• Nexus–Mavenrepositorythattracksdependenciesandbundlesforcheck‐out(Java)
5
iRODSUserMee,ngFebruary17‐18,2011
InfrastructureOverviewSupportscommunity‐basedsoDwaredevelopment
6
iRODSUserMee,ngFebruary17‐18,2011
CollaborativeDevEnvironment
• StartingoutwithJargontotesttheinfrastructure
• OtheriRODSclientshosted:PHP,Windows,Python,etc
• AfulliRODSmirrorwillresideinthisenvironment
• ProvideautomatedcontinuousbuildandtestforiRODSserverandclients
7
iRODSUserMee,ngFebruary17‐18,2011
GForgeAcademiclicense:nolimiton#ofusers
Communitybasedplug‐ins:
https://code.renci.org/gf:
FileReleaseSystemMailinglistsWikiContinuousBuild
ForumsTrackersDocumentManagersNews
8
iRODSUserMee,ngFebruary17‐18,2011
GForgeProjectsinRENCIgforge
https://code.renci.org/gf/project/
9
iRODSUserMee,ngFebruary17‐18,2011
GForge
TrackingFeatureRequestsinJargonTrunk
10
iRODSUserMee,ngFebruary17‐18,2011
GForgeTrackingProjectActivityinJargonTrunk
11
iRODSUserMee,ngFebruary17‐18,2011
GForgeiDropTrackerReportActivity
12
iRODSUserMee,ngFebruary17‐18,2011
GForgeProjectSummaryforiRODSPHP
13
iRODSUserMee,ngFebruary17‐18,2011
Hudson
Buildautomation:
• Continuousbuild/testofsoftwareprojectso easierfordeveloperstointegrate
changestotheprojecto easierforuserstoobtainafresh
build
• Notificationoffailurestohelpkeepsystemshealthy
• Monitorexecutionsofexternally‐runjobso cronjobsandprocmailjobs,even
runningremotelyo Hudsonkeepsjoboutputsand
makesiteasytonoticewhensomethingiswrong
hNps://ci‐dev.renci.org/hudson:
14
iRODSUserMee,ngFebruary17‐18,2011
SonatypeNexus
Nexus‐Mavenartifact repository
• Managessoftwareartifactsfordevelopment,deployment,andprovisioning
• Allowssharingofartifactswithotherdevelopersandend‐users
• Centralizedcontrolofaccessanddeploymentofartifacts
hNps://ci‐dev.renci.org/nexus/
15
iRODSUserMee,ngFebruary17‐18,2011
TowardaUnifiedCross‐PlatformCode
• Migrateplatform‐specificAPIsandsystemcallsawayfromserver‐levelcode
• Providestrategyforfuturesupportonotherplatforms
• Movetog++foraccesstolibrariessuchasBoostC++librariesTargetedAPIsandFunc,ons:• Threading• RegularExpressions• CharacterEncoding• Signals• Fork
CurrentStatus:• porttog++isdoneand
buildispassingdevtest• Movetoboost::threadis
almostcompleteandalsopassingdevtest
16
iRODSUserMee,ngFebruary17‐18,2011
WindowsSupport
• Updateofnon‐iCATiRODSserver
• ImplementationofiCAT‐enablediRODSserver:o IntegrationofMSSQLServerwithiRODSo BuildsofsupportediCATDBsforWindows
• SupportofWindowsiExplorerclient
17
iRODSUserMee,ngFebruary17‐18,2011
iRODS.NETClient• ConnecttoiRODSserverfrom.NETplatform
o iRods.NETClientwillperformmostoftheiRODSclientoperations
o Nativeintegrationwith.NETFrameworko Supports.NET3.5andup
• UsagescenariosforiRODS.NETCliento LinqToIRodsdevelopmenttoqueryiRODSservero Powershellcommandstosimulateicommandso DriveorfoldermountedtoiRODSo .NETwebandwindowsapplicationdevelopment
18
iRODSUserMee,ngFebruary17‐18,2011
DatabaseActivities• DatabaseResourcesTesting
o DBRinteractionwithdatabaseinstances:MySQL,Oracle,PostgresonUbuntu10.10 LocaltoiRODSserver RemotetoiRODSserver
• DevelopingMSSQLServerInterfacetosupportiCATWindowsImplementation
• iCATdataredundancyandfailovermechanisms• iCATSpecialQueryUsageandApplicabilitywith
examples• iCATdatabaseperformancetuningand
recommendedenhancements
19
iRODSUserMee,ngFebruary17‐18,2011
JavaRuleEngine
• AnalyzedesignandresourcerequirementsforaJava‐basedruleengineo Determinebestinter‐processcommunicationmethodbetween(next‐gen)CengineandJava
o SemanticsynchronizationwithCengineo IntegrationwithJargon
• Analysiscompletebysummer2011
20
iRODSUserMee,ngFebruary17‐18,2011
iRODSClients:PHPAPI&WebBrowser
iRODS@RENCIStart‐up:
• AssistDICEwithongoingsupport• MonitorChatdiscussion(alreadyfixed1issue!)• Pullmostcomplete&up‐to‐dateversionsfromDICE/GoogleCode/
Communityandmerge• PutinGForge,Gitrepository.• IncorporatePHPUnitandHudsoncontinuousintegration
21
iRODSUserMee,ngFebruary17‐18,2011
PHPAPI&WebBrowser
CommunityInvolvement:
• WhoisusingPHPAPI/WebBrowser?• Whoiscontributingtosource?• Whowouldliketohelpmaintain?• Whatnewfeaturesareneeded?
22
iRODSUserMee,ngFebruary17‐18,2011
PHPAPI&WebBrowser
Long‐TermPlanning:
• Determinenewfeatureset• Investigateimplementationtechnologies(i.e.PHPinJAVA
VM)• StandardizationamongclientAPIs(i.e.iRODSServer/Client
compatibilityversioning,andstandardizefunction¶meternaming
• Hardening
23
iRODSUserMee,ngFebruary17‐18,2011
SpecialProjects
• NCDCusecases• Hadoopdriver‐genomics
• TUCASIproject–federatedenvironmentforthe3Triangleuniversities(Duke,NCSU,UNC)
• ShibbolethAuthentication–mapexternalauthenticatedusertoiRODSuser(TUCASIproject)
• NARA–CyberinfrastructureforBillionsofRecords• NCBioPreparedness–distributeddataaccess
24
iRODSUserMee,ngFebruary17‐18,2011
Sustainability–theCommunityArchitectureModel
• SupportandformalizethecollaborativemodelforiRODSo Collaborativedevelopmentenvironmento Agiledevelopmentapproach
• RedHatFedoramodel(FedoraLinux,notFedoraDigitalRepository)
• Developservicelevelagreementsfortargetedusergroups
25
iRODSUserMee,ngFebruary17‐18,2011
irods@renci BuildingaRedHatFedoraModel
3‐4monthreleaseschedule;
currentusersupportmodel
iRODScommunity
code
iRODShardenedcode
18‐monthreleaseschedule;
commercialsupportmodel
FundedDevelopment
DICE‐driven irods@renci‐driven
CommunityContribu,ons
andRequests
26
iRODSUserMee,ngFebruary17‐18,2011
DifferencesBetweenCommunityandHardenedReleases
CommunityCode HardenedCode
Users TechnicalenthusiastsusingiRODSinnon‐criticalcomputingenvironments
Userslookingforstable,supported,andcertifiediRODS(business,government,etc)
PrimaryBenefits Bleeding‐edgetechnologyreleasedearlyandoften.
Stable,reliable,andbroadlysupported.Easytodeployandmanage.Manycertifiedapplicationsavailable.
FeatureSelection DICEanddevelopercommunity
DICEandirods@renci
DevelopmentModel OpenSource OpenSource
27
iRODSUserMee,ngFebruary17‐18,2011
CommunityCode HardenedCode
Certifications None Hardware,software
SupportOptions None(communitysupported)
Many,including24x7with1hourresponse.Unlimitedincidents.Includeupgrades
Maintenance&Updates Community&3rdpartydriven
Completeupdateandmanagement
Testers DeveloperCommunity irods@renci,DICE,partners,betateam
Price Freedownload Annualsubscription,multipleofferings
CommunityReleasevs.HardenedRelease
28
iRODSUserMee,ngFebruary17‐18,2011
ServiceLevelAgreements
Theextensionofthecommunitycode
• hardenedcodebase• certifications• specializedservicesfortargetgroups
willleadtoservicelevelagreementstailoredtotargetuserneeds.
29