WRF&WPS:CompilationProcess

KellyWernerNCAR/MMM

November2017

1

InstallingSteps

• Checksystemrequirements• Installinglibraries• Downloadsourcedata• CompileWRFV3• CompileWPS• Downloadinitial/BCdatasets

2

SystemRequirements

• OnwhatkindsofsystemswillWRFrun?– Generallyany32- or64-bithardware,runningaUNIX-likeoperatingsystem

– Youmayalsousedual-bootingintoaUNIX-likeOS(e.g.,WindowswithLinuxbuiltparallel)

• Examplesofacceptablesystems:– Laptops,desktops,andclustersrunningLinux– LaptopsanddesktopsrunningMacOS X– ClustersrunningUnix-like:Linux,AIX

3

CheckSystemRequirements• Webpage:

http://www2.mmm.ucar.edu/wrf/OnLineTutorial/compilation_tutorial.php

4

CheckSystemRequirements• ItismandatorytohaveaFortran(e.g.,

gfortran)compiler,aCcompiler,andcpp onyoursystem.Totestwhethertheseexistonyoursystem,type:– whichgfortran– whichcpp– whichgcc– Ifinstalled,youwillbegivenapathfor

each

• Fortrancompilershouldbeversion4.4.0,orlaterCheckthisbytyping(forcsh):

gcc -–version

• Testsavailableforcheckingthatyourfortran compilerisbuiltproperly,andthatitiscompatiblewiththeCcompiler.

5

AdditionalNecessaryRequirements

• Scriptinglanguages(testingavailableintestpackage):cshperlsh

• UNIXcommands:ar head sed awkhostname sleep cat lnsort cd ls tarcp make touch cutmkdir tr expr mvuname file nm wcgrep printf which gziprm

6

InstallingSteps

• Checksystemrequirements• Installinglibraries• Downloadsourcedata• CompileWRFV3• CompileWPS• Downloadinitial/BCdatasets

7

InstallingLibraries

• NetCDF (neededbyWRFandWPS)– netCDF Version3or4areacceptable– IfusingnetCDF4capabilities

(http://www2.mmm.ucar.edu/wrf/users/wrfv3.5/building_netcdf4.html)

• OptionallibrariesforGRIB2meteorologicaldatasupport– JasPer(JPEG2000“lossy”compressionlibrary)– PNG(“lossless”compressionlibrary)– Zlib(compressionlibraryusedbyPNG)

• OptionalMPIlibrary(forbuildinginparallel):– MPICH2

8

InstallingLibraries

• Installationoftheselibraries(MPICH2,NetCDF,JasPer,zlib,andlibpng)isNOTpartoftheWPSandWRFinstallationscripts

• VERYIMPORTANT!– Makesuretheselibrariesareinstalledusingthesame

compilersaswillbeusedtoinstallWRFandWPS

• Downloadsforthelibraries,withinstallationinstructions,andlibrarycompatibilitytestsarealsoincludedonthecompilationwebsite

9

InstallingLibraries:NetCDFsetenv DIR directory-where-your-tar-files-aresetenv CC gccsetenv CXX g++setenv FC gfortransetenv FCFLAGS –m64 # FCFLAGS may be needed on some systemssetenv F77 gfortransetenv FFLAGS –m64 # FFLAGS may be needed on some systems

tar xzvf netcdf-4.1.3.tar.gz # no ‘.gz’ if downloaded to most Macscd netcdf-4.1.3./configure –-prefix=$DIR/netcdf –-disable-dap --disable-netcdf-4 -–disable-sharedmakemake installsetenv PATH $DIR/netcdf/bin:$PATHsetenv NETCDF $DIR/netcdfcd ..

10

InstallingLibraries:MPICH2• Inprinciple,anyimplementationoftheMPI-2standardshould

workwithWRF;however,wehavethemostexperiencewithMPICH

• AssumingenvironmentvariablesfornetCDF installarealreadyset:

tar xzvf mpich-3.0.4.tar.gz # no ‘.gz’ if downloaded to most Macscd mpich-3.0.4./configure –-prefix=$DIR/mpichmakemake installsetenv PATH $DIR/mpich/bin:$PATHcd ..

11

InstallingLibraries:zlib• AssumingenvironmentvariablesfromnetCDF installare

alreadyset:

tar xzvf zlib-1.2.7.tar.gz # no ‘.gz’ if downloaded to most Macscd zlib-1.2.7./configure –-prefix=$DIR/zlibmakemake installcd ..

12

InstallingLibraries:libpng• AssumingenvironmentvariablesfromnetCDF installare

alreadyset

tar xzvf libpng-1.2.50.tar.gz # no ‘.gz’ if downloaded to most Macscd libpng-1.2.50./configure –-prefix=$DIR/libpngmakemake installcd ..

13

InstallingLibraries:JasPer• AssumingenvironmentvariablesfromnetCDF installare

alreadyset

tar xzvf jasper-1.900.1.tar.gz # no ‘.gz’ if downloaded to most Macscd jasper-1.900.1./configure –-prefix=$DIR/jasper makemake installcd ..

14

InstallingLibraries:Compatibility

• Makesurelibrariesarecompatiblewithcompilers

• Test1– Fortran+C+netCDF

• Test2– Fortran+C+netCDF +

MPI

15

InstallingSteps

• Checksystemrequirements• Installinglibraries• Downloadsourcedata• CompileWRFV3• CompileWPS• Downloadinitial/BCdatasets

16

DownloadWRF&WPSCode• DownloadWRF&WPSsourcecodefrom:

http://www2.mmm.ucar.edu/wrf/users/download/get_source.html– Click‘NewUser,’registeranddownload,or– Click‘ReturningUser,’enteryouremail,anddownload

17

Step1:Clickhereforthelatest

releasedcode(recommended)

Step2:Clickontarfilesto

download

DownloadStaticGeographicalData• FromtheWRFDownloadpage:

http://www2.mmm.ucar.edu/wrf/users/download/get_sources.html

18

Step1:Click‘WPS’box

Step2:Click‘here’togetgeographydata

DownloadStaticGeographicalData• GeographicalInputandDataDownloadPage:http://www2.mmm.ucar.edu/wrf/users/download/get_sources_wps_geog.html

19

geog.tar.gz~15GBwhenuncompressed

Thisistheoneyouwant

InstallingSteps

• Checksystemrequirements• Installinglibraries• Downloadsourcedata• CompileWRFV3• CompileWPS• Downloadinitial/BCdatasets

20

ChoosingaCompiler

21

Compiler CompileTime RunTime

GNU4.8.2**FREE**

12.63Mins 4.18Mins

Intel12.1.5 27.75Mins 3.88Mins

PGI13.3-0 24.86Mins 4.25Mins

*Compile:dmpar/nesting,nolarge-filesupport

*Run:singledomain,smalldomain(74x61),6hours,16processors

Step1:ConfigureforWRFV3• InsidetheWRFV3/directory,type:./configurechecking for perl5... nochecking for perl... found /usr/bin/perl (perl)Will use NETCDF in dir: /usr/local/netcdfPHDF5 not set in environment. Will configure WRF for use without.configure: WRF operating system set to "Linux" via environment variable $WRF_OSconfigure: WRF machine set to "i686" via environment variable $WRF_MACH$JASPERLIB or $JASPERINC not found in environment, configuring to build without grib2

I/O...------------------------------------------------------------------------Please select from among the following supported platforms.

1. Linux i486 i586 i686, gfortran compiler with gcc (serial)2. Linux i486 i586 i686, gfortran compiler with gcc (smpar)3. Linux i486 i586 i686, gfortran compiler with gcc (dmpar)4. Linux i486 i586 i686, gfortran compiler with gcc (dm+sm)5. Linux i486 i586 i686, g95 compiler with gcc (serial)6. Linux i486 i586 i686, g95 compiler with gcc (dmpar)7. Linux i486 i586 i686, PGI compiler with gcc (serial)8. Linux i486 i586 i686, PGI compiler with gcc (smpar)9. Linux i486 i586 i686, PGI compiler with gcc (dmpar)10. Linux i486 i586 i686, PGI compiler with gcc (dm+sm)11. Linux x86_64 i486 i586 i686, ifort compiler with icc (serial)12. Linux x86_64 i486 i586 i686, ifort compiler with icc (smpar)13. Linux x86_64 i486 i586 i686, ifort compiler with icc (dmpar)14. Linux x86_64 i486 i586 i686, ifort compiler with icc (dm+sm)

Enter selection [1-16] :

Compile for nesting? (1=basic, 2=present moves, 3=vortex following) [default 1]:

• Outputfromconfiguration:afilecalled‘configure.wrf’

22

ConfigureOptionsforWRFV3

DebuggingOptions

• ./configure –d– Nooptimization– Extradebugging

• ./configure –D– Nooptimization– Checksuninitializedvariables,

floatingpointtraps,etc.– Usefulforadding/updatingnew

code

• ./configure –r8– DoubleprecisionforIntel,GNU,and

PGI

LargeFileSupport• setenv WRFIO_NCD_LARGE_FILE_SUPPORT 1

- >2GB- Beforeconfiguring- Built-insinceV3.9

- ./configure -hyb

23

HybridCoordinateOption

ParallelCompileOptionforWRFV3

• TobuildWRFinparallel– setenv J“-j2”

24

#ofProcessors TimetoCompiler1 22.8 Mins

2 14.92Mins

3 9.33Mins

4 8.02Mins

5 7.23Mins

6 6.68 Mins

*Around4processors,itreachesstateofequilibrium

*ThistestdonewithGNUcompiler

configure.wrf File:UsefulTips

• NETCDFPATH:internallysetbybuildsystembasedon$NETCDF

• PNETCDF=Foruserswhohaveaccesstoparallelnetcdf,usetheenvironmentvariablePNETCDFidenticallytohowNETCDFisset(pointtothePNETCDFtop-leveldirectory)

25

Step2:CompileWRFV3• IntheWRFV3/directory,type:

./compile em_case >& log.compile

Whereem_case isoneofthefollowing(type./compile toseealloptions)

em_real (3drealcase) em_hill2d_xem_quarter_ss em_squall2d_xem_b_wave em_squall2d_yem_les em_grav2d_xem_heldsuarez em_seabreeze2d_xem_tropical_cycloneem_convrad em_scm_xy (1dideal)

**Compilationshouldtake~30mins**

26

3dIdeal

2dIdeal

Importantincasetherearecompileproblems

SuccessfulCompilation• Ifthecompilationissuccessful,youshouldfindtheseexecutables

inWRFV3/main (non-zerosize):

Realdatacase:

wrf.exe - modelexecutablereal.exe – realdatainitializationndown.exe – one-waynestingtc.exe – fortcbogusing(serialonly)

Idealcase:

wrf.exe – modelexecutableideal.exe – idealcaseinitialization

*Note:Eachidealcasecompilecreatesadifferentexecutable,butwiththesamename

• Theseexecutablesarelinkedto2differentdirectories(WRFV3/run andWRFV3/test/em_real).YoucangotoeitherplacetorunWRF.

27

UnsuccessfulCompilation

• Useyour‘log.compile’filetosearchforerrors!– Searchfor‘Error’withacapital‘E’

• UseourFrequentlyAskedQuestionswebpageforhelp– www2.mmm.ucar.edu/wrf/users/FAQ_files/FAQ_wrf_intallation.html

• Beforerecompiling:– issuea‘clean–a’– Reconfigure:Ifyouneedtomakechangestotheconfigure.wrf file,do

thisafterissuing./configure,andthensavetheeditedfile.– Recompile

• Contactwrfhelp@ucar.edu

28

InstallingSteps

• Checksystemrequirements• Installinglibraries• Downloadsourcedata• CompileWRFV3• CompileWPS• Downloadinitial/BCdatasets

29

Step1:ConfigureforWPS

30

• InsidetheWPS/directory,type:./configure

• ChoosetocompileWPSserially,evenifyoucompileWRFV3inparallel(unlessyouhaveaverylargedomain)**NOTE:ifyoudocompileWPSinparallel,ungrib.exe mustrunserially

• Outputfromconfiguration:afilecalled‘configure.wps’

Step2:CompileWPS

• IntheWPS/directory,type:./compile >& log.compile

• Compilationshouldonlytakeafewminutes

• Ifsuccessful,theseexecutablesshouldbeinyourWPS/directory(andtheyarelinked,respectively,fromtheirsourcecodedirectories):

geogrid.exe -> geogrid/src/geogrid.exe

ungrib.exe -> ungrib/src/ungrib.exemetgrid.exe -> metgrid/src/metgrid.exe

31

UnsuccessfulWPSCompilation

32

Nogeogrid.exe ormetgrid.exe

– WPSmakesuseoftheexternalI/OlibrariesintheWRFV3/external/directory-ThelibrariesarebuiltwhenWRFisinstalled

– Checkthatyouusedtheexactsamecompiler(andversion)asyouusedtocompileWRFV3

– CheckthatyouareusingthesamenetCDF thatyouusedtobuildWRFV3

– HaveyouchangedthenameorpathoftheWRFV3/directory?• Ifso,youneedtochangethefollowinglineintheconfigure.wps file:

WRF_DIR=../WRFV3• Savethefileandrecompile

UnsuccessfulWPSCompilation

33

Noungrib.exe

– Makesureyouhaveinstalledyourjasper,zlib,andlibpng librariescorrectly.

– Makesurethatyouareusingthecorrectpathandformatforthefollowinglinesintheconfigure.wps fileCOMPRESSION_LIBS=-L/${DIR}/UNGRIB_LIBRARIES/lib–ljasper –lpng –lzCOMPRESSION_INC=-I/${DIR}/UNGRIB_LIBRARIES/include

Saveconfigure.wps andrecompile

./clean-a

• The‘./clean–a’commandissomethingthatshouldbeusedwhenyouhavemadecorrectionstoyourconfigure.wrf file,configure.wpsfile,oranychangestotheregistry.Ifyouhavemadeanyofthesechanges,orifyouplantorecompileyourcodefromscratch,youmustissuea‘clean–a’beforerecompiling.

• Ifyoumadeanychangestoanysubroutineswithinthecode,youwillneedtorecompileyourcode,butyoudoNOTneedtoissuethe‘clean–a’command,nordoyouneedtoreconfigure.Youwillsimplyjustrecompile.Thiscompilationshouldtakealotlesstimethanacleancompile.

34

InstallingSteps

• Checksystemrequirements• Installinglibraries• Downloadsourcedata• CompileWRFV3• CompileWPS• Downloadinitial/BCdatasets

35

DownloadDatasets• FromtheWRFUsers’page: http://www2.mmm.ucar.edu/wrf/users/

36

Step1: ClickDownload,thenscrolldownandclick‘InputDatafromNCAR’

Step2: Clickthedatasetyouwishtouse(forthisexample,wewilluse‘FNLfromGFS’)

*Note:TheNOMADSsitehasseveraltypesofusefuldata:http://nomads.ncdc.noaa.gov

DownloadDatasets(continued)

37

Step3: Register,orsignin,ifyoualreadyhaveanaccount

Step5: Click‘WebFileListing’forthespanofyearsyouneed

Step4: Click‘DataAccess’

DownloadDatasets(continued)

38

Step6: Click‘CompleteFileList’

Step7: Clicktheyearyouneed.Afterthis,Youwillclickthemonthyouneed(notshown)

DownloadDatasets(continued)

39

Step8: Clickaboxforeachtimespanthatyouneed

Step9: OnceyouhavechosenAllyourtimes,clickonthe‘ViewSelectedFiles/GetAsaTarFile’buttonTodownloadonetarfilewithallyourDates/times

Questions?

40