1. PERFORMANCEMODELINGByOYEWOLE,MopelolaO.Matno:0908050542. MODELLINGOFCOMPUTERSYSTEMSNETWORKSbyKesaOluwafunmilola

ElizabethMaTNO:0908050263. ModellingComputerSystemsNetworksbyOkoroUgochukwuChristian

Matno:0908050434. DISCRETEEVENTSIMULATIONbyOSINUGAOLUWASEUN

Matno:0908050495. ANALYSISOFSIMULATIONOUTPUTByMATTHEWOMOLABAKEO.

Matno:0908050286. VERIFICATIONANDVALIDATIONOFSIMULATIONMODELSbyAMUDA,

TosinJosephMatno:0908050097. VRIFICATIONANDVALIDATIONOFSIMULATIONMODLSBYSUN

FAPOHUNDAMatno:1008050328. ANALYSISOFSINGLESERVERQUEUEANDQUEUINGNETWORKby

ADIBIOLOGUNFUNKEOLUWASEUNMATRICNO:0908050059. ANALYSISOFASINGLESERVERQUEUEANDQUEUENETWORKSby

ADEYEMIMONSURATADEOLAMATRICNO:100805008

CSC524:DISCRETE EVENT SIMULATION

OSINUGA OLUWASEUN 090805049

LECTURER:Dr.Adewole

DiscreteEventSimulationFirstly, lets define the following terminologies before delving into what discrete eventsimulationisallabout.DiscreteSystem:Adiscretesystemisasystemwithacountablenumberofstateswhichchangeinstantaneouslyatseparatedpointintime.Forexample,thecustomerservicesysteminabankisdiscretesincestate variables (number of customers) change only when a customer arrives or when acustomerfinishesbeingservedanddeparts.Event:An event is an occurrencewhich is instantaneous andmay change the state of thesystem. Simulation:Computersimulationisthedisciplineofdesigningamodelofanactualortheoreticalphysicalsystem, conducting experiments with the model, executing the model on a computer, andanalyzingtheexecutionoutputforthepurposeeitherofunderstandingthebehaviourofthesystemorofevaluatingvariousstrategies(withinthe limits imposedbyacriterionorsetofcriteria)fortheoperationofasystem. WhatthenisDiscreteEventSimulation(DES)?Adiscreteeventsimulation(DES),modelstheoperationofasystemasadiscretesequenceofeventsintime.Eacheventoccursataparticularinstantintimeandmarksachangeofstateinthesystem.Betweenconsecutiveevents,nochangeinthesystemisassumedtooccur;thusthesimulationcandirectlyjumpintimefromoneeventtothenext. Discreteeventsimulationutilizesamathematical/logicalmodeltomodelaphysicalsystemthatportraysstatechangesatprecisepointsinsimulatedtime.A(DES)modelsasystemwhosestatemaychangeonlyatdiscretepointintime. AdvantagesofSimulation1.Simulationhelpsustostudynewdesignswithoutinterruptingrealsystem.2.Simulationhelpsustostudynewdesignswithoutneedingextraresources3.Simulationislessdangerous/expensive/intrusive.4.Simulationhelpsustoimprovetheunderstandingofthesystem.5.Simulationhelpsustomanipulatetime.SimulationisperformedusingModels.

MODELAmodelinscienceisanythingusedasarepresentationofanobject,law,theoryoreventusedasatoolforunderstandingthescienceworld. Therepresentationscaneitherbea.Algorithmic(sequenceofsteps)b.Mathematical(equations)orc.Logical(conditions)Therearetwomaintechniquesforbuildingmodels:a.Abstraction:meansleavingoutunnecessarydetails.Itrepresentsonlyselectedattributesofacustomer. b.Idealisation:meansreplacingrealthingsbyconcepts.Itreplacesasetofmeasurementsbyafunction. Wemodelbecausetheyareoftencheaper,easier,faster,and/orsafertobuildandexperimentonthantheactualsystem.

MODELTAXONOMY

MODELLINGASYSTEM

*VALIDATION:Thisistheprocessofensuringthattherightmodeliscreated. *VERIFICATION:Thisistheactofwritingacorrectprogrami.ebuildingthemodelrightly

ComponentsofaDiscreteEventSimulationModel

1.Systemstate:Thisisthecollectionofstatevariablesnecessarytodescribethesystemataparticulartime. 2.Simulationclock:Thisclockisavariablethatshowsthecurrentvalueofsimulatedtime. 3.Eventlist:Aneventlistcontainsthenexttimewheneachtypeofeventwilloccur.Itcontainsallscheduledevents,arrangedinchronologicaltimeorder

4.Statisticalcounters:Thesearevariablesusedforstoringstatisticalinformationaboutsysteminformation. 5.Initializationroutine:Thisroutineisusedtoinitializethesimulationmodelattime0 6.Timingroutine:Thisroutinedeterminesthenexteventfromtheeventlistandthenadvancesthesimulationclocktothetimewhenthateventistooccur 7.Reportgenerator:Thisisasubprogramthatcomputesestimates(fromthestatisticalcounters)ofthedesiredmeasuresofperformanceandproducesareportwhenthesimulationends. 8.Eventroutine:Thisisasubprogramthatupdatesthesystemstatewhenaparticulartypeofeventoccur.Thereisoneeventroutineforeacheventtype. 9.Libraryroutines:Asetofsubprogramusedtogeneraterandomobservationsfromprobabilitydistributionsthatweredeterminedaspartofthesimulationmodel. 10.Mainprogram:Thisisasubprogramthatinvokesthetimingroutine.Itdeterminesthenexteventandtransferscontroltothecorrespondingeventroutineaswellasupdatethesystemstateappropriatelyandfinallyinvokesthereportgeneratorwhenthesimulationisover.

FLOWCHARTFORANEVENTSIMULATIONMODEL

ASIMULATIONCLASSIC

TheSingleServerQueue

a.Problemformulation: Customerswaittoolonginmybankb.Objectives: Determinetheeffectofanadditionalcashieronthemeanqueuelengthc.Dataneeded: I.Inter-arrivaltimesofcustomers ii.Servicetimesd.Entities:customers;server e.Attributesofacustomer:servicerequired f.Attributesofserver:serversskill(itsservicerate) g.Events:arrivalofacustomer;departureofacustomer h.Activities:servingacustomer,waitingforanewcustomer

TheEventList

The(future)eventlist(FEL)controlsthesimulation.TheFELcontainsallfutureeventsthatarescheduled.TheFELisorderedbyincreasingtimeofeventnotice.ExampleFEL(atsomesimulationtimet1):

ConditionalandPrimaryEvents

AprimaryeventisaneventwhoseoccurrenceisscheduledatacertaintimeE.g.Thearrivalsofcustomers.AconditionaleventisaneventwhichistriggeredbyacertainconditionbecomingtrueE.g.Acustomermovingfromqueuetoservice.

TheEventList

Eventlistconsistsofpendingeventset.Itcontainsallscheduledevents,arrangedinchronologicaltimeorder.Inthesimulator,theeventlistisadatastructuree.g.list,tree.Example:SimulationoftheMensa: Somestatevariables: #Peopleinline1 #Peopleatmealline1&2 #Peopleatcashier1&2 #PeopleeatingattablesOperationsthatcanbeperformedonaneventlist:a.InsertaneventintoFEL(atappropriateposition!)b.RemovefirsteventfromFELforprocessingc.DeleteaneventfromtheFELTheeventlistisusuallystoredasalinkedlistsuchthatwecantraverselistforwardandbackward.SIMULATINGTHEBANKMANUALLYSimulationClock:15minutes.ArrivalInterval CustomerArrives BeginService ServiceDuration ServiceComplete5 5 5 2 71 6 7 4 113 9 11 3 143 12 14 1 15

REFERENCES

1.IntroductiontoSimulation-Graham2.Simulation,Modeling&Analysis(3/e)byLawandKelton,20003.Wikipedia.org

qwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmrtyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxc

ANALYSISOFSIMULATIONOUTPUTDATA

ADEFEMIFOLAMOLUWA

MATRICNO:090805002

SIMULATIONSimulationcanbedefinedasthedesigningofaproposedorexisting

system,executingthismodelonacomputerandanalyzingtheoutputgottenfromtheexecutionofthemodel.Mosttimes,simulationiscarriedoutbecausethephysicalsystemdoesnotexist,costofbuildinganactualsystemishighorbecausemeasuringanactualsystemistimeconsuming.Inall,simulationsofsystems,proposedorexistingistoanalyzeandpredicttoagreatextent,thefunctionalityofthesystem.SIMULATIONOUTPUTSimulationalsocorrectserrorsofsystemsintendedtobebuilt.Therefore,forcorrectanalysisofthesystem,outputofthesystemsimulationhastobecorrectlyanalyzed.Iftheoutputisanalyzedwrong,thesystemwillnotbehaveasexpectedandcaninvalidateallresults. Insimulationstudies,alotoftimeandmoneyisspentonmodeldevelopmentandprogrammingbutnotsomuchisspentonanalysisofsimulationoutputinanappropriatemanner.Sometimestheoutputofasinglesimulationrunofarbitrarylengthissometimestreatedastheactualcharacteristicofthesystem.Outputsofsimulationmusthoweverberegardedasrandomassimulationsarestatisticalsamplingexperiments.Astatisticalapproachmustthereforebegiventoanalysisofoutputdata.Thiswillbedonethroughdiscreteeventcomputersimulation.Simulationexecutionsyieldestimatesofmeasuresofsystemperformanceandnotactualmeasurementvalues.Theseestimatesareerrorprone(subjecttosamplingerrors)andthisshouldbetakenintoconsiderationtomakecorrectinferencesaboutsystemperformance.Simulationoutputalmostneverproduce

rawindependent(datafromsimulationrunsaremosttimescorrelated),

identicallydistributed normaldata. Classical statistical techniques based on independent, identically

distributedtechniquesarenotappliedforcorrectsysteminferences.USINGCLASSICALSTATISTICALMETHODSTOANALYZESIMULATIONOUTPUTDATAItisbelievedthatalloutputsofsimulationsareautocorrelated.Forexample,ifthexthcustomertoarriveinabankwaitsforalongtime,itishighlypossiblethatthe(x+1)thcustomerwillwaitforalongperiodoftimealso.Simulationoutputsarealsononstationaryratherthanidenticallydistributedasitisnotpossibletochoosetheinitialconditionsforthesimulationtoberepresentativeofthetypicaloperationofthesimulatedsystem.Forthispurpose,classicalstatisticalmethodsshouldnotbeusedtoanalyzesimulationoutputdata.TYPESOFSIMULATIONSWithrespecttooutputanalysis,therearetwotypesofsimulations:FiniteHorizon(Terminating)andSteadyStatesimulations.FiniteHorizonSimulations:Theterminationofafinitehorizonsimulationtakesplaceataspecifictimeoriscausedbytheoccurrenceofaspecificevent.Examplesare:

Masstransitsystembetweenduringrushhour. Productionsystemuntilasetofmachinesbreaksdown. Startupphaseofanysystem

Steadystatesimulations:Inthistypeofsimulation,longtermbehaviorsofsystemsareanalyzed.Aperformancemeasureisthereforecalledasteadystateparameterifitisa

characteristicoftheequilibriumdistributionofanoutputstochasticprocess.Examplesare:

Continuouslyoperatingcommunicationsystemwheretheobjectiveisthecomputationofthemeandelayofapacketinthelongrun.

Distributionsystemoveralongperiodoftime.

FINITEHORIZONSIMULATIONSAsystemofinterestoverafinitetimehorizonissimulatedinthiscase.AssumeweobtaindiscretesimulationoutputY1,Y2.Ym,wherethenumberofobservations,mcanbeaconstantorarandomvariable.Example:Theexperimentercanspecifythenumber,mofcustomerwaitingtimesY1,Y2.Ym,tobetakenfromaqueuingsimulation.Ormcoulddenotetherandomnumberofcustomersobservedduringaspecifiedtimeperiod[0,T].Alternatively,wemightobservecontinuoussimulationoutput{Y(t)0tT}overaspecifiedinterval[0,T].Example:Ifweareinterestedinestimatingthetimeaveragednumberofcustomerswaitinginaqueueduring[0,T],thequantityY(t)wouldbethenumberofcustomersinthequeueattimet.Estimatetheexpectedvalueofthesamplemeanoftheobservations,E[m],STEADYSTATESIMULATIONSNowassumethatwehaveonhandstationary(steadystate)simulationoutput,Y1,Y2.Yn,Ourgoal:Estimatesomeparameterofinterest,e.g.,themeancustomerwaitingtimeortheexpectedprofitproducedbyacertainfactoryconfiguration.In

particular,supposethemeanofthisoutputistheunknownquantity.Wellusethesamplemeanntoestimate.Asinthecaseofterminatingsimulations,wemustaccompanythevalueofanypointestimatorwithameasureofitsvariance.

BibliographyGoldsman,D.(2010,May26).SIMULATIONOUTPUTANALYSIS,155.M.Law,A.(1983).OperationsResearch.StatisticalAnalysisofSimulationOutputData,148.

NAME

ADIBIOLOGUN FUNKE OLUWASEUN

MATRIC NO

090805005

COURSE CODE

CSC524

ASSIGNMENT

A WRITEUP ON ANALYSIS OF SINGLE

SERVER QUEUE AND QUEUING NETWORK

LECTURER IN CHARGE

DR ADEWOLE

ANALYSIS OF SINGLE SERVER QUEUE AND QUEUING NETWORK.

ANALYSIS OF A SINGLE QUEUE WHAT DOES QUEUE MEAN?

A queue occurs when a potential customers arrives at a system that offers certain service that the customers wish to use. A queue works almost on the same methodology used at banks or supermarkets, where the customers are treated according to their arrival.

In computer systems, many jobs share the system resources such as CPU, disks, and other devices. Since generally only one job can use the resource at any time, all other jobs wanting to use that resource wait in queues THE SINGLE SERVER QUEUE

It is a queuing model that has only one queue. This kind of model can be used to analyze individual resources in computer systems. For example if all jobs waiting for the CPU are kept in one queue, the CPU can be modeled using results that apply to single queues.

This is one of the most prevalent forms of queuing, which customers enter a system, get priority in terms of their arrival and get served by one single server. With proper enforcement, a high level of social justice will be maintained, while the higher ability of making social comparisons will be emphasized, particularly in the situations where queues are visible to the customers, such as queuing at a bus stop. The central element of the system is a server, which provides some service to the items.

Items from some population of items arrive at the system to be served. If the server is idle, an item is served immediately. Otherwise, an arriving item joins a waiting line. When the server has completed serving an item, the item departs. If there are items waiting in the queue one is immediately dispatched to the server. The server in this model can represent anything that performs some function or service for a collection of items. For example, a processor provides service to processes; an I/O device provides a read or write service for I/O requests; transmission line provides transmission service to packets of data.

Formulas of a Single Server Queue Table below provides some equations for single server queues that follow the M/G/1 model. That is, the arrival rate is Poisson and the service time is general. Making use of a scaling factor, A, the equations for some of the key output variables is straightforward. Note that the key factor in the scaling parameter is the ratio of the standard deviation of service time to the mean. No other information about the service time is needed. Two special cases are of some interest. When the standard deviation is equal to the mean, the service time distribution is exponential (M/M/1). This is the simplest case and the easiest one for calculating results. Table 3b shows the simplified versions of equations for the standard deviation of r and Tr, plus some other parameters of interest. The other interesting case is a standard deviation of service time equal to zero, that is, a constant service time (M/D/1). The corresponding equations are shown in Table 3c.

BIRTH-DEATH PROCESSES: this is used to model a system in which jobs arrive one at a time. The state of a system can be represented by number of jobs n in the system. Arrival of a new job changes to n+1. This is called a Birth. Similarly the departure of jobs changes the system state to n-1. This is called a Death. Therefore the number of jobs in such system can be modeled as a birth-death process.

SUMMARY OF SOME CLASSICAL RESULTS FOR THE SINGLE SERVER QUEUE We focus now on the case s = 1. Quantities of interest are

the arrival rate the intensity of the arrival process an (mean number of customer arrivals per second, also equal to the inverse of the mean interarrival time (Chapter 11) = S (server utilization) where S is the mean service time (Palm expectation of Sn). the residence time Rn = Dn An and waiting time Wn = Rn Sn for customer n the number of customers in the system N(t), the number of customers waiting Nw(t), given by N(t) = _ nZ 1{Ant}1{Dn>t} Nw(t) = (N(t) 1)+ STABILITY An important issue in the analysis of the single server queue is stability. In mathematical terms, it means whether N(t) is stationary. When the system is unstable, a typical behavior is that the backlog grows to infinity. The single server queue is unstable for > 1 and stable for < 1. The first part says that a necessary condition for stability is 1. We give a heuristic explanation for the necessary condition is as follows. If the system is stable, all customers eventually enter service, thus the mean number of beginnings of service per second is . From Littles law applied to the server (see Section 6.3), we have = the probability that the server is busy, which is 1. The proof of the second statement is more complex see [Baccelli88-book] for details. For = 1

Be careful that this intuitive stability result holds only for a single queue

QUEUINGNETWORKA model in which jobs departing from one queue arrive at another queue or possibly the same queue is called queuing network. It describes the system as a set of interacting resources.

A network of queues is a collection of service centers, which represent system resources, and customers, which represent users or transactions. It is a network consisting of interconnected queues. Examples

Customers go from one queue to another in post office, bank, and supermarket. Data packets traverse a network moving from a queue in a router to the queue in

another router.

Queuing network can be classified into three

Open Closed Mixed

(1) An open queuing network is the one that has external arrivals and departures. i.e. it

receive customers from an external source and send them to an external destination. The job enters the system as IN and exits as OUT. The number of jobs in the system varies with time.

In analyzing this system, we assume throughput is equal to arrival time. Analysis of open queuing network

Open queueing network models are used to represent transaction processing systems such as airline reservation systems or banking systems. In these systems, the transaction arrival rate is not dependent on the load on the computer system. The transaction arrivals are modeled as a Poisson process with a mean arrival rate .

(2) A closed queuing network has no external arrivals and departures. It has constant number of customers (finite population). They have a fixed population that moves between the queues but never leaves the queue. The jobs in the queue keep circulating from one queue to the next

The jobs exiting the system immediately reenter the system. The flow of jobs in the Out-to-In link defines the throughput of the closed system. In analyzing a closed system, we assume that the number of jobs is given, and we attempt to determine the throughput (or the job completion rate).

(3) Mixed queuing Network: are networks that are open for some workloads and closed for others. i.e it is open for some classes and closed for others.

The figure below shows an example of a mixed system with two classes of jobs. The system is closed for interactive jobs and is open for batch jobs. The term class refers to types of jobs. All jobs of a single class have the same service demands and transition probabilities. Within each class, the jobs are indistinguishable.

QueuingNetworkModelforComputerSystemsTwooftheearliestqueuingmodelsofcomputersystemsarethemachinerepairmanmodelandthecentralservermodelshowninFigures6and7,respectively.Themachinerepairmanmodel,

as the name implies, was originally developed formodelingmachine repair shops. It has anumber of working machines and a repair facility with one or more servers (repairmen).Wheneveramachinebreaksdown, it isput in thequeue for repairand servicedas soonasarepairman isavailable.Scherr(1967)usedthismodeltorepresentatimesharingsystemwithnterminals.Userssittingattheterminalsgeneraterequests(jobs)thatareservicedbythesystem,whichservesasarepairman.Afterajobisdone,itwaitsattheuserterminalforarandomthinktimeintervalbeforecyclingagain.ThecentralservermodelshowninFigure32.8wasintroducedbyBuzen(19173).TheCPUinthemodel is the central server that schedules visits to other devices. After service at the I/Odevices, the jobs return to the CPU for further processing and leave itwhen the next I/O isencounteredorwhenthejobiscompleted.

FIG6:AMachineRepairmanModel

FIG7:ACentralServerModel

TypesofServiceCentersIncomputersystemsmodeling,weencounterthreekindsofdevices.Mostdeviceshaveasingleserverwhoseservicetimedoesnotdependuponthenumberofjobsinthedevice.Suchdevicesarecalledfixedcapacityservicecenters.Forexample,theCPUinasystemmaybemodeledasafixedcapacityservicecenter.Thentherearedevicesthathavenoqueuing,and jobsspendthesameamountoftime inthedeviceregardlessofthenumberof jobs in it.Suchdevicescanbemodeledasacenterwith infinite serversandarecalleddelaycentersor IS (infinite server).Agroupofdedicatedterminalsisusuallymodeledasadelaycenter.Finally,theremainingdevicesarecalledloaddependentservicecenterssincetheirserviceratesmaydependupontheloadorthe number of jobs in the device. AM/M/m queue (withm e 2) is an example of a loaddependentservicecenter. Itstotalservicerate increasesasmoreandmoreserversareused.Agroup of parallel links between two nodes in a computer network is an example of a loaddependentservicecenter.

Definition - What does Queue mean? A queue, in computer networking, is a collection of data packets collectively waiting to be transmitted by a network device using a per-defined structure methodology. A queue consists of a number of packets. These packets are bound to be routed over the network, lined up in a sequential way with a changing header and trailer and taken out of the queue for transmission by a network device using some defined packet processing algorithm like first in first out (FIFO), last in last out (LIFO), etc. The queue dequeues, or takes out a data packet from the head, when it needs to transfer and trailer by adding new data packets to the queue, which is known as enqueuing. A queue works almost on the same methodology used at banks or supermarkets, where the customer is treated according to its arrival. An example would be FIFO or some other priority if they are a privileged customer. Similarly, a network queue processes data packets based on their arrival, priority, smallest task first and multitasking, FIFO, LIFO, emption and pre-emption.

CSC 524

VERIFICATION AND VALIDATION OF SIMULATION MODELS

AMUDA, Tosin Joseph

090805009

UNIVERSITY OF LAGOS

ii

Abstract

Simulation models are more and more used to solve difficult scientific and social problems and

to aid in decision-making. The developers and users of these models, the decision makers using

information obtained from the results of these models, and the individuals affected by decisions

based on such models are all rightly concerned with whether a model and its results are

correct.

Consequently, no model can be accepted unless it has passed the tests of validation. Therefore,

it is salient to carry out the procedure of validation to ascertain the credibility of a simulation

model. This usually involve a twin process: validation and verification. This rest of this article

will review several literatures on how to verify and validate our simulation models in order to

ensure models credibility to an acceptable level.

iii

Table of Contents Abstract ...................................................................................................................................... ii

Section 1: Introduction ............................................................................................................... 1

Section 2: Verification ............................................................................................................... 3

2.1: Good Programming Practice ........................................................................................... 3

Section 3: Validation.................................................................................................................. 4

3.1 Face Validity .................................................................................................................... 4

3.2 Validation of Model Assumptions ................................................................................... 5

Structural Assumptions ...................................................................................................... 5

Data Assumptions .............................................................................................................. 5

3.3 Validating Input-Output Transformations ....................................................................... 6

Hypothesis Testing............................................................................................................. 6

Model Accuracy as a Range .............................................................................................. 7

Confidence Intervals .......................................................................................................... 7

Section 4: Conclusion ................................................................................................................ 8

References .................................................................................................................................. 9

1

Section 1: Introduction

There is always a need to evaluate and improve the performance of a system that evolves over

time. First, the behaviour of such system must be studied. For one to study the behaviour of a

system, one must first come up with a representation (a close approximation) of such system.

This representation of the construction and working of a system of interest is known as a

Model. In addition, experiment will be carried out on the model in order to imitate the

operations of the actual system. This process- usually carried out on a computer- is known as

Simulation. Generally, a model intended for a simulation study is a mathematical model

developed with the help of simulation software.

Simulation models are approximate imitations of real-world systems with several assumption

and they never exactly imitate the real-world system. Due to the assumptions and

approximation, an important issue in modelling is model validity. Therefore, a model should

be verified and validated to the degree needed for the models intended purpose or application

This concern for quantifying and building credibility in simulation models is addressed by

Verification and Validation (V & V). This paper uses the definitions of V & V given in the

classic simulation textbook by Law and Kelton (1991, p.299): "Verification is determining

that a simulation computer program performs as intended, i.e., debugging the computer

programValidation is concerned with determining whether the conceptual simulation model

(as opposed to the computer program) is an accurate representation of the system under study".

Both verification and validation are processes that accumulate evidence of a models

correctness or accuracy for a specific scenario; thus, (V & V) cannot prove that a model is

correct and accurate for all possible scenarios, but, rather, it can provide evidence that the

model is sufficiently accurate for its intended use.

2

Another popular author on V & V in simulation relate the various phases of modelling with V

& V in Figure 1: Sargent (1991, p.38) states "the conceptual model is the

mathematical/logical/verbal representation (mimic) of the problem entity developed for a

particular study; and the computerized model is the conceptual model implemented on a

computer. The conceptual model is developed through an analysis and modelling phase, the

computerized model is developed through a computer programming and implementation

phase, and inferences about the problem entity are obtained by conducting computer

experiments on the computerized model in the experimentation phase".

Figure 1Simplified Version of the Modeling Process

There is no standard theory on V&V, therefore, there exist a number of philosophical theories,

statistical techniques; software practices, and so on. However, the emphasis of this article is on

statistical techniques, which may yield reproducible, objective, quantitative data about the

quality of simulation models.

This article is organized as follows. Section 2 discusses verification. Section 3 examines

validation. Section 4 provides conclusions. It is followed by a list of references.

3

Section 2: Verification

Once the simulation model has been programmed, the analysts/programmers must check if this

computer code contains any programming errors ('bugs') to ensure that the conceptual model

is reflected accurately in the computerized representation. The objective of model verification

is to ensure that the implementation of the model is correct.

Various processes and techniques are used to assure the model matches specifications and

assumptions with respect to the model concept. Many common-sense suggestions are

applicable, but none is perfect, for example: 1) general good programming practice such as

object oriented programming, 2) checking of intermediate simulation outputs through tracing

and statistical testing per module, 3) comparing (through statistical tests) final simulation

outputs with analytical results, and 4) animation.

Many software engineering techniques used for software verification are applicable to

simulation model verification.

2.1: Good Programming Practice

Software engineers have developed numerous procedures for writing good computer programs

and for verifying the resulting software, in general (not specifically in simulation). One of the

few best software engineering practices are: object oriented programming, formal technical

review, structured walk-throughs, correctness proofs

There are many software engineering testing and quality assurance techniques that can be

utilized to verify a model. Including, but not limited to, have the model checked by an expert

(e.g. chief programmer), making logic flow diagrams that include each logically possible

action, examining the model output for reasonableness under a variety of settings of the input

parameters, and using an interactive debugger.

4

Section 3: Validation

Once the simulation model is programmed correctly, we face the next question: is the

conceptual simulation model (as opposed to the computer program) an accurate representation

of the system under study.

There are many approaches described here from literatures that can be used to validate a

computer model. The approaches range from subjective reviews to objective statistical tests.

By objectively, we mean using some type of mathematical procedure or statistical test, e.g.,

hypothesis tests or confidence intervals. One approach that is commonly used is to have the

model builders determine validity of the model through a series of tests.

Naylor and Finger [1967] formulated a three-step approach to model validation that has been

widely followed:

Step 1. Build a model that has high face validity.

Step 2. Validate model assumptions.

Step 3. Compare the model input-output transformations to corresponding input-output

transformations for the real system.

3.1 Face Validity

A model that has face validity appears to be a reasonable imitation of a real-world system to

people who are knowledgeable of the real world system. Face validity is tested by having users

and people knowledgeable with the system examine model output for reasonableness and in

the process identify deficiencies. An added advantage of having the users involved in validation

is that the model's credibility to the users and the user's confidence in the model increases.

Sensitivity to model inputs can also be used to judge face validity. For example, if a simulation

of a fast food restaurant drive through was run twice with customer arrival rates of 20 per hour

5

and 40 per hour then model outputs such as average wait time or maximum number of

customers waiting would be expected to increase with the arrival rate

3.2 Validation of Model Assumptions

Assumptions made about a model generally fall into two categories: structural assumptions

about how system works and data assumptions.

Structural Assumptions

Assumptions made about how the system operates and how it is physically arranged are

structural assumptions. For example, the number of servers in a fast food drive through lane

and if there is more than one how are they utilized? Do the servers work in parallel where a

customer completes a transaction by visiting a single server or does one server take orders and

handle payment while the other prepares and serves the order. Many structural problems in the

model come from poor or incorrect assumptions. If possible the workings of the actual system

should be closely observed to understand how it operates. The systems structure and operation

should also be verified with users of the actual system.

Data Assumptions

There must be a sufficient amount of appropriate data available to build a conceptual model

and validate a model. Lack of appropriate data is often the reason attempts to validate a model

fail. Data should be verified to come from a reliable source. A typical error is assuming an

inappropriate statistical distribution for the data. The assumed statistical model should be tested

using goodness of fit tests and other techniques. Examples of goodness of fit tests are the

KolmogorovSmirnov test and the chi-square test. Any outliers in the data should be checked.

6

3.3 Validating Input-Output Transformations

The model is viewed as an input-output transformation for these tests. The validation test

consists of comparing outputs from the system under consideration to model outputs for the

same set of input conditions. Data recorded while observing the system must be available in

order to perform this test. The model output that is of primary interest should used as the

measure of performance. For example, if system under consideration is a fast food drive

through where input to model is customer arrival time and the output measure of performance

is average customer time in line, then the actual arrival time and time spent in line for customers

at the drive through would be recorded. The model would be run with the actual arrival times

and the model average time in line would be compared actual average time spent in line using

one or more tests.

Hypothesis Testing

Statistical hypothesis testing using the t-test can be used as a basis to accept the model as valid

or reject it as invalid.

The hypothesis to be tested is

H0 the model measure of performance = the system measure of performance

versus

H1 the measure of performance the measure of performance.

The test is conducted for a given sample size and level of significance or . To perform the test

a number n statistically independent runs of the model are conducted and an average or

expected value, E(Y), for the variable of interest is produced. Then the test statistic, t0 is

computed for the given , n, E(Y) and the observed value for the system 0

7

and the critical value for and n-1 the degrees of

freedom

is calculated.

If

reject H0, the model needs adjustment.

Model Accuracy as a Range

A statistical technique where the amount of model accuracy is specified as a range has recently

been developed. The technique uses hypothesis testing to accept a model if the difference

between a model's variable of interest and a system's variable of interest is within a specified

range of accuracy. A requirement is that both the system data and model data be approximately

Normally Independent and Identically Distributed (NIID). The t-test statistic is used in this

technique. If the mean of the model is m and the mean of system is s then the difference

between the model and the system is D = m - s. The hypothesis to be tested is if D is within

the acceptable range of accuracy.

Confidence Intervals

Confidence intervals can be used to evaluate if a model is "close enough" to a system for some

variable of interest. The difference between the known model value, 0, and the system value,

, is checked to see if it is less than a value small enough that the model is valid with respect

that variable of interest. The value is denoted by the symbol . To perform the test a number,

n, statistically independent runs of the model are conducted and a mean or expected value,

E(Y) or for simulation output variable of interest Y, with a standard deviation S is produced.

8

Section 4: Conclusion

This paper surveyed verification and validation (V&V) of simulation models. It emphasized

statistical techniques that yield reproducible, objective, quantitative data about the quality of

simulation models.

For verification it discussed the following techniques (see Section 2):

1) General good programming practice such as objected oriented programming;

2) Checking of intermediate simulation outputs through tracing and statistical testing per

module

3) Comparing final simulation outputs with analytical results for simplified simulation models,

using statistical tests;

4) Animation.

For validation it discussed the following techniques (see Section 3):

1). Building a model that has high face validity.

2). Validating model assumptions.

3). Comparing the model input-output transformations to corresponding input-output

transformations for the real system.

9

References

1. Banks, Jerry; Carson, John S.; Nelson, Barry L.; Nicol, David M. Discrete-Event System

Simulation Fifth Edition, Upper Saddle River, Pearson Education, Inc. 2010 ISBN

0136062121

2. Sargent, Robert G. VERIFICATION AND VALIDATION OF SIMULATION MODELS.

Proceedings of the 2011 Winter Simulation Conference. http://www.informs-

sim.org/wsc11papers/016.pdf

3. Carson, John, MODEL VERIFICATION AND VALIDATION. Proceedings of the 2002 Winter

Simulation Conference. http://informs-sim.org/wsc02papers/008.pdf

4. NAYLOR, T. H., AND J. M. FINGER [ 1967], Verification of Computer Simulation Models,

Management Science, Vol. 2, pp. B92 B101., cited in Banks, Jerry; Carson, John S.;

Nelson, Barry L.; Nicol, David M. Discrete-Event System Simulation Fifth Edition, Upper

Saddle River, Pearson Education, Inc. 2010 p. 396 ISBN

0136062121,http://mansci.journal.informs.org/content/14/2/B-92

5. Sargent, R. G. 2010. A New Statistical Procedure for Validation of Simulation and Stochastic

Models. Technical Report SYR-EECS-2010-06, Department of Electrical Engineering and

Computer Science, Syracuse University, Syracuse, New York.

6. Law, A.M., and Kelton, W.D. (1991), Simulation Modeling and Analysis, 2nd ed., McGraw-Hill,

New York.

7. Jack P.C (1992), Theory and Methodology: Verification and validation of simulation models,

European Journal of Operational Research 82 (1995) 145-162

DOKAI ANN UNOR 090805020

SINGLE SERVER QUEUE AND QUEUEING NETWORKS

Single Server Queues

The basic scenario for a single queue is that customers, who belong to some population, arrive at

the service facility. The service facility has one or more servers who are capable of performing

the service required by customers. If a customer cannot gain access to a server it must join a

queue, in a buffer, until a server is available. When service is complete the customer departs, and

the server selects the next customer from the buffer according to the service discipline.

In order to describe a service facility accurately we need to know details about each of the terms

emphasized above:

Arrival Pattern of Customers: The ability of the service facility to provide service for an arriving

stream of customers depends not only on the mean arrival rate , but also on the pattern in which they arrive, i.e. the distribution function of the inter-arrival times.

Service Time Distribution: The service time is the time which a server spends satisfying a

customer. As with the inter-arrival stream, the important characteristics of this time will be both

its average duration and the distribution function. If the average duration of a service interaction

between a server and a customer is 1/ then is the service rate.

Server: In a single server queue, the service facility can only serve one customer at a time,

waiting customers will stay in the buffer until chosen for service; how the next customer is

chosen will depend on the service discipline.

Buffer Capacity: Customers who cannot receive service immediately due to unavailability of the

server must wait in the buffer. This leads to buffer being filled up if the buffer has a finite

capacity. If the buffer gets filled up there are two possibilities:

BufferServer

Arrivaltoqueue Departurefromqueue

Either, the fact that the facility is full is passed back to the arrival process and arrivals are suspended until the facility has spare capacity (created by the completion of a customer

who is currently being served);

Or, arrivals continue and arriving customers are turned away and lost until the facility has spare capacity again.

In some systems the buffer capacity is so large as to never affect the behavior of the

customers; in this case the buffer capacity is assumed to be infinite.

Service Discipline: When more than one customer is waiting for service there has to be a rule for

selecting which of the waiting customers will be the next one to gain access to a server. The

commonly used service disciplines are:

FCFS first-come-first-serve (or FIFO first-in-first-out). LCFS last-come-_first-serve (or LIFO last-in-first-out). RSS random-selection-for-service. PRI priority. The assignment of different priorities to elements of a population is one way

in which classes are formed.

Population: The characteristic of the population which we are interested in is usually the size.

Clearly, if the size of the population is fixed, at some value N, no more than N customers will

ever be requiring service at any time. When the population is finite the arrival rate of customers

will be affected by the numbers who are already in the service facility. The arrival rate will be

zero when all the population is already in the facility.

A shorthand notation for these six characteristics of a system is provided by Kendall's notation

for classifying queues. In this notation a queue is represented as A/S/c/m/N/SD:

A denotes the arrival process; usually M, to denote Markov (exponential), G, general, or D,

deterministic distributions.

S denotes the service rate and uses the distribution identifiers.

c denotes the number of servers available in the service facility. C= 1 in a single server case.

m denotes the capacity of the service facility (buffer + server(s)), infinite by default.

N denotes the customer population, also infinite by default.

SD denotes the service discipline, FCFS by default.

The single server queue is stable if on the average, the service time is less than the inter-arrival

time i.e. mean service time < mean inter-arrival time.

Throughput of a single server queue:

Throughput is the average number of completed jobs per unit.

Throughput of a single server queue is the average number of jobs that depart from the queue per

unit time (after they have been serviced)

Example:

The mean service time =10 mins.

-What is the maximum throughput (per hour)?

-What is the throughput (per hour) if the mean inter-arrival time is:

Traffic Intensity

The two most important features of a single queue are the arrival rate of customers , and the service rate of the server(s), . These are combined into a single parameter which characterises a

single or multiple server system, the traffic intensity.

Traffic intensity, = Many of the important performance measures associated with a queue can be expressed in terms

of . These are measures such as utilisation (the proportion of time the server is busy), residence time (the average time spent at the service facility by a customer, both queuing and receiving

service), waiting time (the average time spent queuing), queue length (the average number of

customers at the service facility, both waiting and receiving service), and throughput (the rate at

which customers pass through the service facility).

For the system to be stable, must be less than 1: that is the arrival rate of customers must be less than the rate at which they are served.

Queuing Networks

A queuing network refers to a system where there are several stations of service (identical or

non-identical) and a customer undergoes service at all or few service stations,

There are two types of queuing networks:

1. Open queuing networks: An open queuing network refers to a network in which the

customers arriving to the system, leaves the system after completion of service. The

experiments presented in the lab take into consideration a special class of open queuing

networks called the Jackson networks.

A queuing network is a Jackson network if it satisfies the following conditions:

The network is open and any external arrivals to node i form a Poisson process All service times are exponentially distributed and the service discipline at all

queues is FCFS.

A customer completing service at queue I will either move to some new queue j with probability 1 which is non-zero for some subset of the queues.

The utilization of all of the queues is less than one. 2. Closed queuing networks: In closed queuing networks, work pieces circulate through the

system. The arrival stream at the first station conforms to the departure process at the last

station. In contrast to open queuing systems, the last station of a closed queuing network

may become blocked if the buffer in front of the input station is full of work pieces. This

holds true under the assumption that the buffer capacities within the system are finite.

Furthermore, the first station may become starved if no carriers with work pieces are

available in the buffer in front of the input station.

Littles Law

Littles law says that under steady state conditions, the average number of items in a queuing

system equals the average rate at which items arrive multiplied by the time that an item

spends in the system. Letting

L =average number of items in the queuing system

W= average waiting time in the system for an item, and

= average number of items arriving per unit time, the law is L = W

NAME:DURUDUMEBIJULIANMATRIC:090805021COURSE:CSC524ModellingComputerSystemsNetworksAbstract:Trafficmodelsincomputernetworksareaverycomplicatedsystem.Simulationofthesesystemsisverydifficultbecausetheyshownonlinearbehaviour.However,theprocessisimportantbecauseComputerNetworkAdministratorsneedtoknowthecapabilitiesoftheirnetworks.Theydonotwanttheservertobeoverloadedasaresultofreceivingtoomanyrequeststhanitcanhandle.Therestofthiswriteupisdedicatedtomodellingofcomputernetworks.Introduction:Inlargescalenetworks,datacanfollowavarietyofroutesingettingfromthesourcetothedestination.Theroutefolloweddependsheavilyontheamountofinformationnodeshaveontheirneighbours.Importantparameterslikebottlenecks,congestion,datarates,bandwidthetc.,needtobemodelledandstudied.However,insufficientoperationofanetworktracesbacktocongestion.NetworkElements:Elementsinacomputernetworkaredividedintwobroadgroups.Ontheonehand,thenodesimplythestorageunits,ontheotherhand,thetransfermediaconnectthenodestoeachother..Geometricalconditions: .Distanceofnodes. .Materialsofthetransfermedium. .Numberofusers..Seasonaleffects: .Thenumberofusersusingthenetworkactuallymovesonaverywidescale..Diverseexternalfactors: .Weather

.Electromagneticstorm.Nodes:Nodesaretheactiveelementsofanetworkthatcommunicatewitheachother.TransfermediumsTheprocessofthecommunicationisimplementedthroughthetransfermedium.Thecommunicationisveryfast.Thismakestransferofpossiblylargeamountsofdatafeasiblewithinashorttime.Onewaycommunicationisusuallyenforcedtoavoiddatacollisions.Communication:Communicationbetweenthenodesofacomputernetworkisalwaysdynamic.Thechangesofthedynamicmessagetransmissionsaremodelledwiththehelpofacommunicationmatrix.Thetaskofthismatrixistotakeintoconsiderationallelementsofthenetworkaswellastheirfeatures,whichregulatethedatatrafficbetweenthenodes.Moreparameterswillhavetobeconsidered,forinstance:lengthandnumbersofdatatransfersections,thephysicalparametersofthemedium,thesizesofstorageunitsinthenodes,thedegreeoftheutilisationetc.Somecharacteristicsofcomputernetworkforfaultlessdatatransfer

Inthecourseofdatatransfer,althoughtransfervelocityofonedatabitisconstantintime,transmissiontimesofpacketswiththesamelongitudemaybedifferent.

Inthecourseofdatatransfer,thedatatransfersectionsrunninginparallelwitheachotherdonotaffecteachotherdirectly,butinthenodes,forexampletheappearanceofmultipliedpacketsmakesdisturbingeffects.

Twowaytrafficdoesnotexist. Theintensityofinnercommunicationchangesintimebetweenthenodesdirectlyconnected

toeachother.Incaseofwronglychosenparametersforexamplethisinnercommunicationcapabletocreatepeakloadontheexaminednetworkwithouttrafficarrivingfromtheexternalnetwork.

Tocontrolaffectingmessagetransmission,aninnercommunicationalsystemworksbetweenthenodesconnectedtoeachother.Forexample,thereceivercanreceiveamessagevainlyifthetransmitterdoesnothaveamessagetobesenton.

TrafficsimulationmodelTransactionsincomputernetworksshownonlinearfeatures.Forexample,sometimesnodatatransferbetweentwonodes,becausebufferofreceivernodeisfull,thoughthesenderiscapableoftransferringdata.Acommunicationgraphistheoutputofthesimulationmodelthatimitatesthe

physicalarrangementofthecomputernetwork.Thenodesareconnectedbylinescallededgeswhichareactuallytransfermedia. COMMUNICATIONGRAPHTheabovecommunicationnoderepresentsasimplecommunicationgraphwithroutersandterminals.Eachnodehasabufferforstoringmessages.Thiscapacityismeasuredindatadensity(l)thatshowsrateofrecentdataquantityintimeandthemaximaldataquantityinnodei.B(t)=(numberofrecentdatabitsinbufferofnodeI)=N(t)(buffersizeofnodeI)ADatadensitygivesanideaonthenumberofdatabitstransferredinthenexttimeunit.N(t)andb(t)arevectorsofnelements.

N(t)=A.b(t)IfelementsofN(t)areknown,totalnumberofdatabitsstoredintheobservedareaiscalculated(4)asavectorscalarproduct:

N(t)=A.b(t)=diag(A).b(t)

Nowexaminingstateofthenetworkintimet+t.Theamountofdatastoredinnodeichangesas:

N(t+t)=N(t)+Ninternal+NinputNoutput

Inthenetwork,thetransmissionspeedmeanshowmanybitscanbetransportedinsecondsbetweentwonodes.Transmissionspeedmaybedifferentinsomepartsofthenetwork,butfromnodejtonodeithesamevalueissupposedandmarkedwithvij.Aftersummarisingdatachangesinallnodes,totaldatachangeinanetworkisgivenas:

LimN(t+t)N(t)=t+ttN(t).dt

CommunicationfunctionSomeoftheparameterstoconsiderregardingcommunicationare:.Internalconnectionsofthenetwork.Environmentalconnectionsofthenetwork.Sizeofinternalbuffersinnodes.CapabilitiesofnodestosendandreceivedataThesubfunctionofinternalorenvironmentalconnection,markedwithkgrantstheconnectionwhennodejcancommunicatewithnodei.Eachnodeisexaminedandifaphysicalconnectionexistsbetweennodesjandnodei,fijissetto1,otherwiseitiszero.Hence,fijisasymmetricalmatrixbecauseifnodeIisconnectedtonodej,thennodejisconnectedtonodei.Ifthereexistsnophysicalconnectionbetweenthesetwonodes,thenkisequalto0.

k=fij.pijifconnectionexistsbetweennodesjandnodei.k=0ifnoconnectionexistsbetweennodesjandi.whereijand1ijn.S(t)isaninternalsubfunctionoftransmitternodewithvaluesof1or0.Itshowswhethernodejhasanymessagetosendornot.Theconnectionisdisabledifdatadensityofnodej(b(t))isequalto0.S(t)=1b(t)>0S(t)=0b(t)=0R(t)isanotherinternalsubfunctionofthereceivernodewithvaluesof1or0.Theconnectionisenabledifb(t),thedatadensityofnodeIsmallerthan1,anywayitis0.ZeromeansthatbufferofnodeIhasbeenoverloaded,sonodeIclosesitscommunicationporttodirectionofnodej,thereforenodeIdoesnotreceiveanymessagefromnodej.R(t)=1b(t)

ThecommunicationfunctionbetweennodejandIisaproductofthethreesubfunctions.C=k(t).S(b(t),t).R(b(t),t)

QueueingModelQueueingtheoryhelpsdeterminetimejobsspendonthequeue.Italsohelpspredictresponsetime.Twooftheearliestqueueingmodelsofcomputersystemsarethemachinerepairmanmodelandthecentralservermodel.Themachinerepairmanmodel,asthenameimplies,wasoriginallydevelopedformodelingmachinerepairshops.Ithasanumberofworkingmachinesandarepairfacilitywithoneormoreservers(repairmen).Wheneveramachinebreaksdown,itisputinthequeueforrepairandservicedassoonasarepairmanisavailable.Scherr(1967)usedthismodeltorepresentatimesharingsystemwithnterminals.Userssittingattheterminalsgeneraterequests(jobs)thatareservicedbythesystem,whichservesasarepairman.Afterajobisdone,itwaitsattheuserterminalforarandomthinktimeintervalbeforecyclingagain.ThecentralservermodelshownwasintroducedbyBuzen(1973).TheCPUinthemodelisthecentralserverthatschedulesvisitstootherdevices.AfterserviceattheI/Odevices,thejobsreturntotheCPUforfurtherprocessingandleaveitwhenthenextI/Oisencounteredorwhenthejobiscompleted.Incomputersystemsmodeling,weencounterthreekindsofdevices.Mostdeviceshaveasingleserverwhoseservicetimedoesnotdependuponthenumberofjobsinthedevice.Suchdevicesarecalledfixedcapacityservicecenters.Forexample,theCPUinasystemmaybemodeledasafixedcapacityservicecenter.Thentherearedevicesthathavenoqueueing,andjobsspendthesameamountoftimeinthedeviceregardlessofthenumberofjobsinit.SuchdevicescanbemodeledasacenterwithinfiniteserversandarecalleddelaycentersorIS(infiniteserver).Agroupofdedicatedterminalsisusuallymodeledasadelaycenter.Finally,theremainingdevicesarecalledloaddependentservicecenterssincetheirserviceratesmaydependupontheloadorthenumberofjobsinthedevice.AM/M/mqueue(withme2)isanexampleofaloaddependentservicecenter.Itstotalservicerateincreasesasmoreandmoreserversareused.Agroupofparallellinksbetweentwonodesinacomputernetworkisanexampleofaloaddependentservicecenter.Unlessspecifiedotherwise,itisassumedthattheservicetimesforallserversareexponentiallydistributed. Terminals

System MODELDIAGRAM

QueuingAnalysisrequiresrecognitionthefollowingparameters: Populationsize Numberofservers Systemcapacity Arrivalprocess Servicetimedistribution Servicediscipline

PopulationSize:Numberofpotentialcustomersthatcanenterthesystem.Thisnumbercanbeinfiniteorfinite,dependingontheimplementationofthesystem.Numberofservers:Canbeoneormore.Assumeeachserverimplementsasimilarqueuingsystem.Systemcapacity:Numberthatcanwaitplusnumberthatcanbeserved.Mostsystemshavefinitequeuelengththoughitiseasiertoanalyseaninfinitequeuelength.Arrivalprocess:Systemsarriveatt1,t2,tn.Interarrivaltimescanbegivenby=TjTj1.MostarrivalsarePoissondistributed:f(x)=ex.ServiceTimedistribution:Amountoftimeeachcustomerspendsattheserver.Servicediscipline:Theorderinwhichcustomersareserviced.MostcommonorderisFCFC(FirstComeFirstService).TheseparametersarerepresentedinKendallsnotation.A/S/m/B/K/SDAArrivaltimeSServicetimemNumberofserversBNumberofbuffersKispopulationsizeSDisservicedisciplineNetworkofqueuesTheflowofrequeststhrowasystemmayinvolveanumberofdifferentservicestationsandnavigationpaths.Example:

FlowofIPpacketsthroughacomputernetwork Flowofordersthroughamanufacturingsystem

Floworrequests/messagesthroughawebservicesystem Flowofpaperworkthroughanadministrationoffice.

Theabovesystemscanbeimplementedasnetworksofqueues.Queuescanbelinkedtogethertoformanetworkofqueueswhichreflecttheflowofcustomersthroughanumberofdifferentservicestations.

TrafficequationForMnodes,ifistheexternalarrivalrateintonodeI,andpijisthebranchingpossibility.Thentheeffectivearrivalrateisgivenas:

1=1+p

M=M+p

MODELLING OF COMPUTER SYSTEMS

NETWORKS

SYSTEM PERFORMANCE EVALUATION

CSC 524

By

Kesa Oluwafunmilola Elizabeth

090805026

Lecturer: Dr. A.P. Adewole

Introduction

What is performance?

Performance is the quantitative measure of a system. If you are unable to measure the

performance of a system, then you will be unable to control and manage the system. Taking

the performance measurements of an existing running system is relatively simple compared to

when a system totally or partially does not exist. How do we take the performance measurement

of a non-existing system? In this case, a formal way to take the measurements will be to

estimate the performance measurements by means of some kind of mathematical performance

model (Ramon, 2003).

What is performance model?

Performance model is a mathematical representation of the system behaviour in which we try

to keep with a good representation the most significant mechanisms of the system evolution

along the time and we neglect or simplify the representation of the rest of the mechanisms

(Ramon, 2003).

Performance model is created to define the significant aspects of the way in which a proposed

or actual system operates in terms of resources consumed, contention for resources, and delays

introduced by processing or physical limitations (such as speed, bandwidth of communications,

access latency, etc.) (John, 2004). It should be noted however, that a model is always an

approximate. The only perfect system is the system itself.

Computer networks are telecommunications networks that allows computers to exchange data.

In computer networks, networked computing devices transmit data to each other through

network links (either cable or wireless).

Network success is dependent on several performance attributes. The type and location of the

network deployment will influence performance. Network performance is usually measured by

the quality of service. Typically, the parameters that affect the network performance include

throughput, latency, bit error rate and jitter.

Kinds of Models

A brief overview of the kinds of models is given below

1. Physical model:

A physical model is one which is usually a physical replica, often on a reduced scale,

of the system it represents. A physical model looks like the object it represents and

is also called an Iconic Model. For instance, a model of an airplane (scaled down), a

model of the atom (scaled up).

2. Simulation model:

Simulation is the act of executing, experimenting with or exercising a model for a

specific objective such as acquisition, analysis, education, entertainment, research or

training.

3. Analytical model:

Analytical model is one which is solved by using the deductive reasoning of

mathematical theory. An M/M/1 queuing model, a Linear Programming model, a

nonlinear optimization model are examples of analytical models.

Performance modelling techniques

The main goal of describing the behaviour of some system is to evaluate the time needed by

any entity to cross the system. This time has two main components: the strict time needed for

its execution in the different hardware components and the time spent waiting either to use

some resource.

Modelling may be implemented as a simulator (an operation abstraction) or as an abstract

mathematical representation of the system behaviour. The main existing mathematical

techniques are based on the following formalisms: Queuing networks, Petri nets and Process

algebras.

Queuing networks

Queuing theory is the key analytical modelling technique used for computer systems

performance analysis. A queue can be considered as a service facility with customers from

some population or source entering to receive some type of this service. The concept of

customer is used in the generic sense and therefore may be a person, a job, an inquiry, a

message, a packet, a program etc. the service facility has one or more servers (entities that

provide services to customer in a certain time). If all servers are busy when a customer arrives

at the system, it must join the queue until a server is free. Note, the servers associated with

queues correspond to resources such as CPU, disks and other devices and customers that enter

queues correspond to the elements that constitute the workload of the system itself. Therefore,

a simple queue consists of an arrival process, buffer where customers await service and a

number of servers which must be retained by each customer for the service period. Queuing

theory helps in determining the time that the customer spend in various queues in the system.

Queuing Notation

Kendalls notation is the standard system used to describe and classify a queuing node. A queue

is described as A/S/c/k/m/D where A is the arrival process or interarrival time distribution, S is

the service time distribution, c is the number of servers, k is the system capacity, m is the

population size, and D is the service discipline.

The six parameters identified in the Kendalls notation are briefly described below

1. Arrival Process: If the customers arrive at time 1, 2, , , the random variables =

1 are called the interarrival times. The most common arrival process is Poisson

arrivals, which simply means that the interarrival times are Independent and Identically

Distributed (IID) and are exponentially protected.

2. Service Time Distribution: Service time is the time each customer spends at the

terminal. It is common to assume that the service times are random variables, which are

IID. The distribution most commonly used is the exponential distribution.

3. Number of Servers: The terminal room may have one or more terminals, all of which

are considered part of the same queuing system and any terminal can be assigned to any

customer. If all the servers are not identical, they are usually divided into groups of

identical servers with separate queues for each group.

4. System Capacity: The maximum number of customers who can stay may be limited

due to space availability and also to avoid long waiting times. This number is called the

system capacity. In most systems, the capacity is finite. The system capacity includes

those waiting for service as well as those receiving service.

5. Population Size: The total number of potential customers who can ever come to the

system is the population size.

6. Service Discipline: The order in which the customers are served is called the service

discipline. The most common discipline is First Come, First Served (FCFS). Other

disciplines are Last Come, First Served (LCFS), Last Come, First Served with Preempt

and Resume (LCFS-PR).

A queuing network can be formalised as a directed graph in which the nodes are queues, often

called service centres, each representing a resource in the system. Customers representing the

jobs, users or tasks in the system, flow through the model and compete for these resources. The

arcs of the network represent the topology of the system, and together with the routing

specification (e.g. routing probabilities), determine the paths that customers take through the

network.

Petri nets

Petri nets were first introduced by Carl Adam Petri in 1962. Petri net is a diagrammatic tool to

model concurrency and synchronization in distributed systems. Petri nets are represented with

directed graphs with two types of nodes, places (circles) and transitions (rectangles), and

unidirectional arcs (arrows) between them. Places represent possible states of the system;

transitions are events or actions that cause the change of state; and every arc simply connects

a place with a transition and a transition with a place.

Solving techniques

The formalisms discussed previously have an associated set of analytical techniques used for

their solution of performance metrics. The solving techniques include mathematical methods

such as analytical methods and Markov chains. In addition to these mathematical methods,

simulation is used for its easy and intuitive understanding.

Analytical methods

Depending on the demand for the resources and the service rate that the customers experience,

contention may arise leading to the formation of a queue of waiting customers. Asides the

number of customers in the queue, other processes of interest could be: the actual waiting time

of the nth customer in the queue, the busy period, and the output process. These quantities can

be solved by using analytical or simulation techniques, in addition to transforming them into

the underlying Markov chain.

One of the most successful modelling methods in recent years uses so-called Generalised

Stochastic Petri Nets (GSPN). GSPNs can be solved by using analytical or simulation

techniques, or by transforming them into their underlying Markov chain if this is not too large.

Figure 1: Example of a Petri Net

Markov chains

Markov chain refers to the sequence of random variables such a process moves through, with

the Markov property defining serial dependence only between adjacent periods. It can thus be

used for describing systems that follow a chain of linked events, where what happens next

depends only on the current state of the system. The term is reserved for a process with a

discrete set of times (i.e. a discrete-time Markov chain (DTMC)) although the terminology is

also used to refer to continuous-time Markov chain (CTMC). DTMC or CTMC are state

transition systems in which each transition has an associated probability (in DTMCs) or rate

(in CTMCs). They can therefore be used to model a wide class of concurrent systems that

satisfy the Markov property viz. that the evolution of the system after a given time instant

depends only on the state at that instant and not on any past history.

Simulation

Simulation implies writing programme describing the timing behaviour of the system. The

main advantage of a simulation is that it does not have any theoretical limitation and allows us

to study systems that cannot be modelled in such a way that some analytical method could be

applicable. In addition, simulation is very easy to learn and to apply. So, it is a very popular

modelling technique.

Its main disadvantages are the effort (mainly time) spent in developing and debugging the

simulation programme and its execution time.

Practical Hints

How to use modelling for evaluating the system performance?

Very frequently modelling is used to analyse a system that does not exist. In this case it is

impossible to validate the model and it is convenient to proceed by step refinements.

1. Start with a simple model, analytical if possible because they are easier to debug than

the simulated ones.

2. Build a simulation model, debug it and check if the result agree with the result in step

1.

3. Include some new mechanisms or refine some of what you have represented, debug the

new model and check the consistency of the results with the previously obtained.

With these steps, we can refine our model and arrive at a much accurate representation of the

system.

Modelling example

Let us consider a TCP/IP system composed of two IPSs connected by a link at 2 Mbit/s. Each

ISP manages 5 email stations and a sixth one delivering files. The links between the stations

and the ISP are at 256 Kbit/s. Email station generates 10 message/s with a mean size of 1.5

Kbytes with exponential distribution and uniform destination among the other 9 stations. Each

file transfer station generates 0.5 file/minute with a mean size of 2 MB with exponential

distribution and the other file transfer station as destination. It can be assumed that the

maximum message size allowed in the network is 1 KB.

An important point for studying this system is to decide the size of the buffers. As we have no

reference, initially we will study the system by means of an analytical model. Taking into

account the symmetry of the system, we will consider just half of it. Also, as we cannot

represent exactly the mechanisms of datagram generation, we will do some approximations:

one (optimistic, Model 1 in Annex) assuming that the messages for transferring the files and

sending the emails are generated at random with a size of 1KB and the other (pessimistic,

Model 2 in Annex), assuming that the files and the emails are sent in just one message of its

mean size. The queue sizes in KB are the following:

LINK ISP EMAIL FTP

Mean 1 0.3043 0.1765 0.9231 1.143

Max 1 5 4 9 10

Mean 2 177.75 1.3458 1.4331 2341

Mean 3 129.1 7.1365 3.2328 1705

Losses 4 6.5 x 10-3 6.7 x 10-3 1.2 x 10-6 0.09

Now, we have an idea of the sizes to be allocated to the different buffer and we can build a

simulation model with a better representation of the datagram generation procedure. Initially,

in order to decrease its difficulty we will consider that all buffers have infinite capacity (Model

3 in Annex). The main difference between this model and the previous ones is the

representation of the load because now what is following a Poisson arrival is the generation of

an email message or a file to be transferred; then each file or email is decomposed in messages

of 1KB and a residual message of, at least 128 bytes. The next step will be to include the limited

capacity of the buffers (Model 4 in Annex) to evaluate the losses of each service type for some

6

7

8

9

10

1

2

3

4

5

ISP1 ISP2

A B

Figure 2. Case study network

fixed set of buffer sizes. Respective capacities are: 150 at each ISP, 3000 at each LINK, 160 at

each EMAIL downloading line and 3000 at each FTP downloading time.

Conclusion

I have analysed the main aspects of the performance modelling of computer networks i.e. the

communication network. I have also analysed two main methods of modelling computer

networks and the associative solving techniques. Finally, a simple example has shown the

utility of performance modelling, and the way to proceed in order to reduce the debugging

difficulties.

References

John Daintith. "Performance model." A Dictionary of Computing. 2004. Retrieved May 12,

2014 from Encyclopedia.com:http://www.encyclopedia.com/doc/1O11-

performancemodel.html

Ramon Puigjaner. 2003. Performance modelling of computer networks. In Proceedings of the

2003 IFIP/ACM Latin America conference on Towards a Latin American agenda for network

research (LANC '03). ACM, New York, NY, USA, 106-123. DOI=10.1145/1035662.1035672

http://doi.acm.org/10.1145/1035662.1035672

Modeling and Simulation Glossary. Retrieved May 13 2014 from ACM SIGSIM: www.acm-

sigsim-mskr.org/glossary.htm

090805028

MATTHEW OMOLABAKE O.

CSC524 ASSIGNMENT

ANALYSIS OF SIMULATION OUTPUT

MAY 2014

1

INTRODUCTION

A computer Simulation is the discipline of designing a model of a real life or

hypothetical situation using a computer so that it can be studied to see how the

system works.

The greatest disadvantage of simulation is that you dont get the exact answers and results are only estimates. Therefore, careful design and analysis is needed to make

estimates as valid and prcised as possible and interpret their meaning properly. It

is important that the length of the simulation be properly chosen. If the simulation

is too short, the results may be highly variable. On the other hand, if the simulation

is too long, computing resources and manpower may be unnecessarily wasted.

Output analysis is the modeling stage of simulation that focus on the analysis of

simulation result. Output analysis gives estimate of system performance via

simulation cycle time. Statistical methods are used but its difficult to apply classical statistical techniques to the analysis of simulation output because simulations almost never produce raw output that is independent and identically

distributed normal data e.g

Customer waiting times from a queuing system:

(1) Are not independent: typically, they are serially correlated. If one customer at the post office waits in line a long time, then the next customer is also

likely to wait a long time,

(2) Are not identically distributed: customers showing up early in the morning might have a much shorter wait than those who show up just before the

closing time.

(3) Are not normally distributed: they are usually skewed to the right(and are certainly never less than zero)

The purpose of this analysis is therefore to give methods to perform statistical

analysis of output by

Estimating the standard error or confidence interval

Figure out the number of observations required to achieve desired error.

There are two types of simulations with respect to output analysis: terminating and

non-terminating (steady state). The type of analysis depends on the goal of the

study.

2

Terminating simulation is one where there is a specific starting and stopping

condition that is part of the model e.g. a bank with an opening time of 8.am and

closing time of 5pm. While a steady state simulation is one where there is no

specific starting and ending conditions e.g. an emergency room. Here we are

interested in the steady state behavior of the system.

Terminating simulations

Also known as transient simulation, it is one that runs for some duration of time TE

where E is the specified event (or set of events) which stops the simulation. When

simulating a terminating system, the initial conditions of the system at time 0 must

be specified and the stopping time or event TE must be well defined.

Examples:

A bank that operates from 9am to 4.30pm daily and starts empty and idle at

the beginning of each day. The output of interest may be average wait time

of first 50 customers in the system. Then TE =480mins

A military confrontation between a blue force and a red force. The output of

interest may be the probability that the red force loses half its strength before

the blue force loses half of its strength.

A communication system consists of several components. Consider the

system over a period of time, TE ,until the system fails, E = {A fails, or D

fails or (B and C both fail)}

The objective of analysis of terminating simulations is to obtain a point estimate

(sample mean) and confidence interval for some parameter (average time in system

for n customers, machine utilization, work-in-progress etc). Confidence interval for

terminating simulations usually uses independent replications.

Analysis for terminating simulations

Make n independent replications of the model

Let Yi be the performance measure from the ith replication

Yi = average time in the system, or

3

Yi = Work-in-progress

Yi = Utilization of a critical facility

Performance measures from different replications Y1, Y2 ,Yn are

independent and identically distributed

But, only one sample is obtained from each replication

Apply classical statistics to Yis, not to observations within a runs

Select confidence level 1-a(0.90,0.95,etc)

Approximate 100(1-a) % confidence interval for observations:

unbiased estimator

unbiased estimator of Var(Yi)

covers approximate probability (1 a)

is the Half-Width expression

Example:

Consider the single server (M/M/1) queue. The objective is to calculate a

confidence interval for the delay of customers in the queue.

n = 10 replications of single server queue

Yi= average delay in queue from ith replication

Yis = 2.02, 0.73, 3.20, 6.23, 1.76, 0.47, 3.89, 5.45, 1.44, 1.23

For 90% confidence interval

= 2.64, S2 (10)= 3.96 t9,0.95=1.833

Approximate 90% confidence interval is

4

2.64 1.15, or [1.49,3.79]

Interpretation

100(1-a) % of the time, the confidence interval formed in this way covers

between 1.49 and 3.79

Limitations

The analysis outlined above assumes that the distributions are normal or near

normal in form. If the distributions are seriously skewed (e.g Weibull distributions)

then calculated confidence intervals may deviate significantly from the calculated

(1-a) values.

Non-terminating or steady state Simulations

A non-terminating system is a system that runs continuously or at least over a very

long period of time. A steady state simulation is a simulation whose objective is to

study long run or steady state behavior of a non-terminating system.

The stopping time or event, TE, is determined not by the nature of the problem but

rather by the simulation analyst, either arbitrarily or with certain statistical

precision in mind.

Examples:

An emergency room

A communication system where service must be provided continuously.

An Atm machine

A manufacturing company that operates 16 hours a day. The system here is a

continuous process where the ending condition for one day is the initial

condition for the next day. The output of interest here may be the expected

long-run daily production

Problems that may arise

How should the simulation be started (initialization at time zero)

5

How long must it run before data representative of steady state can be

collected

A steady state condition implies that a simulation has reached a point in time

where the state of the model is independent of the initial startup condition. Before a

simulation can be run, one must provide initial values for all of the simulations

state variables. Since the experiment may not know what initial values are

appropriate for the state variables, these values might be chosen somewhat

arbitrarily.

For instance, we might decide that it is most convenient to initialize a queue as

empty and idle. Such a choice of initial conditions can have a significant but

unrecognized impact on the simulation runs outcome.

Thus, the initialization bias problem can led to errors particularly in steady state

output analysis.

Examples of problems concerning simulation initialization:

Visual detection of initialization effects is sometimes difficult especially in

the case of stochastic processes having high intrinsic variance such as

queuing systems

How should the simulation be initialized? Suppose that a machine stop

closes at a certain time each day, even if there are jobs waiting to be served.

One must therefore be careful to start each day with a demand that depends

on the number of jobs remaining from the previous day.

Initialization bias can lead to point estimators for steady state parameters

having high mean squared error as well as confidence interval having poor

coverage.

Since it raises important concerns, we have to detect and deal with it. We first list

methods to detect it.

Attempt to detect the bias visually by scanning a realization of the simulated

process. This might not be easy, since visual analysis can miss bias. Further,

a visual scan can be tedious. To make the visual analysis more efficient, one

might transform the data (e.g take logs or square roots), smooth it, average it

across several independent replications etc

6

Conduct statistical test for initialization bias: Various procedures check to

see if mean or variance of process changes overtime: change point detection

from statistical literature etc

If initialization bias is detected, one may want to do something about it. Two

simple methods for dealing with bias include

1. Truncate the output by allowing the simulation to warm up before

data are retained for analysis. Experimenter hopes that the remaining

data are representative of the steady state system. But how can one

fine a good truncation point? If the output is truncated too early,

significant bias might exist in the remaining data. If it is truncated

too late, then good observations might be wasted. A reasonable

practice is to average observations across several replications, and

then visually choose a truncation point based on the average run.

2. Make a long run to overwhelm the effects of initialization bias. This

method of bias control is conceptually simple to carry out and may

yield point estimators having lower mean squared errors than the

analogous estimators from truncated data. However a problem with

this approach is that it can be wasteful with observations; for some

systems, an excessive run length might be required before the

initialization effects are rendered negligible.

Analysis for steady state simulation

The objective: Estimate the steady state mean

v = limi E(Yi)

The basic question here is should you do many short runs or one long run?

Many short runs

The analysis is exactly the same as for terminating systems. The (1-a) %

confidence interval is computed before. The problem here is that because of

initial bias, the sample mean may no longer be an unbiased estimator for the

steady state mean.

7

Advantages

1. Simple analysis, similar to the analysis of terminating systems

2. The data from different replications are independent and identically

distributed

Disadvantage

1. Initial bias is introduced several time

One long run

Make just one long replication so that the initial bias is only introduced

once. This way, you will not be throwing out a lot of data. The problem

here is how you estimate the variance because there is only one run.

Advantages

1. Less initial bias

2. No restarts

Disadvantages

1. Sample size of 1

2. Difficult to get a good estimate of the variance

A number of methodologies have been proposed to solve the problem the

long run, they include

1. Batch means: The idea of batch means is to divide one long simulation

run into a number of contiguous batches, and then appeal to a central

limit theorem to assume that the resulting batch sample means are

approximately independent and identically distributed normal

The two important issues here are:

How do we choose the batch size k? We choose k large enough so

that the batch means are approximately uncorrelated.

How many batches n? Due to autocorrelation, splitting the run into

a larger number of smaller batches degrades the quality of each

individual batch. Therefore, 20 to 30 batches are sufficient.

8

Divide a run of length m into n adjacent batches of length k

where m =nk

Let Yj be the sample or batch mean of the jth

batch

The grand sample mean is computed

The sample variance is computed as

The approximate 100(1-a) % confidence interval

2. Standardized time series: One often uses the central limit theorem to

standardize independent and identically distributed random variables into

an (asymptotically) normal random variable.

Schruben and various colleagues generalized this idea in many ways

using a process central limit theorem to standardize a stationary

simulation process into a Brownian bridge process.

Properties of Brownian bridges are then used to calculate a number of

good estimators for Var(Yn) and confidence interval for E. This method

is easy to apply and has some asymptotic advantages over batch means.

3. Spectra analysis: This approach operates in the so-called frequency

domain, whereas batch means uses the time domain.

4. Regeneration: Many simulations can be broken into independent and

identically distributed blocks that probabilistically start over at certain

9

regeneration points. This method effectively eliminates any initialization

problems but on the other hand it may be difficult to define natural

regeneration points, and extremely long simulation runs are often needed

to obtain a reasonable number of independent and identically distributed

blocks.

10

References

Dave Goldsman (May 26 2010). Simulation output analysis

Analysis of Simulation Experiments.Output-analysis.ppt from

http://www.cs.bc.edu

Output Analysis for a single Model. Lecture-9.ppt from

http://www.personal.Cityu.edu.hk

Wiley Computer Publishing, John Wiley & Sons, Inc. (1991). .Art of Computer

Systems Performance Analysis Techniques for Experimental Design Measurements

Simulation and Modeling. Raj Jain

CSC524OGBUAGUNKECHINYEREOGONNA

ANALYSISOFSIMULATIONOUTPUTDATAUSINGSTATISTICALANALYSIS

TableofContentsINTRODUCTION.......................................................................................................................................2WhatisSimulation?.............................................................................................................................2WhyDoWeSimulate?.........................................................................................................................2WhatistheProblemofSimulations?..................................................................................................2WhatistheAim?.................................................................................................................................2Whatmaywewanttoknowaboutthesystem?.............................................................................