AnySee TPDS Delay

8/13/2019 AnySee TPDS Delay

1/12

IEEE TRANSACTIONS ON PARALELL AND DISTRIBUTED SYSTEMS, TPDS-0225-0806 1

Scalable Live Streaming Service Based onInter-Overlay Optimization

Xiaofei Liao, Member, IEEE , Hai Jin, Senior Member, IEEE,

Yunhao Liu, Senior Member, IEEE, and Lionel M. Ni, Fellow, IEEE

Abstract In order to provide scalable live-streaming services, we propose an Inter-Overlay Optimization scheme, IOO. Instead ofselecting better paths in the same overlay, IOO constructs efficient paths using peers in different overlays, so as to (i) improve globalresource utilization of P2P streaming networks; (ii) assign resources based on their locality and delay; (iii) guarantee streamingservice quality by using the nearest peers, even when such peers might belong to different overlays; and (iv) balance the loadamong the group (streaming overlay) members. We compare the performance of IOO with existing approaches through trace drivensimulations. Results show that IOO outperforms previous schemes in terms of resource utilization and the QoS of streamingservices. IOO scheme has been implemented in an Internet based live streaming system, called AnySee. AnySee was successfullyreleased in the summer of 2004 in CERNET of China. Over 60,000 users enjoy massive entertainment programs, including TVprograms, movies, and academic conferences videos.

Index Terms Peer-to-peer, live streaming, inter-overlay optimization, startup delay, P2P topology discovery.

1 INTRODUCTION

ith the improvement of network bandwidth, mul-timedia services based on streaming live media havegained much attention. Significant progress has

been made on the efficient distribution of live streams in areal-time manner over a large population of spectators withgood QoS [8][9]. Due to the practical issues of routers, IPmulticast has not been widely deployed. Therefore, re-searchers have expended a lot of efforts building efficientoverlay multicast schemes based on P2P models, in whichspectators behave as routers for other users. Scalable live-

streaming overlay constructions [5] become a hot topic.Different from traditional distributed systems, streamingoverlays focus on the following four metrics: startup delay,the delay to display the first image for users, including thelogical media dada transmission time and decoding prepar-ing time; source-to-end delay, the delay to receive the firstmedia data block directly or indirectly from the source;playback continuity, the number of segments that arrive

before playback deadlines over the total number of thesegments; and resource utilization, the ratio between theused bandwidth to the total bandwidth. These metrics havea direct bearing on the interactive usability of a live stream-ing system. Large delays would exhaust user patience andunplanned interruptions will spoil the entertainment value.

In order to improve the above metrics, previous studies[6] focused on intra-overlay optimizations, in which each

node joins at most one overlay. With the help of locality-aware strategies [7] and optimization schemes such as DO-Net in Coolstreaming [8], Narada in ESM [9], the QoS oflive streaming P2Ps have significantly improved. Neverthe-less, they still suffer from long delay and unplanned inter-ruptions, especially when a large number of peers join thenetwork simultaneously.

Fig. 1. Intra-overlay optimization: (a) optimal multicast tree rooted at S 1;(b) optimal multicast tree rooted at S 2; (c) physical topology.

Figure 1 shows an example of intra-overlay optimizationwith two logical streaming overlays. Peers A, B, C and D

join the stream originating at S 1 and peers E, F, G, H and K join the stream originating at S 2. The number on each edgerepresents the cost of the link between two nodes. In tradi-tional intra-overlay optimization schemes, two multicasttrees are established as shown in Fig. 1(a) and Fig. 1(b), re-spectively. There are two obvious drawbacks. First, suchoverlay construction is not globally optimal. Considering

Xiaofei Liao and Hai Jin are with the School of Computer Science and Tech-nology, Huazhong University of Science and Technology, Wuhan, China.E-mail: [email protected], and [email protected].

Yunhao Liu and Lionel M. Ni are with the Department of Computer Sci-ence and Engineering, Hong Kong University of Science and Technology,Kowloon, Hong Kong. E-mail: [email protected], and [email protected].

Manuscript received 11, August 2005; revised 19,April 2007; accepted 15, May2007. Recommended for acceptance by Denis Trystram. For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference IEEECS Log Number TPDS-0225-0806.

Digital Object Indentifier 10.1109/TPDS.2007.70708 1045-9219/$25.00 2007 IEEE

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMSThis article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.


2/12

2 IEEE TRANSACTIONS ON PARALELL AND DISTRIBUTED SYSTEMS, TPDS-0519-1205

peer D in Fig. 1(a), the cost S1 D is 8, while if the path S1 S2 D is used, the cost is only 4. Second, resource utili-zation of traditional approaches is relatively low and im-

balanced. Resources of edge peers cannot be utilized toprovide services for other peers. Most of the existing proto-cols are tree based. Consequently, all leaf nodes fail to con-tribute any bandwidth or CPU cycles to the multicast trees.We propose an inter-overlay optimization scheme, IOO, in

which resources can join multiple overlays simultaneously,so as to (i) improve global resource utilization of a P2P livestreaming network; (ii) assign resources based on their lo-cality and delay; (iii) guarantee streaming service quality byusing the nearest peers, even if such peers might belong todifferent overlays; and (iv) balance the load among thegroup (streaming overlay) members to avoid the traffic bot-tlenecks. After IOO optimization on the example shown inFig. 1, better overlays can be constructed as illustrated inFig. 2.

Fig. 2. IOO: inter-overlay optimization.

For a distributed approach like IOO, however, to reachthe above design goal without global network knowledge,

however, is not trivial. Several key issues, including effi-cient neighbor discovery, resource assignment, overlay con-struction and optimization, must be addressed.

To prove the effectiveness of IOO scheme, we conductcomprehensive trace driven simulations based on topolo-gies obtained from real P2P networks. Results show thatIOO scheme outperforms previous schemes in resourceutilization and the streaming QoS. A public implementa-tion, AnySee v.1.1 [2], was released in June 2004. It has beenused to broadcast live-streaming media, including TV pro-grams, movies, and the 2004 International Conference onGrid and Cooperative Computing held in Wuhan, to tens ofthousands of end-users in China Education and Research

Network (CERNET). During the last months, over 60,000users (till March 2006), from 40 universities and 20 cities inChina, have tested AnySee streaming services. The source-to-end delay, resource utilization, and the startup delaywere all quite encouraging.

The rest of this paper is organized as follows. Section 2discusses the related work. Section 3 presents the inter-overlay optimization and IOO. Section 4 describes oursimulation methodology and performance analysis. Wedescribe implementation experiences and show our obser-vations and measurements on AnySee in Section 5. We con-clude this work in Section 6.

2 R ELATED WORKMany efforts have been made in the field of P2P livestreaming system design. The most important issue of suchschemes is to design a streaming overlay with high servicequality and good scalability. In general, there are threetypes of schemes for optimizing live streaming overlays:tree-based [12][13], mesh-based [8][15][19] and hybrid over-

lays [20][21].2.1 Tree-basedBorrowing ideas from IP multicast, tree-based protocols aresimple and easy to implement. The main target of tree-

based approach is to construct overlay spanning trees di-rectlythat is, members explicitly select their parents fromthe members they know. There are two types of tree-basedprotocols, including single tree protocols, such as ESM,ALMI [13], NICE [17] and Zigzag [16], and multiple treeprotocols such as SplitStream [10], Bullet [11], and MDC[12].

Single tree protocols focus on building a scalable multi-cast spanning tree with high efficiency. In ESM, nodes areorganized into an overlay tree over the existing IP networkand streaming data are distributed along the tree. Band-width requirement for media server is shared among thepeering nodes, so as to reduce the burden of the mediaserver. In ESM, the leaf nodes of a distribution tree onlyreceive streaming media, and do not contribute to the con-tent distribution. ALMI [13] employs a topology maintain-ing center to collect topology information of trees. Any peerintending to join a group must first contact the center to getcorresponding group information, and then choose one asits parent. NICE [17] and ZIGZAG [16] first organize peersinto different logical layers. All peers belonging to the 0-thlayer are divided into different clusters via their networklocations. One leader is selected from a cluster in the layer iand automatically becomes the member of a cluster in thelayer i + 1. The source is the leader of the highest layer.Clearly, no leaf peer supports services for others. Consider-ing that the up-stream bandwidth of one peer on theInternet is limited in general, resources, especially band-width, of some leaf peers in these systems are under-utilized.

Multiple tree protocols emphasize the overall resilienceand the load balance of the entire streaming network. Themain idea is to divide the video of one stream into severalparts based on layer concept in CoopNet or patchingideas [18]. The leaving or crash behavior of upper layer

nodes, however, often causes buffer underflow. They alsofail to provide backup services and waste spare resources.

2.2 Mesh-basedIn mesh-based protocols, peers accept media data frommultiple parents as well as providing services for multi-ple children, such as Coolstreaming [8], PROMISE [15]and GNUStream [19]. For example, PROMISE [15], basedon CollectCast, operates at the application level but alsoinfers and exploits properties including topology and per-formance of the underlying network. CollectCast reflectsthe P2P philosophy of dynamically and opportunisticallyaggregating the limited capacity of peers to perform



3/12

LIAN ET AL.: GRADIENT BOUNDARY DETECTION FOR TIME SERIES SNAPSHOT CONSTRUCTION IN SENSOR NETWORKS 3

( ), , 2,dm id MS body

( ), ,1,dm id MS body

Fig. 3. Concept structure of IOO. Fig. 4. Roadmap of detector message initiated by peer MS .

streaming tasks traditionally performed by media servers.The resource utilization of a mesh is higher than that of atree. Meshes based on Gossip protocol can find fresh peersin the single mesh with low management overhead, but notin global P2P networks. Due to the random algorithm forneighbors selection, the startup delay cannot be guaran-teed. Also, to decrease the impact from autonomy of peers,a large buffer space, as used in Coolstreaming [8], is oftennecessary.

2.3 Hybrid overlaysZhang et al proposed a DHT based P2P resource pool,SOMO [20], to manage global resources and optimize mul-tiple ALM (Application Layer Multicast) sessions. The mainidea is to structure all peers strictly [21], ignoring the fea-tures of specific applications. The huge maintenance over-head, however, makes these approaches far from scalable.Indeed, even if we have global knowledge of a P2P net-

work, finding an optimal assignment of resources is NP-hard.Based on a completely distributed heuristic, our proposed

IOO scheme selects streaming paths and uses backup linksor peers potential providers. Inter-overlay optimization isemployed to complement traditional intra-overlay strate-gies.

3 D ESIGN OF S CHEMETo achieve good performance in P2P live streaming sys-tems, IOO scheme faces the following challenges: (i) to findpaths with low delays, including source-to-end delay andstartup delay, in a large-scale P2P network; (ii) to maintainthe service continuity and stability, decreasing the impact offlash crowd and churn caused by peers leaving and joining,and network fluctuations; (iii) to determine the frequencyof optimization operations; and (iv) to reduce the controloverhead.

3.1 OverviewAs illustrated in Fig. 3, the basic workflow of IOO scheme isas follows. First, an efficient mesh-based overlay is con-structed. A location detector based algorithm is employedto match the overlay with the underlying physical topology[22]. Second, the single overlay manager, which is based ontraditional intra-overlay optimization, such as Narada [9],

DONet or nearcast [26], deals with the join/leave opera-tions of peers. Third, the inter-overlay optimization man-ager explores appropriate paths, builds backup links, andcuts off paths with low QoS for each end peer. Fourth, thekey node manager allocates the limited resources, and the

buffer manager manages and schedules the transmission ofmedia data.

3.2 Mesh-based Overlay ManagerIn IOO, all peers belonging to different streaming overlayswill join one substrate, mesh-based overlay first, to con-struct an under layer. In this layer, all peers can be reachedeach other directly or indirectly. The function of the mesh-

based overlay manager is to provide information about theconnections for the inter-overlay optimizations. Every peer,with a unique identifier, first connects the bootstrappingpeers and selects one or several peers to construct logicallinks. Every peer maintains a group of logical neighbors.

The key issue here is to let the mesh-based overlay matchwith the underlying physical topology [23]. The mesh- based overlay manager, a key component of IOO, uses ascheme called as LTM (Location-aware Topology Matching)technique [22], to optimize the overlay, find the closestneighbors, and eliminate slow connections. There are twomajor operations: flooding-based detection with limitedTTL, and updating logical connections.

In the first operation, each peer periodically floods a mes-sage, defined as dm(id, MS, TTL, body), to its neighbors. Themessage dm(id, MS, TTL, body) means that the peer MS ini-tiates a message with id value to peers at most TTL hopsaway. Since our purpose is to find the closest neighbors ofpeer MS, we define TTL=2. To detect the distance of peers,the message body has four parts, including sourceIP (the IPaddress of the source peer), sourceTimestamp (the time-stamp 1 when the source forwards the message), DirectIP(the IP address of one neighbor within one hop) and Di-rectTimestamp (the timestamp when the neighbor withinone hop gets the message). Figure 4 shows the roadmap ofone message from MS. Obviously, a message is broadcast todirect neighbors and 2-hop away neighbors.

1The clocks of all peers are synchronized based on NTP. Current imple-mentation of NTP version 4.1.1 in public domain can reach the synchroniza-tion accuracy down to 7.5 milliseconds [24].



4/12


In the second step, logical links are updated. With thehelp of the timestamps on peers, peer P1 compares the dis-tance between two paths, MS P1, MS N1, and N1 P1. Ifthe former length is smaller, the link N1 P1 will be cut offand the direct path between MS and P1 is established. Allpeers do the same operations as those of peer MS. Afterseveral operations, peers would connect with their closeneighbors [22].

3.3 Single Overlay ManagerThe single overlay manager is responsible for peers leavingand joining operations based on the under layer, mesh-

based overlay. Before inter-overlay optimization, one peer joins one streaming overlay and receives media contentsfrom multiple providers or single provider according tointra-overlay optimization schemes. Here we apply onescheme, named "locality-aware" tree based overlay con-struction scheme [26] to build optimized single overlay.

Generally, each peer maintains one active streaming pathset and one backup streaming path set. Initially all stream-ing paths are managed by the single overlay manager.

Specifically, when a peer P is subscribing the media datafrom media source MS, MS will return the streaming raterate(MS) of the media program to peer P. Then P maintains(i) an active streaming path set with thresholdsize ( ), a P MS , and (ii) a backup streaming path set withthreshold size ( ), b P MS . For the i-th streaming path iSP from MS to P, it has two parameters: delay(SPi, MS, P) is thesource-to-end delay from source MS to peer P on the path

iSP . When a media block is delivered from the mediasource to P on the path iSP , the single overlay managerrecords the timestamp and stores it into the media blocksheader. Upon receiving the media block with the initialtimestamp, P computes the difference of the initial time-stamp and the arriving timestamp. The little difference isthe value of delay(SPi, MS, P), and rate(SPi, MS, P) is thestreaming rate in the last hop of the path. Clearly, we have:

( )( )

( )

( )( )

( )( )

,

1

, ,

1 1

, ,

, , , ,

a

b a

P MS

ii

P MS P MS

i ii i

rate SP MS P rate MS

rate SP MS P q rate SP MS P

=

= =

= (1)

In Eq.(1), the parameter q means that how many times thetotal downloading bandwidth from backup streaming pathset is bigger than that from active streaming path set. Let

( ) D MS denote the threshold for the delay, which is relatedto the requested quality of the streaming service only. Thehigher the requested quality of streaming service, the lessthe threshold of delay will be. In this design, a new attrib-ute is introduced called LastDelay, which is the minimal ofall source-to-end delays from the current node to thestreaming source on different paths. We have,

( )( ), ,1

i

a

Las tDelay Min delay SP MS P

i

=

(2)

With LastDelay, each path to the media source can bemeasured. Peers can evaluate QoS of streaming paths andadjust the directed neighbors according to LastDelay. Theadjustment of streaming path set can be done with the

management of direct connected neighbors in the sameoverlay. Using Gnutella protocol, peers find new neighborcandidates. Comparing pair delays, the fastest neighborsare selected to build direct connections from the candidates.Based on the streaming path sets of all directly connectedneighbors, new streaming path set can be created. Becauseeach direct connected neighbor has maintained

( ) ( ), ,

a b x P MS P MS = +

streaming paths and the delays toall y neighbors can be calculated, the new streaming pathswith smallest source-to-end delays can be selected after thefollowing calculations:

( )

( ) ( ) ( )

1,1 1,2 1, 1 1 1

2,1 2,2 2, 2 2 2

,1 ,2 ,

, , , , , ,

, , , , , ,,

, , ,, , ,

x

x y y

y y y y y y x

y

D D D D D D

D D D D D D DS x DS

D D D D D D

DS y x DS x DS y

= =

= +

(3)

In Eq.(3), the parameter ,i j D denotes the source-to-end

delay on the streaming path j of neighbor i, i D denotes the

direct delay to the neighbor i, ( ) y DS x is the delays matrixof all neighbors streaming paths. ( ) DS y x is the delaysmatrix of all potential streaming paths. From y x stream-ing paths, we can choose ( ),a P MS active streaming pathsand ( ),b P MS backup streaming paths.

3.4 Inter-overlay Optimization ManagerWhen the number of backup streaming paths is less than athreshold, the IOO is conducted to find appropriate stream-ing paths in the global P2P network with the help of themesh-based overlay. When one active streaming path is cutoff, a new streaming path is selected from the backup set.

Fig. 5. Structure of the message ProbM .

We also design a probing message named ProbM asshown in Fig. 5. This message includes two major parts: (i)initial information, including Seq.: sequence number, Peer_0:initial peer ID, Timestamp0: message issuance time, Source:media source ID of the initial peer, LastDelay, current valueof LastDelay, and TTL: current value of TTL; (ii) an arraywith the size of TTL to record peer ID and the arrivingtimestamp of the message. Considering that 95% of peers inthe Gnutella system could be reached within 7 hops bypure flooding [23], the maximal TTL is set to 7.



5/12

LIAN ET AL.: GRADIENT BOUNDARY DETECTION FOR TIME SERIES SNAPSHOT CONSTRUCTION IN SENSOR NETWORKS 5

Fig. 6. An example of reverse tracing algorithm: (a) each peer forwardsto three neighbors; (b) each peer forwards to two neighbors.

The inter-overlay optimization manager mainly has twotasks, backup streaming path set management and activestreaming path set management.

Managements of the backup streaming path setThe major operation in this component is a probing pro-

cedure, called reverse tracing algorithm. This algorithmstarts when the size of the backup set is less than ( ),b P MS .Peer_0 sends out a ProbM message to j of its randomneighbors with the recording array empty. Each receiverrecords the message arrival time and its ID into the mes-sage body of the received message, as shown in Fig. 5. Thereceiver will stop forwarding the message if (i) it finds thatthe delay from the initial peer Peer_0 to this peer is greater

than LastDelay; or (ii) the receiver is the source of thisstreaming service. Otherwise, the message would be for-warded to j of its random neighbors.

With the help of reverse tracing algorithm, the mediasource is able to analyze the arrived messages with ID (Seq.)periodically, and explore the best path from the source tothe message issuance peer. Informed by the source, the peerthen constructs an overlay path accordingly. Figure 6 showsan example of the reverse tracing algorithm based on theoverlay shown in Fig. 1, when j=3 and j=2, respectively. Inthis figure, all delays are replaced with the cost of twopeers. Peer D sends out a message and the possible routesof the message are shown. Some routes are cancelled due toa longer delay than LastDelay. Eventually, a good path S

1

S2 D is found. Then LastDelay is updated. As a large por-tion of ProM messages are stopped during forwardingprocess, the overhead is acceptable.

A streaming path of Peer D is treated as invalid if (i) thesource-to-end delay is larger than a given threshold ( ) D MS ,or (ii) the direct parent of Peer D on the path leaves. In thisdesign, we only disconnect the overlay link between theend peer to its parent node. There are two reasons: (i) theother connections on the path can be reserved to providesupport for new incoming peers, and (ii) frequent discon-nections incur a lot of unnecessary traffic.

Managements of the active streaming path set

There are three operations, maintaining the states of ac-tive streaming paths, cutting off invalid paths, and addingnew active paths from a backup set. When the total bit ratesfrom active streaming paths are lower than rate(MS), themanager will check whether a better path should be acti-vated to replace the current one.

This manager has the following characteristics: (i) it em-ploys a heuristic algorithm to optimized the system step bystep; (ii) probing procedures are originated from the normalpeers, not the source peer, so that the control overhead can

be distributed; (i ii) the number of forwarding neighbors, j,is chosen to balance the tradeoff between the effectivenessand the overhead; and (iv) the frequency of probing andoptimization is dynamic. In IOO, the probing procedure isfeedback-driven based on delay. Log data from AnySee, theprototype of IOO scheme to be discussed in Section 5.4,show that when peers are watching highly popular movies,they are willing to tolerate a larger delay. Hence, it is rea-sonable that different programs define different ( ) D MS .

3.5 Key Node Manager

Admission control is useful when there are too many re-quests. The goal of the key node manager is to determinethe number of requests that a peer should have. Suppose apeer has Q unused connection slots. All requests will beclassified into W types according to the different populari-ties of the target media, and each falls into one of W queues.When we assign the Q slots to W queues, there are two in-teresting cases. First, some queues are assigned with morethan one connection slot, which can be modeled as anM/M/m/K queuing model. Second, some queues onlyreceive one connection, which follows an M/M/1/K queu-ing model [25].

Our goal is as follows. Suppose the arriving rate of queue

j is j , and all arriving rates satisfy 1 ... ... j W . As-sume the service rate to assign one connection slot is andeach connection processor can buffer k requests ( k 1). If theprobability that n requests follow the M/M/m/K queuingmodel is n p , we have

( )0

0

0,1 1!

, 1!

n

n m n

m p n m

n pm

p n m m K m

=

= = +

(4)

wherem

= ; we also have

( ) ( )

( ) ( ) ( )

11

1

0

0 11

0

1 1! ! 1

1 1! !

i m K mm

i

i mm

i

m mi m

pm m

K mi m

+

=

=

+

=

+ + =

(5)Thus, the average utilization of W spare connections of

one peer can be given by:

( ) 01 1!

m K m pk

m

= =

(6)

One connection processor can buffer k requests,



6/12


then K mk = . When the probability that n requests are fol-lowing the M/M/1/K queuing model is 'n p , we have

( )1

'

11

1 01

11

n

K

n p n K

K

+

=

=+

(7)

and

= . Then the average utilization of Q spare connec-

tions of one peer can be given by~

'0 1

11

1

K

K p

+

= =

(8)

and ' K p is the failure probability of requests. Then, the tar-get, resources utilization, can be expressed:

( )( )~

1 21 ,

1

, ,...

1

i k

W i ji j W

W

i ii

Max Q Q Q Max

Subject to Q Q Q Q

=

= +

=

Documents

AnySee TPDS Delay