DHT-Based Peer-to-Peer Search System UsingPseudo-candidate Key Indexing

Yutaka Arakawa, Hiroya Minami, Masato Matsuo, Masayasu Yamaguchi, and Hiroshi Saito

NTT Network Innovation Laboratories, NTT Corporation, Musashino, 180-8585 Japan

SUMMARY

In service in the ubiquitous networking environment,it is expected that flexible interaction between the user andthe things (resource) in the real world will become impor-tant. In order to realize such an interaction, there must be asystem in which the resource to be utilized can be searchedfor with its attribute information as keys. Consequently, thispaper proposes a DHT (distributed hash table)-based P2Psearch system using a new indexing method called pseudo-candidate key indexing. A problem of the conventionalDHT-based P2P search system is that a large amount ofcommunication is needed in the search process. The pro-posed system attempts to deal with the problem by prepar-ing an index for all search conditions which are useful tothe user, thus reducing the amount of communication in thesearch. A simulation verifies that the proposed system isscalable. © 2007 Wiley Periodicals, Inc. Electron CommJpn Pt 1, 90(11): 81–94, 2007; Published online in WileyInterScience (www.interscience.wiley.com). DOI10.1002/ecja.20317

Key words: DHT; P2P; pseudo-candidate key in-dexing; search system.

1. Introduction

With the spreading use of RFID tags and sensors, itis anticipated that networks will be fully occupied with theinformation concerning things in the real world. Such a“thing in the real world” is called a resource in this paper.

We intend to develop a technique which can flexibly findand utilize resources needed by the user. Specifically, con-sider a request such as “I want to be connected to a tele-phone which is presently located close to Mr. A and is idle”or “I want to know the present location of the red bag Ibought last week.” We consider a system that tries to satisfysuch requests by using attribute information representingthe properties or the state of the resource as the search key.

In the cases of RFID tags and sensors, it is expectedthat unique IDs will be assigned. Then the ID can be usedas the search key, but it is not realistic to assume that theuser will remember the individual ID. For the convenienceof the user, it is desirable that the resource be searchable byusing attribute information directly representing the re-source, rather than using an ID which does not carry anyimplications.

The following points are especially important in de-veloping our system.

(1) The system must be able to handle a large amountof resources, which will increase further in the future.

(2) The appearance or disappearance of resources,and also their change of state, should be reflected in thesystem in real time.

When the idea is applied, for example, to SCM (sup-ply chain management) in order to realize a search over anentire business, the number of resources will be larger thanthe present number of Web pages. At present, varioustechniques are used in many Web search engines to speedup information updating, but the system has not reached thestage in which all resources can be updated in real time. In

© 2007 Wiley Periodicals, Inc.

Electronics and Communications in Japan, Part 1, Vol. 90, No. 11, 2007Translated from Denshi Joho Tsushin Gakkai Ronbunshi, Vol. J88-B, No. 11, November 2005, pp. 2158–2170

81

order to realize such a system, the configuration shouldinclude more distributed parallel structures.

The configurations of distributed parallel systems canbe broadly divided into the hierarchical type and the P2Ptype (horizontally distributed type). Typical hierarchicalsystems are DNS and the ONS (object name service) of ePCglobal. This approach has already been recognized as ascalable structure. However, a problem of hierarchicalstructures is that search processing cannot be made efficientunless the search key is matched to the structure. Thehierarchical structure exhibits advantages, since domainname is used as a key in the case of DNS, and the manufac-turer/product classification is used as a key in the case ofONS.

On the other hand, we consider the case in which anyof the registered information has the possibility to be usedas the key, and it is impossible to specify the search keybeforehand. In such a case, it is impossible to realize anadequate hierarchical structure. Consequently, we considerefficient search by a flatter structure, that is, P2P, where thesystem components are related to each other on an equalbasis.

P2P search systems, which have recently been attract-ing interest, have desirable properties in terms of scalabilityand real-time information, making them promising for re-alizing the proposed system [1]. When this system is ap-plied to SCM, the proposed system can be added to theproduct management database owned by each businessorganization so long as the system is of P2P type. Thus,existing facilities can be utilized and a system configurationwith low introduction cost can be realized.

P2P search systems can be broadly divided into twotypes according to the segmentation of the index. In one,the index is segmented for each resource, and in the other,the index is segmented for each keyword. From the view-point of query routing, the former is also called a flooding-based P2P search system, and the latter is also called a DHT(distributed hash table)-based P2P search system. DHT isa technique that manages the hash table in a distributed wayin multiple peers (hosts). Typical systems include CAN [2],Chord [3], Pastry [4], and Tapestry [5]. The floodingscheme is better when there are many copies, as in the caseof file-sharing, but DHT is better if multiple copies are notassumed. The resource in the real world is not copied, butindividual resources should be clearly identified. Conse-quently, it is expected that DHT will be better suited thanthe flooding scheme to searches for resources in the realworld.

However, the following problem has been pointed outin DHT-based P2P search systems. Since the index is seg-mented for each keyword, the amount of peer-to-peer com-munication is increased for complex search queriesincluding multiple attribute information, which increasesthe amount of communication in the search process [6, 7].

In order to deal with this point, this paper proposes anew indexing method called pseudo-candidate key index-ing, which can reduce the upper bound for the amount ofcommunication needed for each search query. In the pro-posed method, the number of resources in the search resultis reduced below a certain value. It then becomes possibleto reduce the upper bound for the amount of communicationneeded for each search query in solving AND conditions(search conditions based on a logical expression in whichterms are connected by the AND operator, which is writtenas ∧ below), based on multiple attribute information items.

Section 2 presents the idea on which the solution ofthe problem is based. Section 3 describes pseudo-candidatekey indexing in detail and outlines the system. Section 4evaluates and discusses the proposed system. Section 5discusses related studies.

2. Problems and Approach Toward Solution

2.1. Problems in DHT-based P2P searchsystem

Consider, as an example, the case in which the net-work camera of X Co. is to be searched for. Suppose thatthe search is performed on the basis of an AND conditioncomposed of two attribute information items, that is, “prod-uct of X Co.” and “network camera.” In DHT, there is a peerholding the resource information connected to the condi-tion “product of X Co.” (called the peer covering theproducts of X Co.), and also a peer holding resource infor-mation connected to the condition “network camera”(called the peer covering network cameras). Due to therandomness of the hash value for each condition, thesepeers are usually different.

Consequently, in order to derive information for re-sources corresponding to an AND condition, the set opera-tion which yields the product of the resource set of “productof X Co.” obtained from the peer covering “products of XCo.” and the resource set of “network camera” obtainedfrom the peer covering “network cameras,” must be per-formed. In order to achieve this, there must be communica-tion between the peer covering the products of X Co. andthe peer covering network cameras, so that the informationon the resource set retained in one side (called intermediatesolution) is sent to the other. Even if only a few resourcesare obtained as a result of forming the product, a tremen-dous amount of communication is required to derive them.

Communication of intermediate solutions does notexist in the flooding-based P2P search system, but is inher-ent to DHT. It causes the problem to DHT that a largeamount of communication is required in the search process.

82

2.2. Restriction of index

The amount of communication required in the searchgreatly depends on the choice of the search key. When thered bag of Mr. A is to be sought, for example, the numberof search results depends greatly on whether the search isperformed with the condition “red” or with the name of Mr.A as the condition. In the former, the search result istremendous, since any red resource is a solution. In thelatter, the search result is restricted to some extent, sinceonly resources related to Mr. A are sought, decreasing theamount of communication. To reduce the amount of com-munication required in the search, it is effective to use onlya search key which is effective in restricting the range, asin the latter case.

When there are too many search results, the usercannot directly utilize the results anyway, and the searchmust be performed again by using a stronger search condi-tion. Thus, even if the solution is presented in such a search,after spending tremendous computer and network re-sources, it is meaningless to the user. On the other hand, bypreparing a mechanism by which the system does notanswer queries for which the search results are not wellrestricted, computer and network resources can be utilizedeffectively, and the user will be guided to an effectivesearch.

Consequently, the system is constructed so that anindex is prepared only for search keys for which the searchresults are restricted to less than a certain number.

2.3. Indexing for AND condition

When only the mechanism in the previous section isprovided, however, the user must use a search key such thatthe result is sufficiently restricted by using only an item ofattribute information. This is a problem in terms of userconvenience. The previous example is the case in which,even though both “product of X Co.” and “network camera”correspond to a larger number of resources, the result couldbe well restricted if their AND condition is formed. Such asearch result is useful to the user, and the search systemshould comply with this kind of query.

For search keys that yield a large number of results ifonly an item of attribute information is used, but the resultis useful if multiple items of attribute information arecombined, the combination itself is defined as the searchkey and is included in the index. By using this mechanism,the range of search keys that the user can use is enlarged,and the convenience for the user is improved. Based on thisidea, AND conditions which can be useful search keys, suchthat the number of search results is restricted to less than acertain number, are registered in the index.

2.4. Use of sequential search

The search process is broadly divided into two ap-proaches. One consists of preparing the index beforehand,and the other consists of matching the pattern sequentiallywithout using an index. When a large amount of data is tobe handled, an index must be used, but the latter approachis sometimes simple and fast if a small amount of data is tobe handled.

It was noted above that in the former, the ANDconditions for which the search result is restricted to lessthan a certain number are registered in the index. Since thereis a large number of such AND conditions, there will arisea new problem of index explosion, instead of the problemof the amount of communication. Consider, as an example,an AND condition, such as “product of X Co. ∧ networkcamera,” where the number of results is restricted to acertain number; here there can be another AND conditionin which the number of results is restricted to a certainnumber, that is, “product of X Co. ∧ network camera ∧manufactured in (month) of (year).”

Consequently, the system is designed as follows. Notall of the AND conditions for which the number of resultsis restricted to a certain number are registered in the index.Only the AND conditions for which the results are notrestricted to a certain number if any of the terms is missingare registered in the index. By using this mechanism, ex-plosion of the index size can be avoided.

When the search is performed by the AND condition“product of X Co. ∧ network camera ∧ manufactured in(month) of (year),” the search result can be restricted to acertain number even if the search is performed by thecondition “product of X Co. ∧ network camera,” which iscontained in the query. After this search, it suffices to checkeach of the search results which are less than a certainnumber sequentially to see whether “manufactured in(month) of (year)” is satisfied. Since it is guaranteed thatthe number of objects of check is less than a certain number,the processing load is also kept within a certain amount.

2.5. Introduction of pseudo-candidate key

Based on the reasoning up to this stage, the conceptof “pseudo-candidate key” is introduced.† The search keydefined as the pseudo-candidate key is one satisfying thefollowing two conditions.

Condition 1: The number of corresponding resourcesis not greater than T.

†In a relational database, an attribute or a set of attributes which canidentify uniquely a tuple (row) in the table in which the identificationceases to be unique if any of the attributes is missing is called a candidatekey. A pseudo-candidate key is a term which is close to that concept.

83

Condition 2: If (in the case of AND condition) any ofthe terms is missing, the number of resources cannot berestricted to T or less.

T is called the threshold. Whether the consideredsearch key is useful is decided in terms of this value. Thepseudo-candidate keys are registered in the index. In thesearch, the pseudo-candidate key contained in the searchcondition is found and the set of registered resources ischecked sequentially. This approach is intended to realizea search system which is convenient for the user whilereducing the amount of intermediate communication.

3. Outline of System

This section describes a DHT-based P2P search sys-tem using pseudo-candidate key indexing. The pseudo-can-didate key indexing method is an algorithm in which thepseudo-candidate key is extracted from the set of resourceattribute information and is registered in an index in theregistration stage, and pseudo-candidate keys are extractedfrom the search condition and the resource is searched fromthe index in the search stage. The method can be used notonly in hash index structures, but also in other structuressuch as the B-tree. The method can be applied in the sameway to both stand-alone databases and distributed systems.

Figure 1 outlines the system. It is assumed that vari-ous resources have unique IDs, and also diversified attributeinformation. As shown in the figure, a huge hash table(simply called a table in the following) is prepared to holdthe information on the set of resources, which is managedin a distributed way by multiple peers forming the DHT.

In the proposed system, two huge tables are distrib-uted and managed by DHT. One of the tables has an indexwith the resource IDs as the keys, and the set of resourceattribute information as the values (called a forward re-source table). The other has an index with pseudo-candidatekeys obtained from the set of resource attribute informationas the keys, and the set of resource IDs corresponding to the

pseudo-candidate keys as the values (called a reverse re-source table).

When we wish to ascertain the attribute informationof the resource with the resource ID as the search key, theforward resource table is used. Conversely, when we wishto ascertain the resource ID with the attribute informationas the search key, the reverse resource table is used. Figure2 shows an example of the forward and reverse resourcetables.

In the reverse resource table, a threshold (T) is definedfor the number of IDs that can be values for each key, andit is not allowed to retain a larger number of IDs, which iscontrary to condition 1 for the pseudo-candidate keys.Figure 2 shows the case of T = 3. In each tuple (row), thedata area for the “overflow flag” is prepared. When a try ismade to register IDs exceeding T, the overflow flag is set.When the overflow flag is set, it implies that the correspond-ing key is not a pseudo-candidate key. Similarly, a data areais prepared in each tuple for the “under-re-registrationflag.” It is used to maintain the consistency of the data. Thismechanism is described later.

Using the above two tables, important functions ofthe search system—registration, deletion, updating, andsearch—are performed. The realization of these functionsis described below.

3.1. Resource information registrationalgorithm

3.1.1. Registration of new resource information

In the registration of resources, information is regis-tered in two tables. When the resource ID and the set ofattribute information of the resource to be registered are

Fig. 1. Overall view of our system.Fig. 2. Forward resource table (upper) and reverse

resource table (lower).

84

received, the information is first registered in the forwardresource table, and then in the reverse resource table. Theprocedure is described below using an example.

Consider the case, as shown in Fig. 3, where theresource with ID 01234 and three attribute informationitems, that is, product category: CD player, manufacturer:xxx.corp., date manufactured: 2004/10/27, is to be regis-tered. For simplicity, the attribute information items arewritten as A, B, and C, respectively, below. The user sendsa registration request query to one of the peers formingDHT. This peer is called the registration-agent peer. Theregistration-agent peer first performs registration into theforward resource table [Fig. 3(a)]. The procedure for regis-tration into the forward resource table is the same as inordinary data registration in DHT.

In the registration into the reverse resource table, onlythe pseudo-candidate key contained in the set of resourceattribute information is registered. Any search key whichcan be composed of arbitrary combinations of attributeinformation can be a pseudo-candidate key. In this case,there are seven possibilities, that is, A, B, C, A ∧ B, A ∧ C,B ∧ C, and A ∧ B ∧ C. The registration-agent peer first triesthe registration into the reverse resource table for each ofthree cases, composed of a single element A, B, and C [Fig.3(b)]. If more than T resources are already registered in thecorresponding tuple, condition 1 of the pseudo-candidatekey is not satisfied and the registration is not allowed.Otherwise, registration as a pseudo-candidate key is per-formed.

In this case, we set T = 2. Suppose that registration ofB succeeds, but the registration of tuples A and C fails sincethree or more resources are already registered [Fig. 3(c)].This implies that B is a pseudo-candidate key, but A and Care not. In tuple C, two resources (resources with ID: 09843and ID: 24425) are originally registered, and the thresholdis exceeded if a new resource is registered at this time. Whenthe threshold is exceeded and an overflow is produced, theoverflow flag is set for the tuple.

Up to this stage, a decision as to whether three can-didates composed of a single element are pseudo-candidatekeys is made for the seven candidates described above. Thisresult can be utilized in the decision for a candidate com-posed of two elements in the next stage. In the case of A∧ B, for example, it is guaranteed that the number ofcorresponding resources does not exceed two, even if con-dition A is missing, since B is already a pseudo-candidatekey. In other words, the situation is contrary to condition 2for pseudo-candidate keys. The situation is the same for B∧ C. As regards candidate A ∧ C with elements A and C,which are not pseudo-candidate keys, condition 2 is satis-fied, but the decision for condition 1 is not obvious. Con-sequently, registration must actually be tried [Fig. 3(d)].The remaining candidate is A ∧ B ∧ C. It is obviously nota pseudo-candidate key, since it also contains B.

Thus, in this way decisions are made for all candi-dates in order, from those with the fewest elements. Whendecision has been completed for all candidates, a “registra-tion complete” message is sent from the registration-agentpeer to the user [Fig. 3(e)].

Fig. 3. Registration process.

85

The procedure performed by the registration-agentpeer can be summarized as follows as the general case.

Suppose that M items of attribute information areretained by the resource to be registered. For all integers isatisfying 1 ≤ i ≤ M, registration into the reverse resourcetable is tried in ascending order from i = 1, for ANDconditions composed of i elements which do not contain analready registered AND condition.

3.1.2. Re-registration of resource information

In Fig. 3(b) in the previous case, T resources havealready been registered in the tuple of C. The registrationof a new resource produces an overflow. This implies thatthe candidate is no longer a pseudo-candidate key as a resultof the new registration. In most such cases, a new pseudo-candidate key (such as A ∧ C) is produced in which C andsome other item of attribute information are combined bythe AND operator (up to this stage, A ∧ C has not been apseudo-candidate key, since the resources are restricted toT or fewer by C alone).

In order to register such a new pseudo-candidate keyin the index, re-registration into the system is performed inaccordance with Fig. 3(b) for the resources already regis-tered in the tuple of C. The re-registration is performed bythe peer that covers C as the registration-agent peer. The setof resource attribute information to be re-registered is ac-quired from the forward resource table. Until all re-regis-tration of resources already registered in the tuple of C iscompleted, the “under-re-registration flag” for the newpseudo-candidate key is set. The reason is as follows.

During the re-registration process, there can be re-sources which should be registered as new pseudo-candi-date keys but have not yet been registered. If a search isperformed in the meantime with the new pseudo-candidatekey as the search condition, the above resource, whichshould be included in the search result, is not returnedcorrectly, since it has not yet been registered. For thisreason, the search for new pseudo-candidate keys is blockedwhile the under-re-registration flag is set.

3.2. Resource information delete algorithm

When a resource is to be deleted, the information isdeleted from the two tables. When the system receives fromthe user the resource ID of the resource to be deleted, itoperates as follows. The tuple of the corresponding re-source is found from the forward resource table and the setof attribute information is acquired. The tuple is then de-leted from the forward resource table. Then the resource isdeleted from the reverse resource table. The procedure issimilar to the case of registration. Among all search keyscomposed from the set of attribute information, for keys

composed of a single element, the corresponding resourceID is deleted from the tuple. If the tuple is already inoverflow, a delete failure message (NG) is returned fromthe peer retaining the tuple, indicating that the key is not apseudo-candidate key. Then, using an AND condition com-posed of two elements which have not been identified aspseudo-candidate keys, deletion from the tuple is per-formed. When there are M items of resource attribute infor-mation, the procedure is repeated up to AND conditionscomposed of M elements.

It is also possible to write the list of pseudo-candidatekeys extracted from the resource information in the forwardresource table at the registration stage, and utilize thatinformation in re-registration or deletion. This will improvethe efficiency of re-registration and deletion.

3.3. Resource information update algorithm

The updating of resource information is performed inthe order of the above deletion and registration procedures.When the system receives a resource ID and a new set ofattribute information from the user, the old set of attributeinformation is acquired from the forward resource table andis deleted from the forward and the reverse resource tables.Then the resource is re-registered with the new set ofattribute information.

3.4. Resource information search algorithm

The flow of the search procedure is as follows. Usingthe reverse resource table, the pseudo-candidate key con-tained in the search condition is found. For T or fewerresources registered in the tuple of the pseudo-candidatekey, the forward resource table is used to ascertain whetherthe entire search conditions are satisfied. The check isunnecessary if the search condition and the pseudo-candi-date key are the same. If the pseudo-candidate key is asubset of the search condition, a check is required. Theresource is checked by acquiring all attribute informationfor the resource from the forward resource table by usingthe resource ID, and deciding whether the search conditionis satisfied.

The procedure to find a pseudo-candidate key fromthe search condition differs entirely from the procedureused in the registration stage to find all pseudo-candidatekeys from the set of attribute information. The followingthree properties are utilized in the procedure

(1) When there is a tuple which is not in overflow thathas a search condition as the key, that condition is thepseudo-candidate key.

86

(2) When a tuple with the search condition is inoverflow, that condition is too weak to restrict the numberof corresponding resources to T or fewer.

(3) When there is no tuple with a search condition asthe key, the condition is so strong that the resource can berestricted to T or fewer (including 0) even if one or moreitem of attribute information in the condition is deleted.

The condition cannot be a pseudo-candidate key if itis too weak or too strong. When the search condition givenby the user is too weak, it is impossible to find a pseudo-candidate key from the condition. When the condition is toostrong, the pseudo-candidate key contained in the originalsearch condition can be found by deleting part of thecomposing attribute information from the condition in or-der to weaken the condition.

Below, the search for A ∧ B ∧ C ∧ D ∧ E is used asan example, and the search procedure is described. As atemporary notation, X ∧ Y is written simply as XY.

[Step 1] Examine the tuple with ABCDE as the key.(i) When the tuple exists and is not in overflow (i.e.,

the pseudo-candidate key is found), the resource ID regis-tered there is the search result (search procedure is com-pleted).

(ii) When the tuple exists and is in overflow, thesearch condition is too weak and the system requests theuser to strengthen the search condition.

(iii) When the tuple does not exist, it implies that thesearch condition is too strong. The procedure goes to step2.

[Step 2] One of A, B, C, D, and E is selected at random(let it be C, as an example). It is deleted from the searchcondition, and the tuple with the condition (ABDE) com-posed of the remaining attribute information is investigated.

(i) When the tuple exists and is not in overflow (i.e.,the pseudo-candidate key is found), the resource ID regis-tered there is checked (for condition C). The resource whichpasses the check is the search result.

(ii) When the tuple exists and is in overflow, it indi-cates that C must be included in the condition (i.e., thecondition is too weak if C is deleted). The procedure goesto step 3.

(iii) When the tuple does not exist, it implies that Cneed not be included in the search condition (the conditionis still too strong without C). The procedure goes to step 4.

[Step 3] One of A, B, D, and E is selected at random.It is deleted from the search condition, and the tuple withthe condition composed of the remaining attribute informa-tion and C is examined.

[Step 4] One of A, B, D, and E is selected at random.It is deleted from the search condition, and the tuple withthe condition composed of the remaining attribute informa-tion is examined.

Then, the items of attribute information contained inthe search condition are deleted one by one and the possi-bility of meeting the conditions for a pseudo-candidate keyis examined. If the condition is too weak, the condition isreturned to the previous form and one of the other items ofattribute information is deleted. The reason for selecting theattribute information to be deleted at random is to avoidconcentration of the load at a particular peer.

After step 2, it is decided in each procedure whetherone of the attribute information items should be included inthe search condition. Thus, in the proposed search proce-dure, one of the pseudo-candidate keys contained in thesearch condition is found by performing at most a + 1 steps(a is the number of attribute information items contained inthe original search condition), regardless of the total num-ber of resources registered in the system. Or, it is decidedthat there is no resource matching the search condition.When the search condition is too weak, it is decided in step1. Then the user is immediately instructed to strengthen thesearch condition.

For a search using OR or NOT, it is impossible toreduce the upper bound for the amount of communication.Consequently, the simplest approach in the design andimplementation of the system is not to allow such searchprocedures. When it is desired to perform an OR search,however, the following procedure can be considered. In thecase of search condition A OR B, for example, the searchis performed for each of A and B by the above searchprocedure, and then the logical sum of the results is formed.This is possible if each of the conditions connected by theOR is a pseudo-candidate key.

When it is desired to perform a NOT search, thefollowing procedure can be considered. In the case of searchcondition NOT A, for example, the search is performed forA as the search condition if A is a pseudo-candidate key.Then, by collecting all resources which are not containedin the search result from the forward resource table, allresources matched to NOT A can be extracted. However,the number of matches can easily become tremendous, andthe processing load will also be proportionally large. If thenumber of resources corresponding to NOT A is smaller,NOT A can be registered as attribute information. ThenNOT A is a pseudo-candidate key, which aids the search.

3.5. Regular updating of reverse resource table

It may happen that more than T corresponding re-sources are reduced to T or less by deleting or updating andthe key becomes a pseudo-candidate key again. The de-lete/update algorithm described above cannot deal withsuch a situation. Thus, an “incorrect key” is produced, forwhich the overflow flag remains set even though an over-flow is no longer produced.

87

In order to restrain the generation of incorrect keys,the reverse resource table is reconstructed regularly. A newreverse resource table is reconstructed from the forwardresource table, separately from the reverse resource table inoperation. As is shown in Fig. 4, the reverse resource tablewhich has been used in the search is discarded at a certaininterval (denoted as τ), and the use of the reconstructedreverse resource table is started. At the same time, thereconstruction of a new reverse resource table is started.

When an ordinary registration, deletion, or updatequery is received, the procedure is applied to each of thetwo reverse resource tables, that in operation and that inpreparation. In order to reconstruct the reverse resourcetable, once in each period each peer registers all resourcescontained in its own forward resource table into the newreverse resource table as a registration-agent peer. In sys-tems in which the resources are always assumed to be online, such as the network devices, however, the responsibil-ity for regular registration can be assigned to the resource.

In order to maintain consistency of the data, theremust be a guarantee that all resources registered in theforward resource table of each peer are registered into thenew reverse resource table before the change of generationof the reverse resource table. For this purpose, the updateperiods of the reverse resource tables in the peers must besynchronized. If the update period τ is short, the load forreverse resource table reconstruction is increased. On theother hand, the number of incorrect keys for which overflowis not produced but the overflow flag remains set is reduced.

When there is an incorrect key, the following situ-ation may arise. Even if the user performs a search using asearch condition containing the pseudo-candidate key, thesystem considers the condition as “in overflow,” that is, thecase in which the search results are not sufficiently re-stricted, and the search is not performed. However, it is thecase that all of the pseudo-candidate keys contained in the

search condition are incorrect. The search can be performedso long as there is at least one correct key.

Even when the search cannot be performed, the useris just requested to add a search condition, which does notgreatly degrade usability. First of all, since the key has onceexceeded the threshold, the key has less power to restrictthe resources and will not be very useful. Thus, the effectis smaller, even if the overflow flag is left set. In thedetermination of the length τ of the update period, suchsituations should be considered in order to realize a trade-off among the frequency of updating or deletion of re-sources, the requests from the user, the load on the network,and the processing load in the peer. The setting of τ isdiscussed further in Section 4.1.3.

In the system in which resources are only added andnever deleted or updated, updating of the reverse resourcetable is unnecessary. In such a case, it suffices to prepareonly a single reverse resource table in which the registrationand search processes are performed.

4. Evaluation and Discussion

The proposed system was evaluated as regardsscalability and applicability to large-scale systems. Specifi-cally, the amount of communication needed in resourcesearch or registration and the memory capacity required tomaintain the two tables are evaluated. The setting of thethreshold T is also discussed.

4.1. Evaluation of amount of communication

4.1.1. Amount of communication needed insearch process

Let Ai be the attribute information. Generally, thecommunication needed in solving search condition A1 ∧ A2

∧ ⋅ ⋅ ⋅ ∧ Aa is composed of the following three items.

(1) Communication to search for the pseudo-candi-date key.

(2) After finding the pseudo-candidate key, commu-nication performed from the peer covering the pseudo-can-didate key to check T or fewer resources registered in thetuple of the pseudo-candidate key.

(3) Transmission of the check results from the peercovering the pseudo-candidate key to the user.

In order to simplify the discussion, it is assumed thatthe number of hops in the communication between peers,including the user terminal, remains constant. The amountof communication is defined as (number of peer-to-peercommunications) × (packet size).Fig. 4. Update process of reverse resource table.

88

(1) Communication to find pseudo-candidate key

As discussed in connection with the search algorithm,the pseudo-candidate key can be found by a + 1 search stepsat the maximum. In each step, the communication neededfor routing in the overlay network to find the peer coveringthe key, and also the communication needed for the re-sponse from the covering peer is performed. Since thecommunication is used only to transmit the resource ID, itshash value, and the flag indicating whether there is anoverflow, the size of a communication packet is at mostseveral tens to several hundreds of bytes. Let it be denotedm. The routing on the overlay network depends on the DHTalgorithm, but is usually realized by O(log P) hops (whereP is the number of peers). Let the number of these transfersbe p. Then the amount of communication needed in findingthe pseudo-candidate key is m(a + 1)(p + 1) at the maxi-mum.

(2) Communication needed for check of T or fewerresources

In the check of each resource, the procedure is asfollows. The whole condition is sent to the peer coveringthe resource ID. The peer checks whether the set of attributeinformation related by the resource ID satisfies the wholecondition, and returns the result. As previously, only thecondition and the hash value are included in the communi-cation packet, and the size is approximated as m. Thenumber of resources to be checked is at most T. Conse-quently, the amount of communication needed for the checkis at most mTp.

(3) Transmission of check result

The result of the check is T or fewer resource IDs.Approximating the amount of communication of a resourceID as m, the required amount of communication is at mostmT.

By the above reasoning, the amount of communica-tion needed in the whole search process is

m(a + 1)(p + 1) + mTp + mT = m(p + 1)(a + 1 + T)

at the maximum, where a is the number of attribute infor-mation items as specified by the user, which will not be verylarge; m is several tens to several hundreds of bytes; p is ofthe order of log P; and T can be set to some value. Thus, theamount of communication needed in the search process canbe reduced to a low value regardless of the number ofresources and almost regardless of the number of peers.

4.1.2. Amount of communication needed inregistration process

The proposed algorithm is designed to reduce theamount of communication in search, which has been theproblem in conventional approaches, by preparing an indexat the registration stage with a greater effort than in theordinary DHT. Because of this approach, a larger amountof communication is required in the registration processthan in the conventional methods. In order to investigatethis point, resource information was artificially constructedand a simulation was performed to simulate the registrationprocess in the proposed algorithm. The amount of commu-nication needed in the registration process was investigated.

As the artificial resource information, N resourceinformation sets were prepared. It was assumed that thenumber of attribute information items to be associated withthe resource followed a Poisson distribution with mean M.There were S kinds of attribute information. The appear-ance frequency of each item was assumed to follow the Zipfdistribution and the patterns of the appearances were as-sumed to be mutually independent. Approximately 600,000items of document information XML data publicized inDBLP [8] were examined, and it was found that the numberof resources and the kinds of attribute information wereproportional, as shown in Fig. 5. Consequently, we set S =N.

We set T = 1000 and M = 20, and the number ofpseudo-candidate keys and the number of registration trialsfor each resource were determined while varying the num-ber of resources N. Figures 6 and 7 show the results. In theexperiment, the efficiency improvement for the re-registra-tion process described in Section 3.2 was applied. Since theamount of communication for each registration trial ism(p + 1), which is a constant, the amount of communica-tion for the whole registration process is proportional to thenumber of registration trials for each resource.

Fig. 5. Relationship between number of resources andnumber of attribute types.

89

Figure 6 shows the change of the total number ofpseudo-candidate keys, and also how many attribute infor-mation items compose the pseudo-candidate keys. It isevident from the figure that pseudo-candidate keys com-posed of a larger number of elements increase with thenumber of resources because the number of keys exceedingT also increases. This implies that the number of registrationtrials increases with the number of resources.

It is evident from Fig. 7, on the other hand, that thenumber of registration trials increases more slowly as thenumber of resources is increased. Simple regression analy-sis was applied to the 11 measured points in Fig. 7. Lettingthe regression expression be

y = c0 ln(N + c1) + c2 (1)

the determining coefficient was high, namely, 0.9997. HereN is the number of resources and y is the number ofregistration trials for a resource. The partial regressioncoefficients were obtained as c0 = 21.94622, c1 = 3913.928,and c2 = –166.6162. Since the number of registration trialschanged as the logarithmic order of the number of re-

sources, it was concluded that the amount of communica-tion needed in the whole registration process is scalable.

4.1.3. Amount of communication needed inupdating of reverse resource table

As discussed in Section 3.5, the reverse resource tablemust be reconstructed regularly in order to maintain con-sistency of the data in the system. The load imposed by thisprocess is estimated as follows.

An incorrect key for which overflow is not actuallyproduced, but the overflow flag remains set and can appearwhen the resource ID information registered in the tuple ofthe pseudo-candidate key including that key is deleted.Since the updating process is a combination of deletion andregistration processes, there is a possibility that an incorrectkey may be produced not only in the deletion process, butalso in the registration process.

Assuming a situation in which resources are deletedor updated in the state in which 100,000 resources areregistered, the percentage with which the key is an incorrectkey was examined. Figure 8 shows the results. The distri-bution of the registered resources was assumed to be thesame as in Section 4.1.2, and we set M = 20 and T = 1000.For simplicity, only the deletion process was applied in theexperiment.

It was assumed that each search query included xpseudo-candidate keys, and that all pseudo-candidate keysin the index were utilized with the same probability in thesearch process. Thus, the probability that the user is re-quested by the system to strengthen the search conditioneven though the search query can keep the number of searchresults below the threshold is given by (percentage ofincorrect keys/100)x × 100 (%). When this probability isallowed to be 0.1% and x = 2 is set, the percentage ofincorrect keys allowed can reach approximately 3%.

According to Fig. 8, the reverse resource table shouldbe reconstructed at an interval such that approximately 80%

Fig. 6. Relationship between number of resources Nand total number of pseudo-candidate keys.

Fig. 8. Relation between number of deleted resourcesand percentage of incorrect keys.

Fig. 7. Relationship between number of resources Nand number of registration trials/resource.

90

of the registered resources are deleted or updated. Theamount of communication needed for the registration proc-ess described in Section 4.1.2 is the load on the network.

Consider the case in which the registration, updating,and deletion processes are each performed Q times a sec-ond, and N resources are always registered. Letting thecommunication cost for a registration process be C, nearlythe same cost is incurred in the deletion process. Conse-quently, costs of 2C and C are incurred in updating anddeletion processes, respectively. That is, a communicationcost of 4QC is required per second for the registration,updating, and deletion processes.

On the other hand, since 2Q updating and deletionprocesses are performed in a second, the time until 80% ofN resources are deleted or updated is 0.8N/2Q seconds.Letting this be the update interval of the reverse resourcetable, since N registration processes are performed in eachperiod, the required communication cost per second is(N/(0.8N/2Q)) * C, that is, 2.5QC. Due to this communica-tion cost spent on updating of the reverse resource table, thecommunication cost is increased by a factor of 1.625 com-pared to the case of ordinary registration, updating, anddeletion processes only. Of course, by increasing the allow-able percentage of incorrect keys, the cost of updating thereverse resource table is reduced.

4.2. Evaluation of memory capacity

The same memory capacity as in the ordinary DHTis required to store the forward resource table, which isnearly equal to the amount of attribute information for allresources. For the reverse resource table, on the other hand,the resource IDs must be preserved in duplicate in multiplekeys, which requires a larger memory capacity than inordinary DHT. The same simulation as in the above experi-ment was performed in order to investigate the memorycapacity required for the reverse resource table.

We set T = 1000 and M = 20, and the number ofduplicate registrations for a resource was investigated byvarying the number of resources N. Figure 9 shows theresults. The number of duplications indicates the number ofmany tuples in which the resource is registered in duplicate.It can be considered as the number of pseudo-candidatekeys per resource. The shape of the curve is similar to thecurve given in Section 4.1.2, indicating that scalability alsoexists in terms of memory capacity.

4.3. Setting of threshold T

It was assumed that the value of the threshold T wasset by the system designer. Consequently, there must beguidelines for design. We set N = 10,000 and M = 20. Thenumber of registration trials and the number of duplicated

registrations for a resource were investigated while varyingT. Figures 10 and 11 show the results. It is evident from thefigures that when T is low, a larger number of keys exceedT. Thus, the number of registrations to the new key isincreased in accordance with the number of their combina-tions, which increases both the number of registration trialsand the number of duplicated registrations.

Based on Figs. 10 and 11 and the results of theevaluations up to this stage, the following points should beconsidered carefully in threshold setting.

When T is too high:

• The amount of communication of intermediatesolutions is increased in the search process.

• The amount of computation for the check processis increased in the search process.

When T is too low:

• Many overflows occur, which increases theamount of communication needed in the registra-tion process.

Fig. 10. Relationship between threshold T and numberof registration trials/resource.

Fig. 9. Relationship between number of resources Nand number of duplications registered/resource.

91

• Many overflows occur, which increases the mem-ory capacity needed for the reverse resource table.

• The user is forced to submit strong conditions.

The designer must determine the value of the thresh-old T, considering not only the available communicationbandwidth and the memory capacity, but also the require-ments of the user (the search frequency and the number ofresources to be retrieved in a search), the number of re-sources, and the properties of the assigned attribute infor-mation.

5. Related Studies

Several studies intended to reduce the amount ofcommunication needed for keyword search in DHT havealready been presented. Reynolds and colleagues proposedcommunication compression by sending intermediate solu-tions in the form of a Bloom filter [9]. Li and colleaguesinvestigated existing techniques, including the method ofReynolds and colleagues, and concluded that a method thatcombines clustering, Gap Compression [10] and AdaptiveSet Intersection [11], can yield the greatest decrease in theamount of communication during the search [12]. They alsonoted that even if these methods are combined, the amountof communication is not reduced sufficiently, indicatingthat it is difficult to realize a Web search engine with aDHT-based P2P search system. In contrast, the proposedmethod can reduce the amount of communication duringthe search to a lower value by setting T low.

Other than these, a further method intended to reducethe amount of communication has been proposed. In thismethod, logical products such as the product of “product ofX Co.” and “network camera” are calculated beforehandand are registered in the index as a search key, as in themethod proposed in this paper (called precomputation) [12,14]. A problem of this approach is that the number of

indexes increases explosively since combinations of key-words are also contained in the index.

In order to cope with this point, Ref. 14 proposes amethod in which only combinations of two keywords arehandled by precomputation. The idea is to prevent explo-sion by not registering combinations of three or more in theindex. Compared to the method presented in Ref. 14, theproposed method has the following advantages.

Useless indexes are not generated: Consider thesearch condition A1 ∧ A2, for example, in which A1 or A2 isa pseudo-candidate key. In the proposed method, the set ofresources connected to that pseudo-candidate key must beextracted and checked. But in the method of Ref. 14, itsuffices to extract the set of resources connected to A1 ∧ A2.It should be noted, however, that such a search conditiondoes not essentially require a large amount of communica-tion, and that queries can be handled at high speed. It is notefficient to prepare an index for such a search conditionfrom the viewpoint of either the registration process or thememory capacity. The proposed method does not constructthe index for such a search condition.

The amount of communication and the processingtime are reduced below some value for any search condi-tion: In the method of Ref. 14, there remain cases for ANDconditions containing three or more keywords in which alarge amount of communication is produced. In the pro-posed method, in contrast, the amount of communicationand the processing time are reduced below certain limits forany search condition, so long as the search condition is suchthat the search result is restricted to T or fewer items.

The advantage of the method presented in Ref. 14 isthe amount of communication in the registration process.Consider the case in which a resource has 20 attributeinformation items. In the method of Ref. 14, the number ofregistration trials is 20 +20 C2, which is 210 times, regard-less of the number of resources. Comparing this to Fig. 7,and calculating on the basis of the regression formula, it isevident that the number of registration trials is larger in theproposed method when the number of resources exceedsapproximately 30 million. It should be noted, however, thatthe number of registration trials increases only in logarith-mic order in the proposed method, and that scalability isguaranteed.

In Ref. 12, on the other hand, a method is proposedin which only keywords used frequently in the searchconditions are combined to form the index. In this methodtoo, a large amount of communication and calculation isrequired and high-search speed is not maintained for ANDconditions which are not precomputed. It is expected in thisapproach that the amount of communication will be re-duced effectively when there is marked nonuniformity inthe use of keywords in the search condition. In the caseassumed in this paper, however, where things in the realworld are considered as resources, the resources that the

Fig. 11. Relationship between threshold T and numberof duplications registered/resource.

92

users try to retrieve will differ, and it is to be expected thatthe keywords will be used in a more diverse way than in thecase of a Web search. In such a case, the effectiveness ofprecomputation based on frequency of use will be reduced.

Other than these, methods to reduce the amount ofcommunication by caching intermediate solutions or searchresults have been proposed [9, 13]. It seems likely that thismechanism will not work effectively for the same reason.In the proposed method, in contrast, the amount of commu-nication can be reduced regardless of the frequency distri-bution of the keywords for the search condition.

In the method proposed in Ref. 15, each peer retainsa copy of all attribute information of the resources. A searchinvolving an AND condition is performed using only oneof the contained attribute information items, and the checkis performed by the corresponding peer. The communica-tion of intermediate solutions is made unnecessary by thismechanism. However, it is likely that the number of checkprocesses will be enlarged, which increases the amount ofcomputation and also the delay. In the proposed method, incontrast, the number of check processes can be reduced toa low value.

6. Conclusions

The purpose of this paper is realize keyword searchwith attribute information as keywords for things in the realworld. A DHT-based P2P search system using pseudo-can-didate key indexing is proposed. The reduction of theamount of communication in the search, which is the majorpurpose of this paper, is evaluated quantitatively, and it isverified that the expected effectiveness is achieved.

To investigate the problems in the proposed methodthat a larger amount of communication is required forregistration (index construction) and that a larger memorycapacity is required to maintain the index than in the con-ventional methods, a quantitative evaluation is performed.It is verified by simulation that scalability is guaranteed.The points to be considered in the determination of impor-tant parameter T in system design are pointed out on thebasis of the results.

It is planned to continue a study of the detaileddetermination procedure for T, and also of dynamic adjust-ment of T.

REFERENCES

1. Shi S, Yang G, Wang D, Yu J, Qu S, Chen M. Makingpeer-to-peer keyword searching feasible using multi-level partitioning. Proc 3rd International Workshop

on Peer-to-Peer Systems (IPTPS’04), p 151–161, SanDiego.

2. Ratnasamy S, Francis P, Handley M, Karp R, ShenkerS. A scalable content-addressable network. ProcACM SIGCOMM 2001, p 161–172, San Diego.

3. Stoica I, Morris R, Karger D, Kaashoek MF, Bala-krishnan H. Chord: A scalable peer-to-peer lookupservice for internet applications. Proc ACM SIG-COMM 2001, p 149–160, San Diego.

4. Rowstron A, Druschel P. Pastry: Scalable, distributedobject location and routing for largescale peer-to-peer systems. Proc 18th IFIP/ACM InternationalConf on Distributed Systems Platforms (Middleware2001), p 329–350, Heidelberg, 2001.

5. Zhao BY, Kubiatowicz J, Joseph AD. Tapestry: Aninfrastructure for fault-tolerant wide-area locationand routing. Tech Rep UCB/CSD-01-1141, UCBerkeley, 2000.

6. Hayami S, Takeno H, Nagase T, Fujimoto N, Hagi-wara K. Proposal and evaluation of scalable WWWparallel search engine designing methods. ISPJ SIGNotes, 2000-DBS-123, Vol. 2001, No. 8, p 45–52,2001. (in Japanese)

7. Loo BT, Huebsch R, Stoica I, Hellerstein JM. Thecase for a hybrid P2P search infrastructure. Proc 3rdInternational Workshop on Peer-to-Peer Systems(IPTPS’04), p 141–150, San Diego.

8. http://dblp.uni-trier.de/9. Reynolds P, Vahdat A. Efficient peer-to-peer keyword

searching. Proc Middleware 2003, p 21–40.10. Witten IH, Moffat A, Bell TC. Managing gigabytes:

Compressing and indexing documents and images.Van Nostrand Reinhold; 1999.

11. Demaine ED, Lopez-Ortiz A, Munro JI. Adaptive setintersections, unions, and differences. Proc 11th An-nual ACM-SIAM Symposium on Discrete Algo-rithms (SODA 2000), p 743–752, San Francisco.

12. Li J, Loo BT, Hellerstein JM, Kaashoek MK, KargerDR, Morris R. On the feasibility of peer-to-peer Webindexing and search. Proc 2nd International Work-shop on Peer-to-Peer Systems (IPTPS’03), p 207–215, Berkeley.

13. Kobatake K, Takemoto D, Tagashira S, Fujita S. Anefficient technique for processing compound querieson DHT-based PSP systems. Tech Rep IEICE2004;104:7–12. (in Japanese)

14. Gnawali OD. A keyword-set search system for peer-to-peer networks. Master’s Thesis, Massachusetts In-stitute of Technology, 2002.

15. Loo BT, Hellerstein JM, Huebsch R, Shenker S,Stoica I. Enhancing P2P file-sharing with an internet-scale query processor. Proc 30th International Confon Very Large Data Bases (VLDB 2004), Toronto.

93

AUTHORS (from left to right)

Yutaka Arakawa (member) received his B.E. and M.E. degrees in electronic engineering from the University of Tokyoin 2001 and 2003 and joined NTT. He is currently working on ubiquitous networking systems at NTT Network InnovationLaboratories. He is a member of IEICE.

Hiroya Minami (member) received his B.S. and M.S. degrees in information science from Seikei University in 1994 and1996 and joined NTT Laboratories. He received the Young Researcher’s Encouragement Award from IEICE in 2004. He iscurrently working on ubiquitous networks at NTT Network Innovation Laboratories. He is a member of IEICE.

Masato Matsuo (member) received his B.E. and M.E. degrees in mechanical engineering from Kyoto University in 1986and 1988 and joined NTT. Since then, he has been mainly involved in research on adaptive network service systems andubiquitous networking systems. He is currently a senior research engineer and supervisor at NTT Network InnovationLaboratories. He is a member of IEICE and IPSJ.

Masayasu Yamaguchi (member) received his B.E., M.E., and Ph.D. degrees in electrical and electronic engineering fromChiba University in 1980, 1982, and 1994. Since he joined NTT Electrical Communication Laboratories in 1982, he has beenmainly involved in research on photonic switching systems. He is presently Senior Research Engineer, Supervisor, and leadingresearcher on ubiquitous networking technologies at NTT Network Innovation Laboratories. He is a member of IEICE andIEEE (LEOS and COMSOC).

Hiroshi Saito (member) graduated from the University of Tokyo with a B.E. degree in mathematical engineering in 1981,an M.E. degree in control engineering in 1983, and a D.Eng. degree in teletraffic engineering in 1992. He joined NTT in 1983.He is currently responsible for ubiquitous service systems at NTT Network Innovation Laboratories as an Executive Manager.He received the Young Engineer Award from IEICE in 1990, the Telecommunication Advancement Institute Award in 1995,and an excellent paper award from the Operations Research Society of Japan (ORSJ) in 1998. He is an IEEE fellow, an ORSJfellow, and a member of IFIP WG 7.3 and IEICE.

94