8/12/2019 06327415

1/5

Personalized Text Summarization Based on Important Terms Identification

Rbert Mro, Mria BielikovInstitute of Informatics and Software Engineering, Faculty of Informatics and Information Technologies,

Slovak University of Technology, Ilkoviova 3, 842 47 Bratislava, Slovakia{robert.moro, maria.bielikova}@stuba.sk

AbstractAutomatic text summarization aims to address the

information overload problem by extracting the most impor-

tant information from a document, which can help a reader to

decide whether it is relevant or not. In this paper we propose

a method of personalized text summarization which improves

the conventional automatic text summarization methods by

taking into account the differences in readers characteristics.

We use annotations added by readers as one of the sources of

personalization. We have experimentally evaluated the

proposed method in the domain of learning, obtaining better

summaries capable of extracting important concepts explainedin the document when considering the relevant domain terms

in the process of summarization.

Keywords-automatic text summarization; personalization;

annotations; relevant domain terms

I. INTRODUCTION

Information overload is one of the most serious problemsof the present-day web. There are various approachesaddressing this problem; we are interested mainly in two:automatic text summarization and personalization.

Automatic text summarization aims to extract the mostimportant information from a document, which can help

readers (users) to decide whether it is relevant for them andthey should read the whole text or not. However, theclassical (generic) summarization methods summarize thecontent of a document without considering the differences inusers, their needs or characteristics, i.e. their interests, goalsor knowledge. On the other hand, personalization aims toadapt the content presented to an individual user or the wayshe accesses the content based on her characteristics.

In this paper we propose a method of personalizedsummarization which extracts from the documentinformation that we assume to be the most important orinteresting for a particular user. Because annotations (e.g.highlights) can indicate a users interest in the specific partsof the document [14], we use them as one of the sources of

personalization. Our proposed method is sufficiently generalto be used independently of the chosen domain; however wefocus on the summarization for revision in the domain oflearning.

II. R ELATED WORK

The first method of automatic text summarization wasproposed by Luhn [7]; it was based on term frequencywhichhe used to compute the significance of terms. The idea wasto extract from a document the most significant sentences,which contained the highest number of occurrences of

significant terms. Edmundson [5] considered not only thefrequency of terms, but also their location. Terms in the title,the first and the last paragraph and the first and the lastsentence in each paragraph were assigned positive weights.

Gong and Liu [6] were first to use latent semanticanalysis (LSA) for the text summarization. This method iscapable of finding salient topics or concepts in the documentand also their relative importance within the document. Eachconcept (topic) is represented in the summary by a sentence,which captures it the best. However, Steinberger and Jeek[10] showed that this approach fails to include into thesummary sentences, which capture many concepts well, buthave the highest score for none of them. They proposeda modification of the selection of sentences; sentences areselected based on their overall score computed asa combination of scores for each concept (topic).

All the methods mentioned so far summarize only thecontent of the document. However, there are many types ofinformation which can indicate a relevance of the sentencesto extract, especially on the Web, e.g. user activity or user-added annotations (comments, highlights, tags etc.). Sun etal. [11] utilized clickthrough data which records how usersfind information through queries; if a user clicks on a link toa web page which is one of the results of her query, it

indicates that terms from the query describe the page and canbe given more weight when summarizing the page. Park etal. [9] summarized not the content, but the comments(descriptions) and tags which users add when they createa bookmark using social bookmarking service such as

Delicious. The advantage of this approach is that it cansummarize also documents with no or minimum text, butwith other multimedia content. On the other hand, it dependson the number of added bookmarks and is unable tosummarize documents with no bookmarks.

Methods of personalized summarization also useadditional information. However, their result is not a genericsummary which is the same for all the users, but a summarythat is adapted (personalized) to the characteristics of

a particular user. Daz et al. [4] personalized summarizationto mirror usersinterests represented by a user model in theform of a vector of weighted keywords; the disadvantage ofthis approach is that the users have to manually insertkeywords and weights into the model. Campana andTombros [3] automatically built a user model from thesentences of the documents recently read by a user.Summary is constructed from the sentences which are themost similar to the most representative sentences from theuser model. Zhang et al. [14] utilized user annotations in theform of highlights by extending the classical tf-idf method.

2012 23rd International Workshop on Database and Expert Sytems Applications

1529-4188/12 $26.00 2012 IEEE

DOI 10.1109/DEXA.2012.47

131

8/12/2019 06327415

2/5

Results of their study suggest that this approach is useful forpersonalization of summarization. They identify determininga subset of annotations suitable for summarization andincluding of collaborative annotating as open problems.

III. PERSONALIZED TEXT SUMMARIZATION

We propose a method of personalized text summarization

based on a method of latent semantic analysis [6][10] whichconsists of the following steps:

A. Pre-processingB. Construction of a personalized terms by sentences

matrixC. Singular value decomposition (SVD)D. Sentences selection

A. Pre-processing

The pre-processing phase includes machine translation,tokenization, terms extraction and segmentation of the textinto sentences.

We use machine translation of the document toa reference language (in our case English) in order to

maximize our methods independency of the summarizeddocuments language. Because the methods of termsextraction and segmentation of the text into sentencesperformed during the pre-processing are language-dependent, the use of machine translation enables us toprovide their implementations only for one (reference)language and effectively cover a wide range of languages.Certainly, this can influence the quality of the resultingsummaries; however, our experiments show that thecommercially available machine translation (using BingTranslator, www.microsofttranslator.com) gives reasonableresults.

B. Construction of a personalized terms-sentences matrix

We have identified the construction of a terms-sentences

matrix representing the document as a step suitable forpersonalization of the summarization. In this step termsextracted from the document are assigned their respectiveweights. Our proposed weighting scheme extends theconventional weighting scheme based on tf-idf method bya linear combination of the multiple raters, which positivelyor negatively affect the weight of each term (see Fig. 1).

We formulate the weighting scheme as follows:

,)()( k

ijkkij tRtw

where w(tij) is a weight of a term tijin the matrix and kis alinear coefficient of a rater Rk. Both the weights w(tij)and the

linear coefficients kcan be any real number. The rater Rkisa function, which assigns each term from the extractedkeywords set Tits weight:

.: TRk

We have designed a set of raters which can be dividedinto two groups:

Generic raters:terms frequency rater, terms locationrater and relevant domain terms rater

Personalized raters: knowledge rater and anno-tations rater

R1 R2 Rn

+

...

tij

w(tij)

1 2 n

Figure 1. Term weighting by a combination of raters.

Generic raters take into account the content of thedocument and some additional information to adaptsummarization regardless of a particular user. On the otherhand, personalized raters consider information about thespecific user and her characteristics. Selection andcombination of the raters (the values of their linearcoefficients) depends on a specific summarization scenarioand the types of available additional information.

Terms frequency rater and terms location rater are thetwo basic generic raters, the design of which was inspired byLuhn [7] and Edmundson [5]. The former assigns theweights based on tf-idf method, the latter based on thelocation of terms; terms in the title and the first and the lastsentence of a document are given positive weights.

Because we focus on the domain of learning, we haveidentified three main sources of personalization andadaptation of the summarization suitable for the chosendomain:

Domain conceptualization in the form of the relevantdomain terms

Knowledge of the users

Annotations added by users, i.e. highlights or tagsWe have designed a relevant domain terms rater to

utilize information contained in a domain model of anadaptive system [13]. Domain models are usuallyconstructed manually by domain experts (however, it isuseful to utilize folksonomies for this purpose as well, as wehave shown in [8]) by capturing their knowledge of the

domain in the form of important concepts (relevant domainterms) and relationships among them which makes themvaluable sources of information for adapting thesummarization (if they are available).

Let Cdbe a set of concepts associated with a documentd; each concept in the set is represented by ordered pair (ti,wi), where ti is a relevant domain term i and weight wirepresents a measure of association between document dand

concept iand it is a real number from the interval 0, 1.

132

8/12/2019 06327415

3/5

We formulate the relevant domain terms rater as follows:

else,0)(

if)(

ij

djiiij

tw

CStwtw

where Sjis a set of terms contained in the sentencej.Educational systems usually use overlay user models to

record the level of knowledge of each concept from thedomain for each particular user. We have designed aknowledge rater as a personalized version of the previousone. It uses information captured in the user model [1]; soinstead of wi representing the measure of associationbetween document dand concept i, we now use kiureflectinga level of knowledge of the concept iby a user u. This way,concepts which are better understood by the user are givenmore weight, which is especially useful in the knowledgerevision scenario.

Although learning systems usually assume that modeledknowledge only grows in time, users in fact do forget a partof their acquired knowledge [2]. The knowledge revisionrepresents a means of re-acquiring the forgotten knowledge.

We believe that summarization is suitable for this scenario,because it can help users to remind them of the importantconcepts explained in the documents.

Annotations ratertakes into account the fragments of thetext highlighted by a particular user. First, we construct a setof all the sentences, fragments of which were highlighted bya particular user:

,0| ujju

H HSSS

where Sj is j-th sentence of the document and Huis a set ofall the highlights made by the particular user u. Similarly, weconstruct a set of all the sentences, fragments of which were

highlighted by all the users, taking only those for which thefollowing condition stands true:

,2

max ll

j

h

h

where |hj| is a number of times j-th sentence Sj washighlighted. Lastly, we make a union of these two sets andassign positive weights to those terms of the document,which are located in the sentences in the resulting set SH:

else,0)(

Sif1)(

ij

Hjij

tw

Stw

where Sj isj-th sentence of the document.

C. Singular Vaule Decomposition

Constructed personalized terms-sentences matrix A isdecomposed using a singular value decomposition [6]:

, TVUA

where U= (uij) is a matrix, the columns of which representleft singular vectors and the rows terms of the document, =diag(1, 2, , n) is a diagonal matrix with diagonalelements representing the non-negative singular values in thedescending order and V = (vij) is a matrix, the columns ofwhich represent right singular vectors and its rows representsentences of the document.

D. Sentences Selection

As the final step, we select sentences with the highestscore computed by a method proposed in [10] (sentences areselected from the original, not the translated document):

,2

1

2

i

n

i

kik vs

whereskis a score of k-th sentence, vkiis a value from matrixV that measures how well concept i is represented bysentence kand iis a singular value that represents relativerelevance of the concept iin the document; nis a number ofdimensions and it is a parameter of the method. The length ofthe generated summary is a parameter of our method as well.

IV. EVALUATION

We have experimented in the domain of learningchoosing the knowledge revision as our specific use case.Our dataset has consisted of the educational materials fromthe Functional and Logic Programming course in theeducational system ALEF.

ALEF (Adaptive Learning Framework) [12] is anapplication framework which merges the concepts of thetraditional web-based educational systems with the principlesof the Web 2.0, i.e. it brings several interactive features suchas tagging, commenting, collaborative task solving etc. Wehave integrated our implementation of the summarizer withALEF (see Fig. 2).

In our experiment, we have focused on evaluation andcomparison of the two variants of summaries:

generic summarization and

summarization considering the relevant domainterms identified by a domain expert.

We have generated both variants for each document inour dataset; because we focus on the summarization forrevision, we have chosen the length of generated summariesto be approximately one third of the document length.

We have asked the Functional and Logic programmingcourse students to evaluate the quality of the generatedsummaries; 17 students have participated in our experiment.Their task has been to read the educational texts in ALEF

and rate presented summaries on a five-point scale withoutknowing what variant of summary is presented to them.Summaries has been placed underneath the text so that thestudents would read the document first and its summarysecond. In the real usage the summary is positioned abovethe text so that it gives users (students) an overview of thetopics of the text and it can help them to decide whether toread the whole text or not.

133

8/12/2019 06327415

4/5

After each summary rating, the students have been askeda follow-up question to further evaluate the summary quality.We have asked them whether the sentences selected for thesummary are representative, whether the summary is suitablefor revision or whether it could help them to decide thedocument relevance. We have also inquired whether thelength of the summary is suitable given the length andcontent of the document and if it is readable andcomprehensible.

Furthermore, we have chosen a group of five excellentstudents as an expert group to compare their summaryevaluation to that of the other students. In contrast to theother participants, they have been presented both summaryvariants (in random order) for each educational text in orderto decide which variant is better or whether they are equal.

We have gathered summaries for 79 educational texts(explanation learning objects) in Slovak, 278 summaryratings and 154 summary variants comparisons fromstudents-experts. Moreover, students have answered 275follow-up questions.

The second variant (summarization considering therelevant domain terms) has on average gained approximately7.2% higher score on a five-point scale compared to the first

variant (generic summarization using only terms frequencyand location rater) as can be seen in Tab. 1.

TABLE I. SUMMARY VARIANTS RATINGS

Statistic Generic Relevant d omain terms

No. of ratings 143 135

Mean 3.538 3.793

Variance (n-1) 1.518 1.419

We have also computed average score for each summaryvariant for each document. The second variant has scoredmore in comparison to the first one in 48% of the cases, thesame in 11% and less in 41%. The comparison of summariesby the students-experts has given us similar results. Thesecond variant has been evaluated as better in 49% of thecases, as equal in 20% and worse in 31% (see Fig. 3).

The relatively high percentage of cases when the genericvariant has been evaluated as better can be attributed to thefact that even though the summaries that have considered therelevant domain terms have managed to extract morerelevant information, they have been sometimes lesscomprehensible and readable compared to the genericvariant.

0%

10%

20%

30%

40%

50%

60%

A B C

students

experts

Figure 3. Comparison of the summary variants, whereAmeans thatsummary considering the relevant domain terms has been evaluated as

better,Bthat generic summary has been evaluated as better andCthat theyhave been evaluated as equal.

Figure 2. Example screenshot of ALEF (in Slovak) with the integrated summarizer (1highlighted by the border). Current user rating isshown in the right top corner (2). Users rated summaries on a five-point scale using stars; after each rating they were automatically asked

a follow-up question. They could have also added feedback in the form of free text (3) or navigate themselves to the next summary byclicking theNextbutton in the right bottom corner (4).

134

8/12/2019 06327415

5/5

Still, our results suggest that considering the relevantdomain terms during the summarization process leads tobetter summaries in comparison to the baseline genericvariant. However, they also suggest that we have to paymore attention to the natural language processing problems,such as anaphora resolution in the future.

Lastly, we have evaluated the students answers to the

follow-up questions. They suggest that our method in generalhas managed to choose representative sentences and thesummaries could be in many cases used for revision or tohelp the students to decide whether the summarizeddocument is relevant. Furthermore, the answers to thefollow-up questions show that the notion of the summaryquality is subjective. Therefore, we believe that it is useful topersonalize summarization and we can get significantlybetter results when we take students annotations intoconsideration.

V. CONCLUSIONS AND FUTURE WORK

In this paper we have proposed a method of personalizedsummarization, which extends existing summarization

methods by considering various user characteristicsincluding context.

Our contribution lies in the proposal of

the specific raters that take into account termsrelevant for the domain or the level of knowledge ofan individual user

the method of the raterscombination which allowsconsidering various parameters or context of thesummarization.

Even though our approach is domain independent, itallows us to identify the sources of personalization andadaptation of the summarization specific for the chosendomain and to adapt the method for a particular scenario (inour case students knowledge revision).

We work towards providing the users with bettersummaries reflecting their particular needs by taking intoaccount also their annotations (highlights) that indicateusers interest in the fragments of a document. Byconsidering not only a users personal annotations, but themost popular ones as well, we can potentially deal with asituation when the user has not yet added any annotations tothe document and also utilize the wisdom of the crowd.

We have evaluated our approach in the domain oflearning in the knowledge revision scenario. In the firstphase of the evaluation, we have focused on the comparisonof the two summary variants: the generic summarization andthe summarization considering the relevant domain terms.Our experimental results suggest that using the relevantdomain terms in the process of summarization can helpselecting representative sentences capable of summarizingthe document, even for revision.

As our future work, we plan to carry out more experi-ments with the two raters: the knowledge rater and theannotations rater. We believe that considering the knowledgeas well as users annotations in the summarization processwill lead to summaries better adapted for a particular users

needs. We also see a potential in extending our method sothat the raters combination parameters wouldnot have to beset manually, but automatically and dynamically based onthe reliability of the raters sources of information or thecategory of the summarized document.

ACKNOWLEDGMENT

This work was partially supported by the grantsVG1/0971/11/2011-2014, APVV-0208-10 and it is thepartial result of the Research & Development OperationalProgramme for the project Research of methods foracquisition, analysis and personalized conveying ofinformation and knowledge, ITMS 26240220039, co-fundedby the ERDF.

REFERENCES

[1] M. Barla, Towards Social-Based User Modeling andPersonalization, Information Sciences and Technologies Bulletin ofthe ACM Slovakia. vol. 3, no. 1, 2011, pp. 52-60.

[2] M. Bielikov, P. Nagy, Considering Human Memory Aspects forAdaptation and Its Realization in AHA!, in Technology EnhancedLearning (EC-TEL 06), W. Nejdl, K. Tochtermann, Eds. Springer

Verlag, LNCS, vol. 4227, pp. 820.[3] M. Campana, A. Tombros, Incremental personalised summarisation

with novelty detection, Proc. 8th Int. Conf. Flexible QueryAnswering Systems (FQAS 09), LNCS, vol. 5822, Springer, Berlin,2009, pp. 641-652.

[4] A. Daz, P. Gervs, A. Garca, Evaluation of a system forpersonalized summarization of web contents, User Modeling 2005,LNCS, vol. 3538, Springer, Berlin, 2005, pp. 453462.

[5] H.P. Edmundson, New methods in automatic extracting, J. of theACM, vol. 16, no. 2, 1969, pp. 264285.

[6] X. Gong, X. Liu, Generic text summarization using relevancemeasure and latent semantic analysis, Proc. 24th Int. ACM SIGIRConf. Research and Development in Inf. Retrieval (SIGIR 01), ACMPress, 2001, pp. 19-25.

[7] H.P. Luhn, The automatic creation of literature abstracts, IBM J. ofResearch Development, vol. 2, no. 2, 1958, pp. 159165.

[8] R. Mro, I. Srba, M. Unk, M. imko, M. Bielikov, TowardsCollaborative Metadata Enrichment for Adaptive Web-basedLearning, Proc. Int. Conf. Web Intelligence and Intelligent AgentTech. (WI-IAT 11), IEEE Computer Society, 2011, pp. 106-109.

[9] J. Park, T. Fukuhara, I. Ohmukai, H. Takeda, S. Lee,Web Contentsummarization using social bookmarks: A new approach for socialsummarization, Proc 10th ACM workshop Web information anddata management (WIDM 08), ACM Press, 2008, pp. 103-110.

[10] J. Steinberger, K. Jeek, Text summarization and singular valuedecomposition, Proc. Advances in Information Systems (ADVIS04), LNCS, vol. 3261, Springer, Berlin, 2005, pp. 245-254.

[11] J. Sun et al., Web-page summarization using clickthrough data,Proc. 28th Int. ACM SIGIR Conf. Research and Development in Inf.Retrieval (SIGIR 05), ACM Press, 2005, pp. 194201.

[12] M. imko, M. Barla, M. Bielikov, ALEF: A framework foradaptive web-based learning 2.0, IFIP Advances in Information andCommunication Technology (KCKS 10), Springer, Berlin, vol. 324,2010, pp. 367-378.

[13] M. imko, Automated Domain Model Creation for Adaptive SocialEducational Environments, Information Sciences and TechnologiesBulletin of the ACM Slovakia, vol. 3, no. 2, 2011, pp. 119-121.

[14] H. Zhang, Z.C. Ma, Q. Cai, A study for documents summarizationbased on personal annotation, Proc. HLT-NAACL Workshop Textsummarization, Assoc. for Comp. Linguistics, 2003, pp. 4148.

135