Prospects for the application of scientometric methods for forecasting

ISSN 0147�6882, Scientific and Technical Information Processing, 2014, Vol. 41, No. 1, pp. 38–46. © Allerton Press, Inc., 2014.Original Russian Text © V.R. Mesropyan, M.V. Ovsyannikov, 2014, published in Nauchno�Technicheskaya Informatsiya, Seriya 1, 2014, No. 1, pp. 19–27.

38

INTRODUCTION

Foresight methodology (the methodology of thestudy of the future, “looking into the future”) is a setof methods and approaches that are used to systemat�ically assess the long�term prospects for the develop�ment of science, technology, economics, and society[1]. In a Foresight project the main task of theresearcher (unlike traditional prediction) does notconsist in a attempt to predict the parameters of somephenomenon or event, but in collectively shaping thetarget image of the future with the participation of allconcerned stakeholders and in the building of inte�grated strategies to achieve it.

The first experiments and projects related to theconstruction of a purposeful vision of the future stateof science and technology were made in the USA bythe RAND Corporation in the framework of internalresearch on the prospects for the development of thescience and technology areas of its interests (the Del�phi survey technique was developed). National Fore�sight activities projects were first organized in the1970s in Japan in order to bridge the gap between thiscountry and the world’s leaders in economic growth[2]. Since the beginning of the 1990s Australia [3, 4],the European Union, China, South Korea, Japan, andThailand have conducted national Foresight projectson a regular basis. The results of the studies are used toselect national priorities and to shape public programs.Figure 1 is a “diamond�illustrated” image of the most

common qualitative and quantitative methods that areused by Foresight methodology, where bibliometricsand the processing of patent data are emphasized.

Interest in formation the perspective shape of thefuture of science and technology, as history shows, istypical not only of developed countries, which are theengine of global science and technology progress(including the USA, Japan, UK, France and Ger�many), but also for young, emerging economies thatare in their infancy and growth (the Asia�Pacific coun�tries, the BRICS, and Latin America). It is an obviousfact that today even a rich and successful state is notable to invest in the development of science and tech�nology in all areas at the same time, since the processesof acquiring of new knowledge require a great degreeof spending on the training, procurement, and cre�ation of experimental equipment. It should be notedthat the escalating competition between countries inglobal high�tech markets leads to a permanent reduc�tion in the life cycle of such products; this trend in turnmakes new demands on participants to increase theeffectiveness of their science and technology policiesand national innovation system (hereinafter, NIS) [1].In this context, we should not forget about global chal�lenges and threats, sometimes with a complex inter�disciplinary nature, which can be realised in the mostunexpected scenarios: it is enough to bring the vividexamples of the global financial and economic crisis,the consequences of which still have dramatic effects

Prospects for the Application of Scientometric Methodsfor Forecasting

V. R. Mesropyana, b and M. V. Ovsyannikovb

aInstitute for Statistical Studies and Economics of Knowledge, National Research University Higher School of Economics, Moscow, 101000 Russia

bBauman State Technical University, Moscow, 105005 Russiae�mail: [email protected], [email protected]

Received November 12, 2013

Abstract—Forecasting studies, which are conducted in all the developed economies of the world, are one ofthe fastest growing applications of scientometrics. Currently, in Russia there is a formed state TechnologyForesight System, which creates a new format for the future development of research and for interactionamong the key players in the national innovation system. Considering the new institutional environment offorecasting projects in Russia and the international trends in the relevant research environments, scientomet�ric methods for the monitoring, analysis, and forecasting of the development of science and technology areidentified as one of the most important methods. Specific demands and trends, which will soon have a sub�stantial impact on the entire area of scientometrics, are formulated.

Keywords: scientometrics, bibliometrics, patent analysis, economic and technological cycles, Foresight, fore�casting, scientific–technical and innovation policy

DOI: 10.3103/S0147688214010080

SCIENTIFIC AND TECHNICAL INFORMATION PROCESSING Vol. 41 No. 1 2014

PROSPECTS FOR THE APPLICATION OF SCIENTOMETRIC METHODS 39

on the economies of individual countries and on theworld economy in general, as well as examples of theongoing crisis in the euro�zone.

Today, against the backdrop of the crisis and fol�lowing events that triggered a slowdown in global eco�nomic growth, two major strategies for their improve�ment and leadership achievement are formed: reform�ing financial and other public institutions (austerity,tax increases, etc.) and optimal development of theircompetitive advantage (primarily, due to the support,development, and accelerated output of innovativeproducts and services on world markets). Both of theseoptions should be used systematically, and in view ofgrowing resource constraints, even for advanced econ�omies, long�term priorities setting studies of innova�tion development have again become relevant. Thesetools should enable the creation of the science andtechnology groundwork and be capable of quicklyforming a base for access to world innovative marketsin the implementation of various global scenarios.

The trend towards the integration of forecastingwork in a wider range of future studies (Forward Look�ing Activities [6]), which has become a standard prac�tice, for example, in the policy�making process of theEuropean Commission) is entirely consistent with thisstatement. It should also be noted that great impor�tance is attached to forecasting activities as a tool todevelop sustainable ideas about the future (visions), toidentify possible disruptive [7] events, and to evaluatethe effects of the application of various policies in thedevelopment and management of the seventh and

eighth framework Programs of the European Union1

(hereinafter, EU) [8]. In this article the Times HigherEducation agency [9] states that the budget of theeighth Framework Program even against the Euro�zone crisis will be a record 80 billion euros (more than

1 Program of the European Union for the development of scienceand technology (the goals and projects in the programs vary indifferent financial periods).

Wild cards

Science fiction

Simulation games

Essays/scenarios

Genius forecast Role playing

SWOTBackcasting Brainstorming

Tree matches Scenario workshops

Road maps Delphi Scan Panel of citizens

Expert interviews Expert interviews; Seminars

Critical technologies Multi�criteria analysis Vote

Quantitative scenarios DMP maps

Interview Cross�impact analysis

Indicators Patents

Bibliometrics Benchmarking

Extrapolation

Literature review

Models

CreativityE

xam

inat

ion

Inte

racr

ion

ConclusivenessFig. 1. The diamond�matrix of Foresight methods [5].

40


MESROPYAN, OVSYANNIKOV

20 billion more than the previous seventh program).This underlines the basic rating of the EU on innova�tive strategies for sustainable development and pros�perity, in which advanced science and technology playkey roles.

ANALYSIS AND PROCESSINGOF SCIENTOMETRIC DATA

ON THE INTELLIGENT INFORMATION SPACE

During the last 20 years monitoring techniques inresearch and development, based on the analysis ofscientific citation information in specialized data�bases, have shown rapid growth. The appearance ofthis trend is caused by the development of technolo�gies for processing text electronic documents and web�based applications that provide access to them. Todate, the analysis of the array of patents (patent min�ing) and citation statistics contained in the scientificliterature are widely used by ministries and agenciesand large corporations and are also used in nationalforecasting projects in many developed countries toscan and to monitor trends and competitive intelli�gence [10–14].

Bibliometric statistics, namely, the rapid growth ofpublications on specific areas or a “splash” of refer�ences to previously published publications that werelittle cited in the first years after their release, allowone to detect various trends and to monitor the devel�opment of science and technology areas, to assess theeffectiveness of individual scientists and researchgroups at the national and international levels, as wellas to carry out other necessary comparisons.

However, according to the authors of this article,scientometric methods can be used much more widelyto predict the most prominent science and technologypriorities. To support this argument using these meth�ods we will further empirically examine the hypothesisof the cyclic dynamics of science and technology,which were supported by the economists N. Kon�dratiev [15], J. Schumpeter [16], K. Perez [17], and S.Glazyev [18] in different periods of the 20th century.

We use the following terms to facilitate further pre�sentation.

The intelligent information space (hereinafter, IIS),is a set of modern information resources, which accu�mulate information about the results of the researchactivities of the entire international community.

The intellectual environment is the totality of IIS,organizational resources (subjects: researchers, man�agers, decision makers, etc.) and of high�tech marketand services.

Today, a new paradigm of the relationships in sci�ence and technology community continues to beshaped under the influence of information and com�munication technology development. Any researchpasses a certain cycle, starting with information

retrieval and analysis of the global experience and end�ing with the publication of its results in the most pres�tigious publications (as much as possible). Thus, thisresearch cycle ends in the IIS, which is certainly a sig�nificant “factor” of science, that possesses the corre�sponding responses to the economic, social, and otherprocesses that occur in the science and technologycommunity. Based on this, it seems appropriate tobuild complex models of the development of scienceand technology fields, entering citation information asone of the significant factors. Such models enrich theForesight methodology and adapt it for use in the cor�porate sector and in the construction of developmentstrategies in higher education sector.

On the Internet there are a large number of servicesthat provide access to databases of scientific literaturein an automated way for publishers and patent agen�cies to develop custom analytical software tools. TheSCOPUS web�based platform of the Elsevier Com�pany [19] and Web of Knowledge (WoK) of ThomsonReuters Company [20] are the world leaders in provid�ing information and analytical support based on anal�ysis of bibliometric (including patent) data.

In this study we obtained a sample of patents relat�ing to CRT (cathode�ray tube, hereinafter CRT) tech�nology from 1969 to 2011, (Fig. 2), using the patentGlobal Patent Index database (hereinafter, GPI). TheEuropean Patent Agency provides access to this data�base in a user�operating mode on the official website[21] and in an open service for developers (protocolsSOAP/WSDL).

This example clearly shows that the development ofthe technical ideas of the electronic ray tube had a pro�nounced upward trend until 1989 and a steady down�ward trend since 1992. Using similar operations, webuilt a histogram of sample patents on LCD technol�ogy (liquid�crystal display), which has the same char�acter as shown in Fig. 2, but is at the beginning of itsdowntrend. In general, we analyzed about 200 equi�distant time series that were obtained in such a mannerand that show an annual increase in the number ofpatents in the GPI database, and they all have thecharacteristic shapes of the trend. Here, it is importantto immediately identify the criteria of the search: theymust be sufficiently clear and transparent in order tohighlight the “clean” sample, so that an additivesuperposition of two or more semantic lexically similarsets would not occur. We emphasize that in all theexperiments we performed, a full�text search was per�formed in the mandatory section of the patents“abstract,” i.e., in the summary.

The histogram shape of the CRT�technology timeseries (see Fig. 2) is interesting; it is obvious that itmeets the hypothesis of the cyclical development ofthe market for goods and services, correlating withN.D. Kondratiev`s theory of the cyclical dynamics ofsocio�economic environments in [15], as well as with



the later works of J. Schumpeter [16], K. Perez [17],and S. Glaziev [18] (Fig. 3).

We should pay attention to the “need” curve onFigure 3: in our case it is the need market participantshave for screens and displays, but in general it is theneed for a visual human–machine interface. We nowconsider the curve below, the “n” technology, whichfor the most part repeats the form of the obtainedgraphical result on CRT technology development.The family of curves that are located below the curveconsists of the specific subtypes of the selected tech�nology. According to the estimated source [22], webelieve that the peak sales of devices with CRT tech�nology occurred in 2000 and stood at 11 million unitsworldwide; since then sales have steadily fallen. Wecan then assume that the sales curve should have asimilar shape with a peak near 2000. A total of 9 yearshave passed since the breaking of the trend in the tech�nological curve in 1991 that we found and the declinein sales in the end of second millennium. Thus, sincethe beginning of 1992 most of the patents that wereproduced had either a serving or improving localnature or improving character. Thus, if we as a devel�oper focused on trends of intellectual environment, wewould obtain timely signals of changes in technologi�cal trends and the need to boost research and techno�logical development for technologies that replaceCRTs.

Building on Kondratiev`s theory of cyclicaldynamics, the given CRT technology lifecycle is a

medium�term cycle; it was replaced by the followingscientific and technological cycle, LCD technology inthe current generation of visual human–machineinterface technology (Fig. 4).

Both the scientific and technical cycles should bedeveloped under the long�term development cycle ofthe generation of visual human–machine interfacetechnology, which is confirmed by the behavior of theincrement of patents in the study of the intelligentinformation space.

Figure 5 shows a histogram of patents growth in theGPI database that is relating to the technology ofimage display, from 1969 to 2011. Based on the volumeand dynamics of the development of the given set ofbibliographic entities in IIS (patents), we concludethat they are in a long�term compliance (opportunis�tic) dynamic cycle of the generation of display� andscreen�technology development according to the pro�visions of the socio�economic theory of N.D. Kon�dratiev. We note the fact that both the medium�cyclesaccounted for the phase of the large increase of theconjunctural cycle, which has entered a decline. Thesefindings suggest that the following cycles within thegiven generation of screen technology will be charac�terized by lower levels of success and more extendedperiods of crisis.

Comparing the dynamics of the array of publica�tions and patents (under publications we mean a set ofarticles in scientific journals), we concluded that the

2000

1800

1600

1400

400

1200

200

0

1000

800

600

1969

1970

1971

1972

1973

1974

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2007

2008

2009

2010

2011

2006

Fig. 2. Distribution of patents relating to CRT technology, in 1969–2011.

42



rate of development trends in the patent environmentis ahead of the processes in environment publications.Ultimately information on fundamentally new tech�nological concepts and results should be sought inIIPS publications, and the priorities of applied

research areas should be analyzed and constructedaccording to IIS patent data.

The question of the nature of the mutual influenceof the behavior of real production capital, innovationagents in the economy and the events that unfold in

Need

Technical and technological

Vo

lum

e in

dic

ato

r

Technology «n»

solution «m»

Product type«e»

The specific

product brand«e»

Time

Fig. 3. Life�cycle model of demand and technologies.

8000

7000

2000

6000

1000

0

5000

4000

3000

1969

1970

1971

1972

1973

1974

1975

1976

1977

1978

1979

1980

1981

1982

1983

198419

8519

8619

8719

8819

8919

9019

9119

9219

9319

9419

9519

9619

9719

9819

992000

20012002

20032004

200520

072008

20092010

201120

06

CRT

LCD

Fig. 4. Diagram of patents for the combined distribution for CRT and LCD technologies from 1969 to 2011.



the intelligent information space is of some interest. Inorder to illustrate the presence of systemic linkagesbetween these areas, we provide an example from thehistory of science and technology: the mass produc�tion of vehicles using rotary engines began in the late1960s; in particular the AUDI NSU Ro 80 was one ofthe most advanced and ahead of its time models. Thismodel was serial produced from 1967 until 1977 and in1969 it was named the “European Car of the Year”[23]. The unprecedented wave of popularity and obvi�ous advantages in terms of engine displacement\powermade rotary engines the main candidate to replacetheir piston counterparts. However insurmountabletechnological limitations on the length of service lifeand low fuel efficiency led to the fact that the car waspoor in market relations; the last production modelswere sold for low sums and in 1977 the series was com�pletely terminated. The histogram of the time series ofthe patents on the technologies of rotary engines (Fig. 6)clearly shows how the intellectual environment“responded” to the above facts. Until 1975, the com�pany engineers and scientists who worked in differentsectors of the economy believed in the potential ofrotary technology, which provided stable investmentgrowth and as a consequence, “intellectual capital.”However, the rapid collapse of the automotive marketapparently led many followers to abandon furtherresearch and development in this direction; thus, theintellectual environment was capable of providing sig�nals even before the market data, as it responded toevents at the first stage in the linear model “researchand development–technology—innovation.”

CURRENT TRENDS IN THE DEVELOPMENT OF FORECASTING STUDIES IN RUSSIA

The scope of science and technology forecasts inthe last 60 years has passed through a long evolutionarypath: different types of projects were implemented atdifferent times with reference to the structures of thestate institutions of executor countries and it is easy tosee how the major trends that unfold in politics and inthe social and global economy affected the develop�ment of Foresight methodology itself and its place inscience, technology, and innovation policy. Thesetrends are reflected in the form of the five currentlyallocated generations of forward�looking studies [24]:if in the initial stage the role of forecasts, which waslargely confined to informing decision makers aboutthe internal dynamics of the development of scienceand technology (first generation), then the followingtypes of studies cover the likely contribution of scienceto the solution of certain economic and social prob�lems (second generation); wider social measurementand analysis of prospects for the development of alter�native institutions (third generation); intersectoralcoordinated assessments of the future of science andinnovation (fourth generation); and finally, the pros�pects for the development of the national innovationsystem (NIS) structures and the scientific and techno�logical aspects of socio�economic development ingeneral (fifth generation). Thus, we can clearly tracethe global trend of the changing role of forward�look�ing studies from a purely informative function to thefullest possible integration in the process of the forma�tion and updating of science and technology policies

160000

140000

120000

100000

80000

60000

40000

20000

0

1969

1971

1973

1975

1977

1979

1981

1983

1985

1987

1989

1991

1993

1994

1997

1999

2001

2003

2005

2007

2009

2011

Fig. 5. Distribution of patents relating to the technical concepts of image displays, from 1969 to 2011.

44



[8]. Based on the example of planned major projectsthat have been undertaken in the Russian Federation,the five above�mentioned generations can be traced inseries: from pure forecasts of the first generation sci�ence, technology, and engineering development (thefirst round of the development of the priority areas ofscience, technology, and engineering, and the list ofcritical technologies) to complex projects that coverthe scientific, technological, and socio�economicspheres (long�term Foresights of science and technol�ogy development carried out under the auspices of the

Ministry of Science and Education of Russia2, the

results of which contain a wide range of recommenda�tions for science, technology, and innovation policies.It is noteworthy that Russian researchers have takenthis evolutionary path in less than 20 years, which sug�gests a high degree of mobility and involvement in theinternational academic community of related profes�sionals. At the same time according to the experts,modern domestic studies (probably with the exceptionof the long�term prognosis of the science and technol�ogy development of Russia, which has a high qualitystandard and is patronized by an international advi�sory council) belong to the “4 +" generation, whichmeans the formation of a high�quality vector from thefourth to the fifth stage of development. Thus, thetransition to the authentic fifth generation of forward�looking studies in Russia demanded the establishmentof effective mechanisms for making linkages betweenthe goals and objectives of projects, the coordinationof the methodological and organizational approachesof executors and greater integration of research results

2 Long�term forecasting of scientific–technological develop�ments in Russia for the period until 2030. URL:http://prognoz2030.hse.ru/ 12).

into policy and procedures for making strategic deci�sions.

A significant change in the framework conditionsfor the science, technology, and innovation policy ofthe Russian Federation was the answer to this inquiry:on July 1, 2013 in accordance with the decree of thePresident of the Russian Federation, a TechnologyForesight System that is focused on ensuring the futureneeds of the manufacturing sector, with the develop�ment of key manufacturing technologies (hereinafter,the system of technological forecasting) was formed.An interministerial committee on technology Fore�sight of Presidium Council under the President of theRussian Federation on economic modernization andinnovative development of Russia (hereinafter, theIMC) was previously established in order to ensure theorganizational and coordination support of systemactivities.

The goals of the System can be divided into twomain groups:

1. The achievement of ultimate outcomes:a) The development of a national long�term Fore�

sight of the science and technology development ofRussia;

b) The formation and updating of national lists ofthe priority directions of science and technologydevelopment of critical technologies;

c) The development of industrial long�term scienceand technology Foresights;

d) The formation and actualization of industrialcritical technologies;

e) The organizing and conductance of the annualmonitoring of science and technology development;

f) The development of roadmaps of economy sec�tors and areas of science and technology development;

1950

1953

1957

1960

1963

1966

1969

1972

1975

1978

1981

1984

1987

1990

1993

1996

1999

2002

2005

2008

700

600

500

400

300

200

100

0

Fig. 6. Histogram of the time series of patents relating to rotary�engine technology, from 1950 to 2011.



g) The preparation of proposals for the use of theresults of the Technology Foresight System in the stra�tegic documents of state planning.

2. Ensuring the effectiveness of the system’s workprocesses:

a) The coordination and methodological supportof the federal executive bodies and other stakeholdersin the conducting of future studies;

b) The support and development of a national data�base of experts;

c) The development of communication platformsfor the discussion and use of forecasts;

d) The improvement of the methodology and thedevelopment of common work standards within theTechnology Foresight System;

e) The creation of a united public database of fore�casting research materials.

It should be noted that at present, a system for thestate strategic planning of the socio�economic devel�opment of Russia has been formed (in accordancewith the above�mentioned decree of the President ofthe Russian Federation), which is provided by thedraft federal law “On the state strategic planning”,which regulates the coordination, strategic manage�ment, and fiscal policy measures (hereinafter, billdraft).

In accordance with the reorganization of the insti�tutional conditions for the development of a long�termForesight of the science and technology developmentof Russia until 2030 science, technology and innova�tion policy (hereinafter, the LTF) is one of the mostimportant documents in the system of state strategicplanning, which is aimed at methodical, information,and expert�analytical support for the development ofmanagement decisions. Its main task involves thedevelopment of the priorities of the science and tech�nology development of Russia for the most effectiveimplementation of its competitive advantages.

Following the results of the LTF, development ofthe most perspective areas, in terms of their imple�mentation in Russia, was chosen from a wide range ofareas of science and technology development. Fore�sight results are of a diverse character; not only werethe directions of research and development defined,but also the promising markets and product groupswithin which their results may find application.

The national Technology Foresight System pro�vides the gradual alignment of mechanisms for the useof the forecast results:

Stage 1: the informing of stakeholders about theresults of the forecast, the joint development of mech�anisms for their use with the participation of the fed�eral executive bodies and organizations;

Stage 2: the formation of mechanisms for the use ofthe forecast results in the development, implementa�tion, and adjustment of the public programs of the

Russian Federation, including the federal target pro�gram for scientific and technological orientation;

Stage 3: the creation of mechanisms for the use ofthe forecast results in the development, implementa�tion, and adjustment of the state strategic�planningdocuments of the socio�economic development of theRussian Federation.

Accordingly, the conclusions and recommenda�tions that are contained in the long�term prognosis forthe science and technology development of Russia canbe used by different stakeholders for different pur�poses:

• by federal executive authorities: in course of theworking�out, implementation, and adjustment ofpublic�sector strategic�planning documents and gov�ernment programs of the Russian Federation, includ�ing the federal target program for science and technol�ogy orientation;

• by companies with state participation, imple�menting innovative development programs, technol�ogy platforms, innovative regional clusters, in theimplementation and adjustment of the relevant policydocuments;

• by universities and research institutions: in deter�mining the priority areas of the development, imple�mentation, and adjustment of strategic�developmentdocuments;

• by private business: in the development andimplementation of research and production programsand projects, in finding of technology partners.

CONCLUSIONS ON NEW DEMANDSAND NEEDS FOR BIBLIOMETRIC METHODS

WITHIN THE DEVELOPMENT OF THE TECHNOLOGICAL

FORECASTING SYSTEM

The Technology Foresight System presents a set ofbasic requirements for future studies. These requeststo the research community are formed on the basis ofthe analysis of foreign and Russian experience in thefield of Foresight methodology. Thus, one of the maintrends that was observed in the last decade, involves anorientation toward forecast building that has a sub�stantial quantitative “evidence” database that isaccepted by all project participants and the next trendis the integration of qualitative and quantitative meth�ods for the estimation and forecasting of scientific andtechnological developments; during this integrationthe second component gains a stronger position in theresearch environment [25]. In this regard, it is difficultto overestimate the role of bibliometric studies inmodern Foresight projects and we should note thegrowing popularity of this direction of scientific andtechnological development analysis throughout theworld and in Russia, in particular. However, the accel�erating progress of information and communicationtechnologies and data�processing methods, as well as

46



current trends in science, technology, and innovationpolicy, create a demand on Scientometric models andtools from the community of forecasting researchers(including a new institutional framework of the stateTechnology Foresight System).

According to the authors of this article, currentlyone can distinguish four main groups of trends that inthe medium and long term will significantly impactthe future trajectory of scientometrics in its theoreticaland practical areas.

1. The development of forward�looking studiesindicates the need for new methods and forecastingmodels for the development of research projects basedon the processing of bibliometric and other informa�tion.

2. Improving the accuracy and depth of informa�tion processing is based on scientometric achieve�ments in semantic data�processing methods, high�performance search algorithms, clustering, classifica�tion, network analysis, etc.

3. The competition between key suppliers of scien�tific data and software in the international market isincreasing now between large firms; thus, the size ofthe market is growing and there are many “niche”products around flagship platforms. We cannotexclude the possibility of future revolutionary newbusiness models that will radically change the land�scape balance of the forces on the affected markets.

ACKNOWLEDGMENTS

This work was supported by the Russian Founda�tion of the Humanities, grant no. 13�03�00273.

REFERENCES

1. Sokolov, A.V., Foresight: A look into the future, Fore�sight�Russia, 2007, vol. 1(1), pp. 8–15.

2. Miles, I., New horizons and challenges for future�ori�ented technology analysis, Proc. 1st Int. Seville Sem. onFuture�Oriented Technology Analysis: Impact of FTAApproaches on Policy and Decision�Making, Seville,2002.

3. Freeman, C., Technology Policy and Economic Perfor�mance: Lessons from Japan, London: F. Pinter, 1987.

4. Gaponenko, N.V., Forsait. Teoriya. Metodologiya. Opyt(Foresight. Theory. Methodology. Experience), Mos�cow: YuNITI�DANA, 2008.

5. Popper, R., Methodology: Foresight practices andtools, in International Handbook on Foresight and Sci�ence Policy: Theory and Practice, Georghiou, l. et al.,Eds., London: Edward Elgar, 2006.

6. European Commission Facing the Future: Time for theEU to Meet Global Challenges, Luxembourg: Publica�tions Office of the European Union, 2010.

7. Christensen, M., Roth, E., and Anthony, S., Seeingwhat’s Next: Using Theories of Innovation to PredictIndustry Change, Cambridge: Harvard Business, 2004.

8. Sokolov, A.V. and Chulok, A.A., Russian Science andTechnology Foresight – 2030: Key Features and FirstResults, Foresight�Russia, 2012, vol. 6(1), pp. 12–25.

9. “Triple miracle” sees huge rise in EU funds for frontierresearch. http://www.timeshighereducation.co.uk/416952.article

10. Ruotsalainen, L., Data mining tools for technology andcompetitive intelligence. http://www.vtt.fi/inf/pdf/tiedotteet/2008/T2451.pdf

11. Kirsch, G. and Brown, C., Using patents in competitiveintelligence SCIP Compet. Intel. Mag., 2006, vol. 9(1),pp. 17–21.

12. Cabena, P., Hadjinian, P., Stadler, R., Verhess, J., andZanasi, A., Discovering Data Mining: from Concept toImplementation, New York: Englewood Cliffs, 1997.

13. Hehenberger, M. and Coupet, P., Text mining appliedto patent analysis, Proc. 1998 Annual Meeting of Ameri�can Intellectual Property Law Association (AIPLA),Arlington, 1998.

14. Krier, M. and Zacca, F., Automatic categorizationapplications at the European patent office, WorldPatent Inf., 2002, vol. 24(3), pp. 187–196.

15. Kondrat’ev, N.D., Bol’shie tsikly kon"yunktury i teoriyapredvideniya (Large Cycles of Conjuncture and Theoryof Foresight), Moscow: Ekonomika, 2002.

16. Shumpeter, J.A., Teoriya ekonomicheskogo razvitiya(Issledovanie predprinimatel’skoi pribyli, kapitala, kred�ita, protsenta i tsikla kon"yunktury) (Theory of Eco�nomic Development (Study of Profit of Enterprise,Capital, Credit, Percent and Conjuncture Cycle), Lon�don: George Alien and Unwin, 1952; Moscow:Progress, 1982.

17. Perez, C., Technological Revolutions and Financial Cap�ital: The Dynamics of Bubbles and Golden Ages, London:Elgar, 2002.

18. Glaz’ev, S. and Mikerin, G., Dlinnye volny NTP i sot�sial’no�ekonomicheskoe razvitie, (Long Waves: Scien�tific and Technical Progress and Social�EconomicDevelopment), Moscow: Nauka, 1989.

19. Sciverse. http://www.sciverse.com

20. Web of Knowledge. http://wokinfo.com/

21. http://www.epo.org/searching/subscrip�tion/expert.html

22. Khomchuk, O.P., Review of screen technology devel�opment. http://www.podberi.tv/review/479

23. Article on the NSU Ro 80 motor car. http://ru.wikipe�dia.org/wiki/NSU_Ro_80

24. Georghiou, L., Cameron, H., Keenan, M., Miles, I.,and Saritas, O., An Evaluation of the UK Foresight Pro�gram: Final Report, Manchester: Univ. Manchester,2006.

25. Haegeman, K., Marinelli, E., Scapolo, F., Ricci, F.,and Sokolov, A., Quantitative and qualitativeapproaches in future�oriented technology analysis(FTA): From combination to integration? Technol.Forecasting Soc. Change, 2013, No 80, pp. 386–397.

Translated by O. Kupriyanova�Ashina

Documents

Prospects for the application of scientometric methods for forecasting