Mendeley as a Source of Readership by Students and Postdocs? Evaluating Article Usage by Academic Status

Mendeley as a Source of

Readership by Students and Postdocs?

Evaluating Article Usage by Academic Status

[email protected]

@stefhaustein

Stefanie Haustein

& Vincent Larivière

Introduction

Measuring use of scholarly documents

• reshelving, interlibrary loan

• citation analysis

• electronic full text access

• social reference managers and bookmarking systems

Introduction

Research questions:

• Can Mendeley readership counts be used to monitor use

of scholarly documents?

• Does use differ between scientific fields?

• Can different user sectors and user types be identified

based on the academic status?

• Can the data be used to determine whether specific user

groups predict citation impact?

Introduction

Introduction

• 2.8 million users, 275,860 groups, 535 user documents (02/2014)

• monthly growth rate of 3.7% (documents) and 2.3% (users) 2013

• 68 million unique publications (08/2012; 281 million user

documents)

Mendeley statistics based on monthly user counts from 10/2010 to 02/2014 on the Mendeley website accessed through the Internet Archive

Data sets & methods

• 1,161,145 PubMed papers covered by WoS

• publication years: 2010-2012

• document types: articles & reviews

• NSF disciplines: Biomedical Research, Clinical Medicine, Health, Psychology (journal-based classification)

• open citation window

• Mendeley readership data collected via API

• Levenshtein distance (5%) to account for errors in

metadata

• document title (long titles = 70 characters, 5 words)

• document title and first author name (short titles)

1.7% false positives, 0.7% false negatives

Data sets & methods

• aggregating reader counts of multiple entries

Data sets & methods

• number of readers per academic status

• number of missing readership status per paper

29% 7 PhD students

21% 5 Master students

8% 2 Doctoral students

58% 14 readership status available

42% 10 missing readership status

Data sets & methods

• aggregating academic status information

Results: disciplines

available missing

all disciplines 1,161,145 7.5 65.9% 9.6 8.9 0.512 ** 70.0% 30.0%

Biomedical Research 286,398 10.3 72.4% 14.3 11.8 0.575 ** 69.5% 30.5%

Clinical Medicine 779,707 6.8 62.8% 7.6 8.2 0.492 ** 70.5% 29.5%

Health 59,073 4.4 67.0% 6.5 4.3 0.434 ** 72.8% 27.2%

Psychology 35,967 6.1 81.0% 14.0 6.6 0.545 ** 67.5% 32.5%

ρReadership statusNSF discipline

Papers

PubMed

& WoS

Mean

citation

rate

Papers with readers

%Mean

reader rate

Mean

citation rate

• two-thirds of papers saved at least once on Mendeley

• reader rate comparable to citation rate

• Spearman correlations between citations and reader counts

moderately positive (ρ=0.445** / ρ=0.512**)

• academic status not available for 30% of reader counts

Results: specialtiesDifferences between specialties

%Mean

reader rate

all disciplines 65.9% 9.6 0.512 **

Biomedical Research 72.4% 14.3 0.575 **

Anatomy & Morphology 68.2% 5.5 0.380 **

Biochem & Mol Biol 71.6% 12.4 0.550 **

Biomedical Engineering 74.9% 10.4 0.513 **

Biophysics 78.6% 11.8 0.537 **

Cell Biol, Cytol & Histol 74.7% 14.3 0.584 **

Embryology 79.2% 13.2 0.649 **

Gen Biomed Research 72.5% 35.1 0.689 **

Genetics & Heredity 74.1% 17.3 0.558 **

Microbiology 72.7% 10.4 0.555 **

Microscopy 72.5% 6.7 0.494 **

Misc Biomed Research 74.3% 8.8 0.585 **

Nutrition & Dietetic 66.9% 6.6 0.494 **

Parasitology 66.0% 6.1 0.436 **

Physiology 72.1% 8.0 0.457 **

Virology 68.9% 7.1 0.534 **

Papers with readers

ρNSF discipline or specialty

Size of data points represents mean reader rate.

Anatomy & Morphology

General Biomedical Research

Embryology



%Mean

reader rate


Clinical Medicine 62.8% 7.6 0.492 **

Addictive Diseases 68.2% 5.8 0.436 **

Allergy 69.8% 8.3 0.582 **

Anesthesiology 63.0% 6.8 0.497 **

Arthritis & Rheumatology 63.3% 6.3 0.488 **

Cancer 62.8% 7.3 0.550 **

Cardiovascular System 56.6% 7.4 0.555 **

Dentistry 68.5% 5.6 0.398 **

Dermat & Venerial Dis 51.3% 4.2 0.433 **

Endocrinology 64.4% 7.1 0.518 **

Environ & Occupat Health 66.1% 6.9 0.501 **

Fertility 64.4% 4.3 0.417 **

Gastroenterology 58.1% 6.0 0.508 **

Gen & Internal Medicine 51.8% 8.2 0.519 **

Geriatrics 73.5% 7.5 0.494 **

Hematology 59.5% 6.9 0.557 **

Immunology 65.8% 9.1 0.561 **

Misc Clinical Medicine 70.6% 9.1 0.458 **

Papers with readers


Veterinary Medicine

Psychiatry

Neurology & Neurosurgery



%Mean

reader rate


Clinical Medicine 62.8% 7.6 0.492 **

Nephrology 63.9% 5.3 0.458 **

Neurol & Neurosurgery 73.1% 13.6 0.554 **

Obstetrics & Gynecology 60.4% 4.3 0.420 **

Ophthalmology 63.0% 4.4 0.486 **

Orthopedics 66.0% 6.9 0.449 **

Otorhinolaryngology 59.7% 4.1 0.383 **

Pathology 60.1% 5.3 0.503 **

Pediatrics 62.0% 5.8 0.469 **

Pharmacology 63.4% 6.5 0.501 **

Pharmacy 55.9% 4.8 0.405 **

Psychiatry 72.1% 9.2 0.583 **

Radiol & Nucl Medicine 63.9% 6.8 0.467 **

Respiratory System 65.1% 6.8 0.487 **

Surgery 58.0% 4.2 0.420 **

Tropical Medicine 65.4% 5.8 0.478 **

Urology 54.8% 4.1 0.432 **

Veterinary Medicine 66.3% 7.5 0.236 **

Papers with readers


Veterinary Medicine

Psychiatry

Neurology & Neurosurgery



%Mean

reader rate


Health 67.0% 6.5 0.434 **

Geriatrics & Gerontology 69.8% 7.3 0.540 **

Health Policy & Services 66.1% 6.8 0.421 **

Nursing 62.0% 5.1 0.378 **

Public Health 66.0% 6.0 0.439 **

Rehabilitation 73.0% 8.0 0.434 **

Social Sciences, Biomed 76.0% 9.2 0.495 **

Social Studies of Med 49.5% 3.1 0.281 **

Speech-Lang Path & Audio 79.0% 7.7 0.436 **

Psychology 81.0% 14.0 0.545 **

Behav Sci & Compl Psych 83.4% 12.2 0.503 **

Clinical Psychology 80.7% 11.1 0.536 **

Develop & Child Psych 80.3% 13.2 0.531 **

Experimental Psychology 85.6% 19.2 0.582 **

General Psychology 68.5% 9.3 0.493 **

Human Factors 84.2% 9.2 0.434 **

Misc Psychology 79.3% 11.4 0.531 **

Psychoanalysis 39.5% 3.6 0.137

Social Psychology 82.4% 24.8 0.687 **

Papers with readers


Social Studies of Medicine

Geriatrics & Gerontology

Psychoanalysis

Social Psychology

Results: sectors

Results: sectors

• Biomedical Research papers mostly used by readers from

scientific sector

• more professionals in Clinical Medicine

• more educational and professional users in Health

• more educational, less professional users in Psychology

%Mean

reader rate

Mean

citation rateScientific Educational Professional missing

all disciplines 65.9% 9.6 8.9 0.512 ** 48.5% 15.7% 5.8% 30.0%

Biomedical Research 72.4% 14.3 11.8 0.575 ** 54.9% 12.0% 2.6% 30.5%

Clinical Medicine 62.8% 7.6 8.2 0.492 ** 44.2% 17.6% 8.7% 29.5%

Health 67.0% 6.5 4.3 0.434 ** 38.1% 27.3% 7.4% 27.2%

Psychology 81.0% 14.0 6.6 0.545 ** 46.6% 19.0% 1.8% 32.5%

NSF discipline

Papers with readers Sector type of readership status

ρ

Results: sectors

y = 0.0031x + 0.3823R² = 0.433

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Ve

teri

nar

y M

edic

ine

De

nti

stry

Mis

c C

linic

al M

edic

ine

Nu

rsin

gSo

cial

Stu

die

s o

f M

edR

eh

abili

tati

on

An

esth

esio

logy

Ob

ste

tric

s &

Gyn

eco

logy

Ort

ho

ped

ics

Uro

logy

De

rmat

& V

ene

rial

Dis

Nu

trit

ion

& D

iete

tic

Surg

ery

Pu

blic

Hea

lth

Tro

pic

al M

edic

ine

Pe

dia

tric

sFe

rtili

tyH

ealt

h P

olic

y &

Ser

vice

sO

ph

thal

mo

logy

Ph

arm

acy

Oto

rhin

ola

ryn

golo

gySp

eec

h-L

ang

Pat

h &

Au

dio

Art

hri

tis

& R

he

um

ato

logy

Ad

dic

tive

Dis

eas

es

Gen

& In

tern

al M

ed

icin

eC

ard

iova

scu

lar

Syst

emP

sych

oan

alys

isR

esp

irat

ory

Sys

tem

Nep

hro

logy

Ger

iatr

ics

Alle

rgy

Envi

ron

& O

ccu

pat

He

alth

Soci

al S

cie

nce

s, B

iom

ed

Pat

ho

logy

Hu

man

Fac

tors

Ph

arm

aco

logy

Gas

tro

ente

rolo

gyP

aras

ito

logy

Gen

eral

Psy

cho

logy

End

ocr

ino

logy

Rad

iol &

Nu

cl M

edic

ine

Mis

c B

iom

ed

Re

sear

chG

eria

tric

s &

Ge

ron

tolo

gyC

linic

al P

sych

olo

gyP

sych

iatr

yA

nat

om

y &

Mo

rph

olo

gyC

ance

rP

hys

iolo

gyB

iom

edic

al E

ngi

ne

erin

gM

isc

Psy

cho

logy

Soci

al P

sych

olo

gyH

emat

olo

gyV

iro

logy

Imm

un

olo

gyB

eh

av S

ci &

Co

mp

l Psy

chD

eve

lop

& C

hild

Psy

chM

icro

bio

logy

Exp

erim

enta

l Psy

cho

logy

Neu

rol &

Ne

uro

surg

ery

Mic

rosc

op

yB

ioch

em

& M

ol B

iol

Bio

ph

ysic

sG

enet

ics

& H

ere

dit

yEm

bry

olo

gyC

ell

Bio

l, C

yto

l & H

isto

lG

en B

iom

ed R

ese

arch

Perc

enta

ge o

f re

ader

s p

er s

ecto

r

Spea

rman

co

rrel

atio

n b

etw

een

cit

atio

ns

and

rea

der

co

un

ts

Professional Educational Scientific missing Spearman's ρ

Results: users

Results: users

Results: users

0.575**

0.559**

0.534**

0.478**

0.435**

0.426**

0.410**

0.396**

0.353**

0.318**

0.224**

0.089**

0.234**

0.135**

0.183**

0.059**

0.071**

0.059**

0.066**

0.074**

0.049**

0.042**

0.040

0.051

all readers

Postdoc

PhD Student

Researcher (Academic)

Student (Postgraduate)

Researcher (Non-Academic)

Professor

Assistant Professor

Student (Bachelor)

Associate Professor

Other Professional

Librarian

Biomedical Research

all documents (n=207,255) 100% available reader status (n=80,858)

All documents

• postdocs and PhD

students most

similar to citations

• librarians least

similar

100% status info

• PhD students and

Postdocs most


• other professionals

and associate

professors least

similar

Results: users

0.492**

0.451**

0.425**

0.410**

0.408**

0.364**

0.361**

0.317**

0.300**

0.183**

0.137**

0.055**

0.238**

0.075**

0.093**

0.174**

0.121**

0.067**

0.059**

0.056**

0.079**

0.050**

0.030**

0.029**

all readers



PhD Student

Postdoc

Assistant Professor

Professor

Associate Professor

Other Professional


Student (Bachelor)

Librarian

Clinical Medicine


All documents

• researchers most



similar

100% status info


Postdocs most


• Bachelor students

and librarians least

similar

Results: users

0.434**

0.340**

0.329**

0.320**

0.307**

0.282**

0.280**

0.276**

0.266**

0.250**

0.214**

0.083**

0.196**

0.127**

0.099**

0.038

0.000

0.093**

0.021

0.004

0.076**

0.044

0.058**

-0.028

all readers

PhD Student



Postdoc


Professor

Assistant Professor

Other Professional

Associate Professor

Student (Bachelor)

Librarian

Health


All documents

• low correlations

• PhD students,

researchers and

postdocs most

similar

100% status info


Postdocs most


• no similarity for

librarians and

postdocs

Results: users

0.545**

0.480**

0.480**

0.425**

0.403**

0.400**

0.368**

0.356**

0.321**

0.299**

0.282**

0.107**

0.189**

0.158**

0.125**

0.082**

0.070

0.052

0.076*

0.048

0.037

0.120**

0.048

-0.069

all readers

PhD Student

Postdoc


Professor


Assistant Professor

Student (Bachelor)


Other Professional

Associate Professor

Librarian

Psychology


All documents


postdocs most



similar

100% status info

• PhD students,

postdocs and other

professionals most


• negative

correlation for

librarians

Conclusions: general results

• Mendeley important source of documents’ usage

• 2.8 million users, 535 million user documents

• 65.9% of sampled documents saved 9.6 times on average

• reader counts reflect similar but broader use of scholarly

documents than citations

• Spearman’s ρ = 0.445**/0.512**

• PhD students, postgraduate students and postdocs largest

user group, librarians the smallest

Conclusions: general results

• differences between disciplines and specialties

• coverage:39.5% (Psychoanalysis) – 85.6% (Experimental Psychology)

• reader rate:3.1 (Social Studies of Medicine) – 35.1 (General Biomedical Research)

• correlation with citations:0.137 (Psychoanalysis) – 0.687** (Social Psychology)

• user sector

• scientific:27.7% (Veterinary Medicine) – 63.2% (Microscopy)

• educational:8.6% (General Biomedical Research) – 32.5% (Dentistry)

• professional:0.7% (Experimental Psychology) – 18.6% (Veterinary Medicine)

Limitations

• metadata quality

• academic status self-reported

need to verify whether accurate and up-to-date

• restriction to top 3

• differences between user groups cannot be determined due

to data restriction

• similarity with citation patterns of different user groups cannot

be accurately determined

• even more problematic for countries and disciplines

complete data needed for detailed and accurate statistics

Stefanie Haustein

Thank you for your attention!

Questions?

[email protected]@stefhaustein

Data & Analytics

Mendeley as a Source of Readership by Students and Postdocs? Evaluating Article Usage by Academic Status