Embed Size (px)
Citation preview
Author's gender a�ects rating of academic article:
Author's gender a�ects rating of academic article:evidence from an incentivized, deception-free laboratory experiment
(published in European Economic Review)
Magdalena SmykMichaª Krawczyk
Findings from the GENDEQU ProjectUniversity of Warsaw
June 22, 2016
Author's gender a�ects rating of academic article:
Table of contents
1 Introduction: an idea
2 Our experiment
3 Results
4 Conclusions
Author's gender a�ects rating of academic article:
Introduction: an idea
Introduction: an idea
Author's gender a�ects rating of academic article:
Introduction: an idea
Gender equality at the Univeristy
Gender-stereotypes - do they exist in academia?
Does gender of the scientist a�ect evaluation of his/herwork?
Author's gender a�ects rating of academic article:
Introduction: an idea
Inspiration
How to check whether individuals take into account gender of a scientistwhen they evaluate her/his work?
in the real life: confounding e�ects (like di�erences in quality etc.)
ideally: comparing two researchers who di�er only by gender
SOLUTION: create such circumstances in an EXPERIMENT
Author's gender a�ects rating of academic article:
Introduction: an idea
Inspiration
How to check whether individuals take into account gender of a scientistwhen they evaluate her/his work?
in the real life: confounding e�ects (like di�erences in quality etc.)
ideally: comparing two researchers who di�er only by gender
SOLUTION: create such circumstances in an EXPERIMENT
Author's gender a�ects rating of academic article:
Introduction: an idea
Inspiration
How to check whether individuals take into account gender of a scientistwhen they evaluate her/his work?
in the real life: confounding e�ects (like di�erences in quality etc.)
ideally: comparing two researchers who di�er only by gender
SOLUTION: create such circumstances in an EXPERIMENT
Author's gender a�ects rating of academic article:
Introduction: an idea
Inspiration - other experiments
Borsuk et al. (2009): reviewing article with four di�erent authordesignations (female, male, initials, and no name at all)
result: no gender bias was observed
Knobloch-Westerwick et al. (2013): reviewing conference abstractsostensibly signed with female or male names
result: `male-authored' texts were rated higher on average, especiallywhen the topic was male-typed
Author's gender a�ects rating of academic article:
Introduction: an idea
Inspiration - other experiments
Borsuk et al. (2009): reviewing article with four di�erent authordesignations (female, male, initials, and no name at all)
result: no gender bias was observed
Knobloch-Westerwick et al. (2013): reviewing conference abstractsostensibly signed with female or male names
result: `male-authored' texts were rated higher on average, especiallywhen the topic was male-typed
Author's gender a�ects rating of academic article:
Introduction: an idea
Inspiration - other experiments
Borsuk et al. (2009): reviewing article with four di�erent authordesignations (female, male, initials, and no name at all)
result: no gender bias was observed
Knobloch-Westerwick et al. (2013): reviewing conference abstractsostensibly signed with female or male names
result: `male-authored' texts were rated higher on average, especiallywhen the topic was male-typed
Author's gender a�ects rating of academic article:
Introduction: an idea
Inspiration - other experiments
Borsuk et al. (2009): reviewing article with four di�erent authordesignations (female, male, initials, and no name at all)
result: no gender bias was observed
Knobloch-Westerwick et al. (2013): reviewing conference abstractsostensibly signed with female or male names
result: `male-authored' texts were rated higher on average, especiallywhen the topic was male-typed
Author's gender a�ects rating of academic article:
Our experiment
Our experiment
Author's gender a�ects rating of academic article:
Our experiment
Treatments
Articles
5 pairs written by mixed-gender couples
in each pair - one paper published in top economic journal (e.g.TheQuarterly Journal of Economics) and one working paper
no names
two by two design: �a female economist�, �a male economist�, �ayoung female economist�, �a young male economist�
Author's gender a�ects rating of academic article:
Our experiment
What was the experiment about?
Task:
Each subject had to read one article from the pool and answer on ascale 0-7 how she evaluate:
author's competencelanguage qualitymethodologyliterature reviewscienti�c signi�cancescienti�c qualityintelligibility
Author's gender a�ects rating of academic article:
Our experiment
What was the experiment about?
Task:
At the end, we asked if they believe the paper was published in oneof the top economic journals or not at all.
Reward: 80 zlotys (about 20EUR \25USD)+ 10 zlotys - if the guess on publication was correct.
Author's gender a�ects rating of academic article:
Our experiment
What's di�erent/new?
entire paper and ample time to evaluate a scientist (instead of justan abstract or short paper)
conditions preventing consultation with external sources
deception free experiment
additional dimension: young
novel way of introducing a treatment
several additional questions
Author's gender a�ects rating of academic article:
Our experiment
What's di�erent/new?
entire paper and ample time to evaluate a scientist (instead of justan abstract or short paper)
conditions preventing consultation with external sources
deception free experiment
additional dimension: young
novel way of introducing a treatment
several additional questions
Author's gender a�ects rating of academic article:
Our experiment
What's di�erent/new?
entire paper and ample time to evaluate a scientist (instead of justan abstract or short paper)
conditions preventing consultation with external sources
deception free experiment
additional dimension: young
novel way of introducing a treatment
several additional questions
Author's gender a�ects rating of academic article:
Our experiment
What's di�erent/new?
entire paper and ample time to evaluate a scientist (instead of justan abstract or short paper)
conditions preventing consultation with external sources
deception free experiment
additional dimension: young
novel way of introducing a treatment
several additional questions
Author's gender a�ects rating of academic article:
Our experiment
What's di�erent/new?
entire paper and ample time to evaluate a scientist (instead of justan abstract or short paper)
conditions preventing consultation with external sources
deception free experiment
additional dimension: young
novel way of introducing a treatment
several additional questions
Author's gender a�ects rating of academic article:
Our experiment
What's di�erent/new?
entire paper and ample time to evaluate a scientist (instead of justan abstract or short paper)
conditions preventing consultation with external sources
deception free experiment
additional dimension: young
novel way of introducing a treatment
several additional questions
Author's gender a�ects rating of academic article:
Our experiment
Design
10 sessions (+ 2 additional blind treatment sessions)
190 subjects ( + 34 subjects in blind treatment sessions)
Subjects
Subjects/ Young Female Young Male BlindTreatment female regardless male regardless
of age of age
Males 19 18 23 13 18Females 26 31 25 35 16
Economists 20 28 31 28 13Other majors 14 11 9 12 18Non-students 6 5 3 3 3
Mean age 22.7 22.6 22.4 22.7 24.8
Author's gender a�ects rating of academic article:
Our experiment
Design
10 sessions (+ 2 additional blind treatment sessions)
190 subjects ( + 34 subjects in blind treatment sessions)
Subjects
Subjects/ Young Female Young Male BlindTreatment female regardless male regardless
of age of age
Males 19 18 23 13 18Females 26 31 25 35 16
Economists 20 28 31 28 13Other majors 14 11 9 12 18Non-students 6 5 3 3 3
Mean age 22.7 22.6 22.4 22.7 24.8
Author's gender a�ects rating of academic article:
Our experiment
Hypotheses
Hypothesis 1
Female authors will receive lower ratings than authors of unspeci�edgender and, especially, males, and their papers will less often be judgedas published.
Hypothesis 2
Likewise, young female economists will be judged as inferior with regardto female economists of unspeci�ed age.
Intuition
The gender e�ect may have been greater for the incentivized and indirectquestion (whether the paper was published)
Author's gender a�ects rating of academic article:
Results
Results
Author's gender a�ects rating of academic article:
Results
Fewer subjects believed that female-authored papers hadbeen published.
However, authors' young age was insigni�cant. And there was nodi�erences between male and female subjects.
Author's gender a�ects rating of academic article:
Results
Di�erences in ratings were mostly insigni�cant
Mean Mean Mean Mean Meanyoung females young males blindfemales regardless males regardless
of age of age
Judged published 0.53 0.41 0.60 0.60 0.560.46 0.60
Competence 5.22 4.9 5.25 5.06 0.324.99 5.14
Language 5.02 4.92 5.08 4.8 54.92 4.96
Methodology 5.18 4.47 5.15 4.56 4.854.76 4.82
Literature 5.04 4.35 4.81 4.63 4.594.66 4.77
Signi�cance 4.41 4.04 5.21 3.96 4.384.18 4.06
Overall 4.73 4.47 4.75 4.63 4.534.56 4.67
Author's gender a�ects rating of academic article:
Results
Was the paper judged published?
probit model ( 1 = �YES�) (1) (2) (3) (4)
Female author -0.34* -0.50* -0.54** -0.56**(0.09) (0.27) (0.25) (0.24)
Young author 0.16 0 -0.07 -0.07(0.23) (0.23) (0.22) (0.22)
Young female author 0.32 0.36 0.35(0.48) (0.47) (0.47)
Female subject -0.41* -0.38(0.23) (0.30)
Student 0.26 0.27(0.30) (0.30)
Economics student -0.17 -0.17(0.17) (0.18)
Female author*male subject 0.06(0.32)
Paper e�ects included YES YES YES YESObservations 190 190 190 190
Author's gender a�ects rating of academic article:
Results
Controversy in the interpretation of the results
�Interpretation of results is very important to draw proper conclusions:are female authors rated less likely to have published because they are(wrongly) judged less competent than male authors or because subjectsbelieve there exists a gender bias in publication (due to genderdiscrimination)?�
Lack of the di�erences in ratings, BUT high correlation with �judgedpublished�.
Reviewer and some of the commenters - this explanation is too weak.
Additional sessions:
After �rst predictions, we told subjects that the paper was eithermale or female-authored
Do you want to change your answer?
If they choose to change - we asked them about the reason
Author's gender a�ects rating of academic article:
Results
Controversy in the interpretation of the results
�Interpretation of results is very important to draw proper conclusions:are female authors rated less likely to have published because they are(wrongly) judged less competent than male authors or because subjectsbelieve there exists a gender bias in publication (due to genderdiscrimination)?�
Lack of the di�erences in ratings, BUT high correlation with �judgedpublished�.
Reviewer and some of the commenters - this explanation is too weak.
Additional sessions:
After �rst predictions, we told subjects that the paper was eithermale or female-authored
Do you want to change your answer?
If they choose to change - we asked them about the reason
Author's gender a�ects rating of academic article:
Results
Controversy in the interpretation of the results
�Interpretation of results is very important to draw proper conclusions:are female authors rated less likely to have published because they are(wrongly) judged less competent than male authors or because subjectsbelieve there exists a gender bias in publication (due to genderdiscrimination)?�
Lack of the di�erences in ratings, BUT high correlation with �judgedpublished�.
Reviewer and some of the commenters - this explanation is too weak.
Additional sessions:
After �rst predictions, we told subjects that the paper was eithermale or female-authored
Do you want to change your answer?
If they choose to change - we asked them about the reason
Author's gender a�ects rating of academic article:
Results
Controversy in the interpretation of the results
Predictions:
If subjects were aware of gender bias in publication, they wouldsometimes change decision from published to unpublished for femaleauthors...
and from unpublished to published for males.
Due to the fact the there is a gender bias in publication.
No one decided to change their answer.
Rather unconscious beliefs than knowledge on gender discrimination.
Author's gender a�ects rating of academic article:
Results
Controversy in the interpretation of the results
Predictions:
If subjects were aware of gender bias in publication, they wouldsometimes change decision from published to unpublished for femaleauthors...
and from unpublished to published for males.
Due to the fact the there is a gender bias in publication.
No one decided to change their answer.
Rather unconscious beliefs than knowledge on gender discrimination.
Author's gender a�ects rating of academic article:
Conclusions
Conclusions
Author's gender a�ects rating of academic article:
Conclusions
Conclusions
The gender of the author, but not his or her age, a�ect evaluation ofa paper (at least in the �eld of economics).
Why those results are important?
The chances that our subjects' views of the role of women inacademia will change are minor.Our subjects were approaching the age at which important earlycareer decisions are made and when inexperienced researchers mayeasily be discouraged.Young women fail to see successful female economists that couldserve as role models.
Author's gender a�ects rating of academic article:
Conclusions
Conclusions
The gender of the author, but not his or her age, a�ect evaluation ofa paper (at least in the �eld of economics).
Why those results are important?
The chances that our subjects' views of the role of women inacademia will change are minor.Our subjects were approaching the age at which important earlycareer decisions are made and when inexperienced researchers mayeasily be discouraged.Young women fail to see successful female economists that couldserve as role models.
Author's gender a�ects rating of academic article:
Conclusions
Thank you!
