Mid-Term Report of my research project

Mid-term report of the projectDIPF Frankfurt, July 18

Supervisor: Dr. Ivan HabernalStudent: Anil Narassiguin

Identification of Argumentative Texts in User-Generated Content

on Educational Controversies

Persuasion The act of persuading or seeking to persuade... Persuade 1- To prevail on (a person) to do something, as by advising or urging. 2- To induce to believe by appealing to reason or understanding.

17/07/14 | DIPF Frankfurt | UKP Lab - Prof. Dr. Iryna Gurevych | Dr. Ivan Habernal | Anil Narassiguin 2

Persuasive or not ?

“Purposely raising your children to be outcasts is child abuse, I don't care what kind of cult you belong to. „

17/07/14 | DIPF Frankfurt | UKP Lab - Prof. Dr. Iryna Gurevych | Dr. Ivan Habernal | Anil Narassiguin 3

Debate about homeschooling

Persuasive

Persuasive or not ?

17/07/14 | DIPF Frankfurt | UKP Lab - Prof. Dr. Iryna Gurevych | Dr. Ivan Habernal | 4

I have no intention of waiting for scientists to reach agreement on this issue before I decide how I will educate my children. Leave us alone.

Debate about single sex education

Non Persuasive

Persuasive or not ?

You are a bad parent if you do not get your child the best education that is feasible for him/her under your circumstances. The notion that your child(ren)

should be used to indulge some social agenda is pathetic.

Debate about public and private schools

Not obvious...

Summary

Presentation of the task

Our approach

Problems encountered and future work

PRESENTATION OF THE TASK

Argumentation mining

ArgumentationMining

NLP Information Retrieval

MachineLearning

Philosophy

Psychology

Our project

Classification of user-generated text documents as persuasive (P1) and non-persuasive (P2).

Supervised Learning Three annotators classified manually a corpus of 990 texts as P1 or P2.

Cohen's kappa: 0.50 – 0.60

6 domains or topics such as: redshirting, prayer in schools, homeschooling, single sex education, mainstreaming and public private schools.

Our project

Cross Validation (CV) - Full CV : Cross Validation performed on all the data - In Domain CV : Cross Validation performed on texts of only one domain - Cross Domain Validation: One domain is tested on the 5 other ones.

Our data

Classredshirting prayer

in schools

homeschooling

single sex education

mainstreaming public private schools

P1 38 77 86 26 10 287

P2 30 66 138 24 19 189

Total 68 143 224 50 29 476

OUR APPROACH

Feature Engineering

Baseline features Based on the tokens, we extract the 10,000 most common 1, 2 and 3-grams.

Part of speech (POS) features Total number and Ratio of all the POS defined in DKPro.

Syntactic features - Statistics (mean, max) about the number of clauses per sentence.- Average and maximal depth of the dependency trees- Presence of a dependency rule (still need to be done)(Ref: Identifying Argumentative Discourse Structures in Persuasive Essays, Christian Stab)

Lexical features- Statistics about sentences and tokens- Length of a post, number of token per sentence and number of tokens with more than 6 letters (Ref: "Stance Classification of Ideological Debates: Data, Models, Features, and Constraints" Kazi Saidul Hasan and Vincent Ng)

- Ratio of punctuation marks and presence of multiple punctuation

Sentiment featuresIntegration of GPL Stanford Deep Learning for Sentiment Analysis in DKPro

Feature Engineering

Sentiment featuresSentiment Analysis tool output: for each sentence, 5 scores are computed. We are creating 20 features for those scores.

Classifier used

We're currently using the common classifier SVM

At the moment, we don't consider to compare the results for several classifiers

Results

Full CVBaseline 69.0 / 68.9

All the features

67.9 / 67.8

Baseline+Sentiment Analysis

68.0 / 68.0

Class redshirting prayer in schools

homeschooling single sex education

mainstreamingpublic private schools

Baseline 51.5 / 51.4

74.1 / 73.9

74.1 / 70.5

65.5 / 39.6

70.0 / 68.5

All the features

54.4 / 54.1

73.4 / 73.3

75.0 / 71.5

68.0 / 67.5

65.5 / 39.6

72.1 / 70.6

54.4 / 54.1

73.4 / 73.3

75.0 / 71.5

68.0 / 67.5

65.5 / 39.6

72.1 / 70.6

In Domain CV

– Blue: Accuracy– Orange: F-measure

Results

Cross Domain

Class redshirting prayer in schools

homeschooling single sex education

mainstreaming public private schools

Baseline 54.4 / 54.1

53.9 / 52.6

65.6 / 61.0 64.0 / 62.5

65.5 / 61.8

66.1 / 65.9

All the features

55.9 / 55.5

53.9 / 53.0

65.1 / 61.2 62.0 / 60.1

65.5 / 61.8

65.3 / 64.9

54.4 / 54.1

73.4 / 73.3

74.5 / 70.9 66.0 / 65.3

65.5 / 39.6

71.0 / 69.6

PROBLEMS ENCOUNTERED AND FUTURE WORK

17/07/14 | DIPF Frankfurt | UKP Lab - Prof. Dr. Iryna Gurevych | Dr. Ivan Habernal |

Problems encountered

Knowledge in NLP

Getting hands on DKPro

Computation times (might solve it soon...)

17/07/14 | DIPF Frankfurt | UKP Lab - Prof. Dr. Iryna Gurevych | Dr. Ivan Habernal |

Future work

Error Analysis

Feature selection

Hyper parameter optimization for SVM

Bootstrap the model with new data from debate portals

Questions ?

Mid-Term Report of my research project

Data & Analytics

สรุป econ mid-term

Team Mid-term Microeconomics Project

Sejarah Paper 3 Mid Term

Tahun 1 Mid Term Matematik 2016

N I TEISHO MID TERM

DST-T2 mid term 2.doc

Kerja Rumah Terancang Mid Term 2016

Mid-terM Evaluation 11.30.2012 in moratuwa

lk mid term f4 2014

Mid Term English Form4

Mid term evaluace realizace strategií CLLD Příloha č ...ní z… · 1 Mid pro Datum: -term evaluace realizace strategií CLLD Příloha č. : Šablona Evaluační zprávy mid-term

MSI Mid-Term Performance Report

overall details before the mid-term test

Progress of Mid-Term Management Plan SOMPO Holdings, Inc./media/hd/en/files/doc/... · 5/28/2020 · Progress of mid-term management plan 4 Businesses progress of mid-term management

Mid Term-paper 2

Mid term learning portfolio

004 ecoeduca mid term

Semester 2 Mid Term Review

Mid Term Report

Mid Term Organik & Tak Organik