E-Survey Workshop : Guidelines I and Practice 曾淑芬 元智大學資訊社會學研究所

E-Survey WorkshopE-Survey Workshop : :

Guidelines I and PracticeGuidelines I and Practice

曾淑芬元智大學資訊社會學研究所

What is Electronic Survey?Email SurveyWeb-based Survey

The rapid development of surveys on the WWW is leading some to argue that soon internet surveys will replace traditional methods of survey data collection. Major advantages have been mentioned: cost savings, speed, limited geographical constraint, ease of contacting respondents, more efficient data processing, ability for customization. Others are urging caution or even voicing skepticism about the future role Web surveys will play.

Why E-Survey?

Promises and Challenges

Promises Strong reach / Penetration Fast response speed Low cost Response flexibility Control of anonymity Minimized data-entered error Minimized interviewer bias

Promises and Challenges

Challenges Generalizability and Response Rate:Generalizability and Response Rate:

response rate to internet based surveys have been declining over the past 10 years

Accessibility:Accessibility:no national or global online directory of email addresses exist

Design Concerns:Design Concerns:use design survey response features judiciously to maximize data quality and minimize error

Sources of Errors in Web-based Survey

Coverage ErrorThe result of all units in a

defined population not having a known nonzero probability of being included in the sample drawn to represent the population. A mismatch between the target population and the frame population.

Sources of Errors in Web-based SurveyCoverage Error

Universal coverage of the Web remains quite limited. Coverage error represents the biggest threat to the representativeness of sample surveys conducted via the internet.

Some populations do not exhibit large coverage problems: employees of certain organizations, members of professional organizations, certain types of businesses, students at many universities and college, and groups with high levels of education

Sampling errorThe result of surveying a sample of the population rather than the entire population. Not all members of the frame population are measured.

Sources of Errors in Web-based Survey

Sampling error

Sources of Errors in Web-based Survey

While coverage error refers to people missing from the frame, sampling error arises during the process of selecting a sample from the frame population, necessitating a means of identifying people on the frame.

A misguided assumption behind many Web surveys that large samples necessarily mean more valid responses.

Sources of Errors in Web-based Survey

Measurement error The result of inaccurate responses that stem from poor question wording, poor interviewing survey mode effects and/or some aspect of the respondent’s behavior.

Sources of Errors in Web-based Survey

Measurement error

Measurement error is the deviation of the answers of respondents from their true values on the measure.

The appearance of a survey can vary from respondent to respondent because of different browser settings, user preferences, variations in hardware, and so on.

There is much work to be done to determine optimal designs for different groups of respondents and types of surveys.

Sources of Errors in Web-based Survey

Non-response error The result of non-response from people in the sample, who, if they had responded, would have provided different answers to the survey questions than those who did respond to the survey.

Sources of Errors in Web-based Survey

Non-response error

Non-response error arises through the fact that not all people included in the sample are willing or able to complete the survey.

Non-response error is a function of both the rates of non-response and of the differences between respondents and non-respondents on the variables of interest.

There is at present little experimental literature on what works and what does not, in terms of increasing response rates to web surveys.

Types of Web survey (Couper, 2001)

Probability-based Web-based surveysIntercept surveysList-based samples of high coverage populationsMixed-mode designs with choice of completion methodPre-recruited panels of internet usersProbability samples of full population

Non-Probability-based Web-based surveyPolls as entertainmentUnrestricted self-selected surveysVolunteer opt-in panels

Seven Response Types for Web-based Surveys (Bosnjak and

Tuten, 2001)

1. Unit non-responders2. Complete responders3. Answering drop-outs4. Lurkers5. Lurking drop-outs6. Item non-responders7. Item non-responding dropouts

Source: Bosnjak and Tuten,2000 (http://ascusc.org/jcmc/vol6/issue3/boznjak.html)

Seven Response Types for Web-based Surveys (Bosnjak and

Tuten, 2001)

Design of Web-bases Surveys (Dillman and Bowker, 2001)

1. Introduce the web questionnaire with a welcome screen (Non-response error)

2. Provide a PIN number (Sampling, Coverage)

3. First question should be interesting to most respondents, easily answered, and fully visible on the first screen of the questionnaire. (Non-response)

4. Present each question in a conventional format similar to that normally used on paper self-administered questionnaires. (Measurement, Non-response)

Design of Web-bases Surveys (Dillman and Bowker, 2001)

(Cont.)5. Restrain the use of color so that

figure/ground consistency and read-ability are maintained, navigational flow is unimpeded, and measurement properties of questions are maintained. (Measurement)

6. Avoid differences in the visual appearance of questions that result from different screen configurations, operating systems, browsers, partial screen displays and wrap-around text. (Coverage, Measurement, Non-response)

7. Provide specific instructions on how to take each necessary computer action for responding to the questionnaire and other necessary instructions at the point where they are needed. (Non-response)

8. Use drop-down boxes sparingly, consider the mode implication, and identify each with a “click here” instruction. (Measurement)

9. Do not require respondents to provide an answer to each question before being allowed to answer any subsequent ones. (Non-response)

Design of Web-bases Surveys (Dillman and Bowker, 2001)

(Cont.)

10. Provide skip directions in a way that encourages marking of answers and being able to click to the next applicable question. (Measurement)

11. Construct web questionnaires so they scroll from question to question unless order effects are a major concern, and/or telephone and web survey results are being combined. (Coverage, Measurement, Non-response)

12. When the number of answer choices exceeds the number that can be displayed in a single column on one screen, consider double-banking with an appropriate grouping device to link them together. (Measurement)

Design of Web-bases Surveys (Dillman and Bowker, 2001)

(Cont.)

13. Use graphical symbols or words that convey a sense of where the respondent is in the completion process, but avoid ones that require significant increases in computer memory. (Coverage, Non-response)

14. Exercise restraint in the use of question structures that have known measurement problems on paper questionnaires, e.g. check-all-that-apply and open-ended questions. (Measurement, Non-response)

Design of Web-bases Surveys

(Dillman and Bowker, 2001) (Cont.)

Other Web Survey Design Suggestion: Zanutto (2001)

1. Use a cover letter with the questionnaire2. Make the survey simple, and have it take

no longer than 20 minutes3. Give an estimated time that it will take to

complete the survey4. Be sure the first question is interesting,

easy to answer, and related to the topic of the survey

5. Be concerned about privacy issues for the respondents and the data that is collected.

6. Allow an alternate mode of completion if people are concerned about privacy, i.e. print and mail in the survey.

E-Survey WorkshopE-Survey Workshop : :

Part III Part III 實作教學實作教學

元智大學資訊社會學研究所黃心怡

開始你的 Web-survey

分析結果確認研究主題

設計問卷

問卷程式設計 問卷前測 發送 E-mail 邀請

資料收集

架設你的網路問卷平台自行設計 套裝軟體

優點 可客製化所需的問卷形式

容易使用，不需太多前置作業許多套裝軟體的功能也包括了資料整理與分析

缺點 自行設計所花的時間較多需有簡單程式基礎

所需的金錢花費較高

執行 利用 HTML 或是 ASP 等程式語言，設計一問卷網頁，放置於自己的網路伺服器上，儲存受訪者的回覆。

中文網站如 : http://my3q.com 或是 spss 等軟體都提供現成的程式，研究者只要輸入題目與簡單設定即可上線使用。

The Advantage of Web-Survey

Cost savingEase of contacting

respondentsMore efficient data

processingAbility for customization

How to designSTEP1:IntroductionSTEP1:Introduction

(1) 選擇問卷形式傳統問卷格式 題組形式每題一頁

(2) 設定版面大小較常使用的是 800*600 ，建議在網頁中加註讓受訪者瀏覽時較好觀看

STEP2:LayoutSTEP2:Layout

Frame( 表單 ) Radio( 按鈕類 )Checkbox( 勾選類 )Drop-Down( 下拉式 )Text( 文字欄 )Hyperlink( 超連結 )

STEP3:Webpage ComponentSTEP3:Webpage Component

文字檔簡單 HTML 語法 , 藉由網頁製作軟體如 : FrontP

age 、 Dreamwaver 等，受訪者在完成線上問卷後，可直接回傳結果至研究者的網路伺服器，以文字檔儲存

資料庫當調查規模龐大且題目眾多時，資料庫有著良好的儲存與管理的功能；相對的，與資料庫連結的網路問卷也需要較複雜的程式技巧與工具，其與法如 ASP 、 PHP 、 JAVA 。

STEP4:Save ResultSTEP4:Save Result

網路問卷降低了過去人工 coding 的時間與金錢

線上調查的電子檔容易讓統計軟體讀取 ( 通常 .txt, .csv 為最普遍的可讀取格式 )

STEP5:Analyze ResultSTEP5:Analyze Result

STEP BY STEP: STEP BY STEP: Form (Form ( 表單表單 ))

STEP BY STEP : FormSTEP BY STEP : Form

STEP BY STEP : Pixels(STEP BY STEP : Pixels( 瀏覽像素瀏覽像素 ))

STEP BY STEP : PixelsSTEP BY STEP : Pixels

STEP BY STEP : Table(STEP BY STEP : Table( 表格表格 ))

STEP BY STEP : TableSTEP BY STEP : Table

STEP BY STEP : TableSTEP BY STEP : Table

STEP BY STEP : Radio ButtonSTEP BY STEP : Radio Button

STEP BY STEP : Radio ButtonSTEP BY STEP : Radio Button

STEP BY STEP : Radio ButtonSTEP BY STEP : Radio Button

名稱 = Q1

是 = 1

否 = 0

不選擇預設值

STEP BY STEP : Radio ButtonSTEP BY STEP : Radio Button

STEP BY STEP : Check BoxSTEP BY STEP : Check Box

STEP BY STEP : Check BoxSTEP BY STEP : Check Box

名稱 = Q2_1

有勾選 = 1

不選擇預設值

STEP BY STEP : Check BoxSTEP BY STEP : Check Box

STEP BY STEP : DropdownSTEP BY STEP : Dropdown

STEP BY STEP : DropdownSTEP BY STEP : Dropdown

STEP BY STEP : DropdownSTEP BY STEP : Dropdown

名稱 = Q3

新增選項 = 台北市

勾選指定值 =01

起始狀態 = 未選取

STEP BY STEP : DropdownSTEP BY STEP : Dropdown

STEP BY STEP : DropdownSTEP BY STEP : Dropdown

驗證功能 :

受訪者送出問卷後，檢查其填寫狀態，避免漏答

STEP BY STEP : Text BoxSTEP BY STEP : Text Box

STEP BY STEP : Text BoxSTEP BY STEP : Text Box

名稱 = Q4

字元寬度 =50

密碼欄位 = 否

STEP BY STEP : Text BoxSTEP BY STEP : Text Box

驗證功能 :

文字格式 = 文字

STEP BY STEP : STEP BY STEP : 跳答跳答

STEP BY STEP : STEP BY STEP : 跳答跳答

STEP BY STEP : STEP BY STEP : 跳答跳答

STEP BY STEP : STEP BY STEP : 跳答跳答

STEP BY STEP : STEP BY STEP : 跳答跳答

STEP BY STEP : SubmitSTEP BY STEP : Submit

STEP BY STEP : SubmitSTEP BY STEP : Submit

STEP BY STEP : SavingSTEP BY STEP : Saving

STEP BY STEP : SavingSTEP BY STEP : Saving

檔案格式 :

文字資料庫使用 tab鍵來區隔

STEP BY STEP : SavingSTEP BY STEP : Saving

確認頁的 URL:

Thank_you.html

STEP BY STEP : SavingSTEP BY STEP : Saving

表單儲存 :

1. 檢查是否每一題皆為獨立題號

2. 可選擇日期格式與受訪者的其他資訊

實作時的提醒

1.選擇調查方法時仍需因時地而制宜，免得不符研究初衷

2.在應用資訊科技進行調查之時，最重要的是能夠充分運用電子化後的便利與快速。因此，千萬要讓你的網路調查設計的比傳統問卷調查更易於整理、分析

3.跳答與漏答的項目需事先規劃，並寫在程式中

4.盡量遵循著 Dillman的 web-survey設計準則，將有助於回覆率與可能的失誤

Reference

Bosnjak, M. M. and Tuten, T. L., 2001. “Classifying Response Behaviors in Web-Based Surveys” Journal of Comupter-Mediated Communication, Vol6, No.3, at http://www.ascusc.org/jcmc/vol6/issue3/boznjak.html.

Couper, M. P., 2000. “Web Surveys a Review of Issues and Approaches” Public Opinion Quarterly, Vol64, No.4, pp.464-481.

Dillman, D. A., 2000. Mail and Internet Surveys: The Tailored Design Methods. Second edition. New York: Wiley.

Dillman, D. A., 2002. “Navigating the Rapids of Change: Some Observations on Survey Methodology in the early 21st century” presidential address to the American Association for Public Opinion Research.

Dillman, D. A. and Bowker, D. K., 2001. “The Web Questionnaire Challenge to Survey Methodologists” at http://survey.sesrc.wsu.edu/dillman/zuma_paper_dillman_bowker.pdf.

Reference

Dillman, D. A., Tortora, R. D. and Bowker, D., 1998. “Principle for Constructing Web Surveys” Pullman, Washington. SESRC Technical Report 98-50, at http://survey.sesrc.wsu.edu/dillman/papers/websurveyppr.pdf.

Smith, T. W., 2001. “Are Representative Internet Surveys Possible?” proceedings of Statistics Canada Symposium.

Sheehan, K. B., 2002. “Online Research Methodology: Reflections and speculations” Journal of Interactive Advertising, Vol3, No.1.