A Study of Internet-based Survey Methodology
--Where can we go?
Ke Huixin, Huang Gang Beijing Broadcasting Institute Email: hxke@public.bta.net.cn Karlhg@263.net
Huixin Ke, SSI
1
2008-11-3
Part 1 Background of Internet survey’s quiet rising Part 2 Adoption and application of Internet survey in china Part 3 Present Obstacles and problems Part 4 Orientations and frameworks of future research
2
Huixin Ke, SSI
2008-11-3
Part 1 Background of Internet survey’s quiet rising
1 Impulse from the market 2 Obstacles of traditional survey 3 Advantages of Internet survey 4 External conditions and circumstance
Huixin Ke, SSI
3
2008-11-3
1 Impulse from the market
China’s entry of WTO Customers’ request to the timeliness, universality and low-cost of market information New current of marketing modes Impulse from E-business and change of commercial modes Trend of economic globalization …………………
Huixin Ke, SSI
4
2008-11-3
2 Obstacles of traditional survey
Data quality descends Cost of face-to-face interviews at home ascends Response rates of face-to-face interviews at home descends Quoted prices declines Profit decreases while operation increases Internet-based survey goes up in the world
Huixin Ke, SSI
5
2008-11-3
A study on MR in SARS period
Conducted by CMRA By telephone interviews From May 12 to June 12, 2003 Once a week, four times in total 95 companies responded 31 cities covered 6
2008-11-3
Huixin Ke, SSI
A study on MR in SARS period Introduction
Date
City amount 10
City name Wuhan,Wenzhou,Hefei,Shijiazhua ng,Taiyuan,Guangzhou,Nanning,H aikou,Jinan,Fuzhou
Company amount 22
May 12-14
May 19-20
10
Shanghai,Hangzhou,Nanjing,Chan gsha,Zhengzhou,Chongqing,Kunm ing,Shenyang,Ningbo,Shenzhen Chengdu,Ha’erbin,Dalian,Xi’an,Na nchang,Lanzhou,Qingdao,Tianjin, Yinchuan,Changchun Chengdu,Xi’an,Beijing,Shanghai,S henyang,Shenzhen,Chongqing,Gu angzhou,Wuhan
28
May 26-27
10
21
June 11-12
9
24
Huixin Ke, SSI
7
2008-11-3
A study on MR in SARS period Company scale
50% 40% 30% 20% 10% 0% Large Medium Small 21.1% 41.1% 37.9%
Huixin Ke, SSI
8
2008-11-3
A study on MR in SARS period
1) fieldwork conducted normally
120% 100% 80% 60% 40% 20% 0% May 14
Huixin Ke, SSI
Face-to-face at home CLT（Hall test) Focus groups Street faceto-face B2B research Telephone survey May 20 May 27 June 12
9
2008-11-3
A study on MR in SARS period
2) fieldwork in Beijing (2003, 5-22)
Internet survey Mail survey Telephone survey B2B research Street face-to-face Focus groups CLT（Hall test) Face-to-face at home
0% no business greatly difficult alomost as usual
Huixin Ke, SSI
20%
40%
60%
80%
100%
can't be conducted somewhat difficult not influenced
10
2008-11-3
Growth of Internet survey worldwide
800 600 400 200 0 US Europe (EU) Remaining worldwide Total
Huixin Ke, SSI
Source: BMRA
2000 258 55 16 329
2001 392 77 23 492
2002 578 88 33 699
2008-11-3
11
3 Advantages of Internet survey
Timeliness Low cost Little effect of interviewers longer and more questions than telephone interview More convenient for respondents Changeability of survey content Multiplicity and flexibility of questionnaire forms Less artificial interference and higher data quality Anonymity helps getting some certain information Possibility to reach difficult people Pertinence to Netizens
Huixin Ke, SSI
12
2008-11-3
4 External conditions and circumstances
Popularization of Internet Multiplization of Netizens
Age Gender Education Income ……….
Huixin Ke, SSI
13
2008-11-3
Popularization of Internet
8000 7000 6000 5000 4580 4000 3000 2000 1000 0 62 117.5 210 400 890 2250 1690 3370 2650
Unit: 10 thousand
6800 5910
97- 98-7 99-1 99-7 00-1 00-7 01-1 01-7 02-1 02-7 03-1 03-7 10
Huixin Ke, SSI
14
2008-11-3
Profile of Netizens--age
80% 70% 60% 50% 40% 30% 20% 10% 0%
00-1
Huixin Ke, SSI
under 18 18-30 31-50 51 or older
00-7
01-1
01-7
02-1
02-7
03-1
03-7
2008-11-3
15
Profile of Netizens--age
100% 80% 60% 40% 20% 0% 0-14
22.9% 7.0% 17.1% 0.8% 70.2% 82.1%
15-64 65 or older
0-17
18-60 60 or older
Populations (5th census, 2000-11)
Huixin Ke, SSI
Netizens (CNNIC, 2003-7)
16
2008-11-3
Profile of Netizens--gender
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 97- 98-7 99-1 99-7 00-1 00-7 01-1 01-7 02-1 02-7 03-1 03-7 10
Huixin Ke, SSI
male female
17
2008-11-3
Profile of Netizens--gender
70% 60% 50% 40% 30% 20% 10% 0% male female
51.6% 60.1%
polulations(5th census, 2000-11) Netizens(CNNIC, 2003-7)
48.4% 39.9%
Huixin Ke, SSI
18
2008-11-3
Profile of Netizens--education
90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 99-7 00-1 00-7 01-1 01-7 02-1 02-7 03-1 03-7
Huixin Ke, SSI
low medium medium high high
19
2008-11-3
Profile of Netizens--education
90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 13.2% 4.3%
polulations(5th census, 2000-11)
82.5%
low medium high
55.2%
30.9% 13.9%
Netizens(CNNIC, 2003-7)
Huixin Ke, SSI
20
2008-11-3
Profile of Netizens--income
40% 35% 30% 25% 20% 15% 10% 5% 0% 01-7
Huixin Ke, SSI
without income 500 or less 501-1000 1001-3000 3001-6000 6001 or more
02-1
02-7
03-1
03-7
Unit: RMB
2008-11-3
21
Part 2 Adoption and application of Internet survey in China
--- A Study on MR Industry
Methodology Main findings about Internet-based survey
Popularity Adoption history Frequency of adoption Providers characters Way of use Domain of application Survey methods
Huixin Ke, SSI
22
2008-11-3
A Study on MR Industry
Methodology
Self-administered survey by mail In-depth interviews
Huixin Ke, SSI
23
2008-11-3
A Study on MR Industry
Survey
Attached to 2003 Annual Survey of CMRA Conducted by CMRA from June to September Self-administered By mail and e-mail 41 companies responded Response rate: 18.6% 24
2008-11-3
Huixin Ke, SSI
A Study on MR Industry
Survey--Respondents profile
Missing others scientific research joint venture joint-stock private enterprise
2.4% 2.4% 4.9% 12.2% 17.1% 61.0%
0%
Huixin Ke, SSI
20%
40%
60%
80%
2008-11-3
25
A Study on MR Industry
In-depth interviews
Conducted in September 2003 Interviewees:
4 from MR companies (2 in china, 1 in Japan, 1 in Korea) 2 from counseling company 1 from website company 1 from scientific research institute
By face-to-face, telephone and e-mail 26
2008-11-3
Huixin Ke, SSI
A Study on MR Industry
Result 1: Popularity
Adoption of Internet-based survey Yes 22.0%
No 78.1%
Huixin Ke, SSI
27
2008-11-3
A Study on MR Industry
Result 2: Adoption history
Adoption history 10 9 8 7 6 5 4 3 2 1 0 1995
Huixin Ke, SSI
Companys
1998
2000 2001 Year
2002
2003
2008-11-3
28
A Study on MR Industry
Result 2: Adoption history
Adoption history 5 private enterprise 4 joint venture joint-stock
Companies
3 2 1 0
1995
Huixin Ke, SSI
1998
2000 2001 Year
2002
2003
2008-11-3
29
A Study on MR Industry
Result 3: Frequency of adoption
Frequency of adoption
3
companies
2 1 0
1
2
3 Projects
12
14
Huixin Ke, SSI
30
2008-11-3
A Study on MR Industry
Result 3: Frequency of adoption
Frequency of adoption 2
private enterprise joint venture joint-stock
Companies
1
0
1
2
3 Projects
12
14
2008-11-3
Huixin Ke, SSI
31
A Study on MR Industry
Result 4: Providers characters--types
Types of companies
scientific research
Type
joint-stock private enterprise joint venture
0
1
2
3
4
Number
Huixin Ke, SSI
32
2008-11-3
A Study on MR Industry
Result 4: Providers characters--types
scientific research private enterprise joint-stock joint venture 0%
Huixin Ke, SSI
0.0%
12.0%
28.6%
80.0%
10% 20% 30% 40% 50% 60% 70% 80%
33
2008-11-3
A Study on MR Industry
Result 4: Providers characters—services
Types of services
8 7 6 5 4 3 2 1 0
Number of companies
all-around
counseling mainly
implementing mainly
Services
Huixin Ke, SSI
34
2008-11-3
A Study on MR Industry
Result 4: Providers characters--scale
Scale of company 100 or more
Scale
51-100 11-50
10 or less 0
Huixin Ke, SSI
1 Number
2
3
2008-11-3
35
A Study on MR Industry
Result 5: Way of use
Traditional mainly Internet mainly Internet only 0%
Huixin Ke, SSI
12.82%
30.77%
56.41%
10%
20%
30%
40%
50%
60%
2008-11-3
36
A Study on MR Industry
Result 5: Way of use
Way of use
12 11 10 10 8 6 4 4 2 2 0 0 0 3 2 7
Number of companies
private enterprise joint venture joint-stock
Internet only
Internet mainly
Traditional mainly
Way
Huixin Ke, SSI
37
2008-11-3
A Study on MR Industry
Result 6: Domain of application
Developing new product Testing new advertisement Customers / employees satisfaction TV program tracing inspection Pop issues on Internet Others 38
Huixin Ke, SSI
2008-11-3
A Study on MR Industry
Result 7: Research methods
in-depth interviews online focus groups pop up e-mail survey web survey
11.1% 11.1% 33.3% 33.3% 77.8%
0%
Huixin Ke, SSI
20%
40%
60%
80%
2008-11-3
39
Part 3 Present Obstacles and problems
Obstacles of external environments Problems or limitations of technology and methodology
Huixin Ke, SSI
40
2008-11-3
Obstacles
Customers
Insufficient identify with Internet-based survey methodology
Netizens
Quality, concept, attitude and consciousness Scale and structure Short of order / market Restriction in capital, personnel and operation
MR companies
Huixin Ke, SSI
41
2008-11-3
Problems or limitations
Limitations of survey methods
Limited in traditional survey methods Limited in quantificational structural questionnaire survey Copying the traditional questionnaire measures only
Huixin Ke, SSI
42
2008-11-3
Problems or limitations
Lack of scientificity in questionnaire survey
Authenticity Representativeness Response rates
Lack of study on survey methodologies
Research design, questionnaire design, sample design, implementing and data processing……
Huixin Ke, SSI
43
2008-11-3
Part 4 Orientations and frameworks of future research
Orientations of research to various Internet survey methods Frameworks of research for Internetbased questionnaire survey Whole framework Concrete framework 44
2008-11-3
Huixin Ke, SSI
Orientations
Quicken the development and application of new methods beyond traditional methods Solve the problems rising from the transplant of traditional methods to the Internet research 45
Huixin Ke, SSI
2008-11-3
Frameworks—whole framework
Survey modes Sampling methods Survey instruments Incentive measures
………………
Timeliness Cost Authenticity Representativeness Nonresponse Users’ satisfaction
Nonresponse includes unit non-response, item non-response and partial non-response.
Huixin Ke, SSI
46
2008-11-3
Frameworks—survey modes
Contact modes Response modes Follow-up modes Timeliness Cost Authenticity Representativeness Nonresponse Users’ satisfaction
Contact, response, and follow-up can be conducted in (a) different mode(s) respectively: 1-Interviewer-administered: in-person, telephone 2-Self-administered: mail, Internet
Huixin Ke, SSI
47
2008-11-3
Frameworks—sampling
Sampling Category Optional Method Convenience Uncontrolled instrument distribution Systematic sampling of Web site visitors (pop up)
Volunteer panel Probability Sample from a closed population list Sample from a general population Pre-recruited panel
Huixin Ke, SSI
48
2008-11-3
Frameworks—survey design features
Content:
instructions topics Style of expression helping and explaining information
Form: display of content
color advanced technological features space of a whole page length of questionnaire whole style
Timeliness Cost Authenticity Nonresponse Users’ satisfaction
Technological control
Huixin Ke, SSI
49
2008-11-3
Frameworks—incentives
Types of incentives Value of incentives Time for incentives …… Timeliness Cost Authenticity Representativeness Nonresponse Users’ satisfaction and loyalty
Huixin Ke, SSI
50
2008-11-3
Summary
Internet-based survey in demand The adoption going up Be aware of the problems Well designed and specified research on methodologies required – our next research target
Huixin Ke, SSI
51
2008-11-3
Thank you !
52
Huixin Ke, SSI
2008-11-3