How far can you rely on a concept test: the generalizability of testing over occasions
Document Type
Journal article
Source Publication
International Journal of Market Research
Publication Date
2010
Volume
52
Issue
3
First Page
363
Last Page
382
Abstract
In practice, product managers have to assume consumer evaluations of concepts generalise from the time (and research environment) of concept testing to the time (and market environment) of market introduction. However, little is known about the temporal stability or generalisability of the results of concept testing over occasions. Rarely have concept-testing studies incorporated testing of the same concepts on the same respondents on more than one occasion. This research investigates the importance of occasions as a source of error variance in estimates of the generalisability of concept test scores for both minor and major innovations within the context of Generalisability theory. The study collected concept evaluations of ten innovations from members of an online panel on three occasions, approximately a month apart. The results show that the three-way interaction among subjects, concepts and occasions is a substantial contributor to variation in concept testing of both major and minor innovations, with the contribution for major innovations even more substantial than for minor innovations. Moreover, failure to recognize occasions as an explicit source of variance in the generalisability analyses will lead managers to overestimate the generalisability of their decision studies. However, the impact of neglecting occasions varies by purpose of measurement and associated object of measurement. This research provides insight about how well concept testing can generalise over occasions. Concept test evaluations provided on an initial exposure are more favourable than will be received on any later occasions, and apparent differences in consumer evaluations of a particular concept in an initial test do not provide a generalisable basis for identifying which consumers will respond most favourably to it on a later occasion. For concept testing to be used for targeting or segmentation, more occasions will need to be sampled.
DOI
10.2501/S1470785309201314
Print ISSN
14707853
Publisher Statement
© 2010 The Market Research Society
Access to external full text or publisher's version may require subscription.
Full-text Version
Publisher’s Version
Language
English
Recommended Citation
Peng, L., & Finn, A. (2010). How far can you rely on a concept test?. International Journal Of Market Research, 52(3), 363-382. DOI: http://dx.doi.org/10.2501/S1470785309201314