You have already added 0 works in your ORCID record related to the merged Research product.
You have already added 0 works in your ORCID record related to the merged Research product.
Evaluating intuitiveness of vertical-aware click models
Evaluating intuitiveness of vertical-aware click models
Modeling user behavior on a search engine result page is important for understanding the users and supporting simulation experiments. As result pages become more complex, click models evolve as well in order to capture additional aspects of user behavior in response to new forms of result presentation.We propose a method for evaluating the intuitiveness of vertical-aware click models, namely the ability of a click model to capture key aspects of aggregated result pages, such as vertical selection, item selection, result presentation and vertical diversity. This method allows us to isolate model components and therefore gives a multi-faceted view on a model's performance. We argue that our method can be used in conjunction with traditional click model evaluation metrics such as log-likelihood or perplexity. In order to demonstrate the power of our method in situations where result pages can contain more than one type of vertical(e.g., Image and News) we extend the previously studied Federated Click Model such that it models user clicks on such pages. Our evaluation method yields non-trivial yet interpretable conclusions about the intuitiveness of click models, highlighting their strengths and weaknesses.
- University of Amsterdam Netherlands
- University of Edinburgh United Kingdom
- University of Amsterdam (UvA) Netherlands
Microsoft Academic Graph classification: Information retrieval Computer science media_common.quotation_subject Federated search Click path Image (mathematics) Search engine Presentation Selection (linguistics) Key (cryptography) media_common
Microsoft Academic Graph classification: Information retrieval Computer science media_common.quotation_subject Federated search Click path Image (mathematics) Search engine Presentation Selection (linguistics) Key (cryptography) media_common
1312
[1] O. Chapelle and Y. Zhang. A dynamic Bayesian network click model for web search ranking. In WWW, 2009.
[2] O. Chapelle, T. Joachims, F. Radlinski, and Y. Yue. Large-scale validation and analysis of interleaved search evaluation. TOIS, 2012.
[3] D. Chen, W. Chen, and H. Wang. Beyond ten blue links: Enabling user click modeling in federated web search. In WSDM, 2012.
[4] A. Chuklin, A. Schuth, K. Hofmann, P. Serdyukov, and M. de Rijke. Evaluating aggregated search using interleaving. In CIKM, 2013. [OpenAIRE]
[5] A. Chuklin, P. Serdyukov, and M. de Rijke. Using intent information to model user behavior in diversified search. In ECIR, 2013. [OpenAIRE]
[6] G. Dupret and B. Piwowarski. A user browsing model to predict search engine click data from past observations. In SIGIR, 2008. [OpenAIRE]
[7] T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In SIGIR, 2005. [OpenAIRE]
[9] T. Sakai. Evaluation with informational and navigational intents. In WWW, 2012.
[10] A. Schuth, K. Hofmann, S. Whiteson, and M. de Rijke. Lerot: an online learning to rank framework. In Living Labs for Information Retrieval Evaluation workshop at CIKM, 2013. [OpenAIRE]
[11] S. Sushmita, H. Joho, M. Lalmas, and R. Villa. Factors affecting click-through behavior in aggregated search interfaces. In CIKM, 2010. [OpenAIRE]
- 1
- 2
citations This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).6 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Average influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Average citations This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).6 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Average influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Average
Citations provided by BIP!Popularity provided by BIP!- Netherlands Organisation for Scientific Research (NWO) (NWO)
- 612.001.116
- Netherlands Organisation for Scientific Research (NWO) (NWO)
- 612.066.930
- Netherlands Organisation for Scientific Research (NWO) (NWO)
- 727.011.005
- University of Amsterdam Netherlands
- University of Edinburgh United Kingdom
- University of Amsterdam (UvA) Netherlands
Modeling user behavior on a search engine result page is important for understanding the users and supporting simulation experiments. As result pages become more complex, click models evolve as well in order to capture additional aspects of user behavior in response to new forms of result presentation.We propose a method for evaluating the intuitiveness of vertical-aware click models, namely the ability of a click model to capture key aspects of aggregated result pages, such as vertical selection, item selection, result presentation and vertical diversity. This method allows us to isolate model components and therefore gives a multi-faceted view on a model's performance. We argue that our method can be used in conjunction with traditional click model evaluation metrics such as log-likelihood or perplexity. In order to demonstrate the power of our method in situations where result pages can contain more than one type of vertical(e.g., Image and News) we extend the previously studied Federated Click Model such that it models user clicks on such pages. Our evaluation method yields non-trivial yet interpretable conclusions about the intuitiveness of click models, highlighting their strengths and weaknesses.