Time: Noon
Location: TBA
Speaker: Jaime Arguello, Language Technologies Institute, School of Computer Science, CMU
Title: Vertical Selection in the Presence of Unlabeled Verticals.
Abstract:
Vertical aggregation is the task of incorporating results from specialized search engines or verticals (e.g., images, video, news) into Web search results. Vertical selection is the subtask of deciding, given a query, which verticals, if any, are relevant. State of the art approaches use machine learned models to predict which verticals are relevant to a query. When trained using a large set of labeled data, a machine learned vertical selection model outperforms baselines which require no training data. Unfortunately, whenever a new vertical is introduced, a costly new set of editorial data must be gathered. In this paper, we propose methods for reusing training data from a set of existing (source) verticals to learn a predictive model for a new (target) vertical. We study methods for learning robust, portable, and adaptive cross-vertical models. Experiments show the need to focus on different types of features when maximizing portability (the ability for a single model to make accurate predictions across multiple verticals) than when maximizing adaptability (the ability for a single model to make accurate predictions for a specific vertical). We demonstrate the efficacy of our methods through extensive experimentation for 11 verticals.
This is joint work with Fernando Diaz and Jean-Francois Paiement from Yahoo! Labs and will be presented at SIGIR 2010.
Tuesday, July 13, 2010
Tuesday, July 6, 2010
Grace Yang -- Friday July 9th
Time: Noon
Location: GHC 4405
Speaker: Grace Hui Yang, Language Technologies Institute, School of Computer Science, CMU
Title: Collecting High Quality Overlapping Labels at Low Cost.
Abstract:
This paper studies quality of human labels used to train search engines’ rankers. Our specific focus is performance improvements obtained by using overlapping relevance labels, which is by collecting multiple human judgments for each training sample. The paper explores whether, when, and for which samples one should obtain overlapping training labels, as well as how many labels per sample are needed. The proposed selective labeling scheme collects additional labels only for a subset of training samples, specifically for those that are labeled relevant by a judge. Our experiments show that this labeling scheme improves the NDCG of two Web search rankers on several real-world test sets, with a low labeling overhead of around 1.4 labels per sample. This labeling scheme also outperforms several methods of using overlapping labels, such as simple k-overlap, majority vote, the highest labels, etc. Finally, the paper presents a study of how many overlapping labels are needed to get the best improvement in retrieval accuracy.
This paper is published in Proceedings of the 33th Annual ACM SIGIR Conference (SIGIR2010), Geneva, Switzerland, July 19-23, 2010.
Location: GHC 4405
Speaker: Grace Hui Yang, Language Technologies Institute, School of Computer Science, CMU
Title: Collecting High Quality Overlapping Labels at Low Cost.
Abstract:
This paper studies quality of human labels used to train search engines’ rankers. Our specific focus is performance improvements obtained by using overlapping relevance labels, which is by collecting multiple human judgments for each training sample. The paper explores whether, when, and for which samples one should obtain overlapping training labels, as well as how many labels per sample are needed. The proposed selective labeling scheme collects additional labels only for a subset of training samples, specifically for those that are labeled relevant by a judge. Our experiments show that this labeling scheme improves the NDCG of two Web search rankers on several real-world test sets, with a low labeling overhead of around 1.4 labels per sample. This labeling scheme also outperforms several methods of using overlapping labels, such as simple k-overlap, majority vote, the highest labels, etc. Finally, the paper presents a study of how many overlapping labels are needed to get the best improvement in retrieval accuracy.
This paper is published in Proceedings of the 33th Annual ACM SIGIR Conference (SIGIR2010), Geneva, Switzerland, July 19-23, 2010.
Friday, May 28, 2010
Lee Jensen (Ancestry.com) -- Thursday, June 10
Speaker: Lee Jensen (Ancestry.com)
Date/Time: Thursday, June 10, 2010, noon
Place: GHC 6115
Title: Exploiting Sequential Relationships for Familial Classification
Abstract:
Information hidden in sequences of related data provides a significant exploitable source for information extraction. In this work we demonstrate techniques for increasing the signal of classifications from sequences of dependent data. This includes using algorithms, as well as generating features, that take advantage of the sequence oriented nature of the data. The efficacy of these techniques is evaluated with the classification of familial relationships within United States census data. The census data proves an interesting corpus for this work because of the highly sequential nature of the instances, and the explicit classification relationship between one instance and an instance previously found in the sequence.
Date/Time: Thursday, June 10, 2010, noon
Place: GHC 6115
Title: Exploiting Sequential Relationships for Familial Classification
Abstract:
Information hidden in sequences of related data provides a significant exploitable source for information extraction. In this work we demonstrate techniques for increasing the signal of classifications from sequences of dependent data. This includes using algorithms, as well as generating features, that take advantage of the sequence oriented nature of the data. The efficacy of these techniques is evaluated with the classification of familial relationships within United States census data. The census data proves an interesting corpus for this work because of the highly sequential nature of the instances, and the explicit classification relationship between one instance and an instance previously found in the sequence.
Thursday, April 8, 2010
Vasudeva Varma -- Friday April 16, 2010, noon
Speaker: Dr. Vasudeva Varma (Institute of Information Technology, Hyderabad, India)
Date/Time: Friday April 16, 2010, noon
Place: Newell-Simon Hall 3002
In this talk, I briefly explain the cross language information retrieval research for Indian languages. There is a national wide mission mode project on cross language information access sponsored by Ministry of Communications and Information technology, Government of India involving ten Universities. I will talk about the major research issues and a possible roadmap for the CLIR research in Indian setting. I will also share our experiences with multi-lingual summarization.
Vasudeva Varma is a faculty member at International Institute of Information Technology, Hyderabad Since 2002. His research interests include search (information retrieval), information extraction, information access, knowledge management, cloud computing and software engineering. He is heading Search and Information Extraction Lab and Software Engineering Research Lab at IIIT Hyderabad. He is also the chair of Post Graduate Programs since 2009. He published a book on Software Architecture (Pearson Education) and over sixty technical papers in journals and conferences. In 2004, he obtained young scientist award and grant from Department of Science and Technology, Government of India, for his proposal on personalized search engines. In 2007, he was given Research Faculty Award by AOL Labs.
Monday, July 13, 2009
Jaime Arguello -- Thursday July 16th, 2009, 11:00 AM
Speaker: Jaime Arguello (Language Technologies Institute, School of Computer Science, Carnegie Mellon University)
Time/Date: Thursday July 16th, 2009, 11:00 AM.
Place: Wean Hall 7220
Title: Sources of Evidence for Vertical Selection
Web search providers often include search services for domain-specific subcollections, called verticals, such as news, images, videos, job postings, company summaries, and artist profiles. We address the problem of vertical selection, predicting relevant verticals (if any) for queries issued to a search engine's main web search page. In contrast to prior collection selection tasks, vertical selection is associated with unique resources that can inform the classification decision. We focus on three sources of evidence: (1) the query string, from which features are derived independent of external resources, (2) logs of queries previously issued to the vertical directly by users, and (3) corpora representative of vertical content. These sources of evidence are integrated as features in a classification-based approach. We make use of and compare against prior work in federated search and retrieval effectiveness prediction. Our evaluation focuses on 18 different verticals, which differ in terms of semantics, media type, size, and level of query traffic. An in-depth error analysis reveals unique challenges across different verticals and provides insight into vertical selection for future work.
Based on work conducted at Yahoo! Labs Montreal to be presented at SIGIR 2009.
Time/Date: Thursday July 16th, 2009, 11:00 AM.
Place: Wean Hall 7220
Title: Sources of Evidence for Vertical Selection
Web search providers often include search services for domain-specific subcollections, called verticals, such as news, images, videos, job postings, company summaries, and artist profiles. We address the problem of vertical selection, predicting relevant verticals (if any) for queries issued to a search engine's main web search page. In contrast to prior collection selection tasks, vertical selection is associated with unique resources that can inform the classification decision. We focus on three sources of evidence: (1) the query string, from which features are derived independent of external resources, (2) logs of queries previously issued to the vertical directly by users, and (3) corpora representative of vertical content. These sources of evidence are integrated as features in a classification-based approach. We make use of and compare against prior work in federated search and retrieval effectiveness prediction. Our evaluation focuses on 18 different verticals, which differ in terms of semantics, media type, size, and level of query traffic. An in-depth error analysis reveals unique challenges across different verticals and provides insight into vertical selection for future work.
Based on work conducted at Yahoo! Labs Montreal to be presented at SIGIR 2009.
Tuesday, May 26, 2009
Kai-min Kevin Chang -- Friday May 29, 2009, noon
Speaker: Kai-min Kevin Chang (Language Technologies Institute, School of Computer Science, Carnegie Mellon University)
Time/Date: Friday May 29, 2009, noon.
Place: NSH 1507 (Note the room is not the usual NSH 3002)
Title: Quantitative modeling of the neural representation of adjective-noun phrases to account for fMRI activation
Abstract: Recent advances in functional Magnetic Resonance Imaging (fMRI) offer a significant new approach to studying semantic representations in humans by making it possible to directly observe brain activity while people comprehend words and sentences. In this study, we investigate how humans comprehend adjective-noun phrases (e.g. strong dog) while their neural activity is recorded. Classification analysis shows that the distributed pattern of neural activity contains sufficient signal to decode differences among phrases. Furthermore, vector-based semantic models can explain a significant portion of systematic variance in the observed neural activity. Multiplicative composition models of the two-word phrase outperform additive models, consistent with the assumption that people use adjectives to modify the meaning of the noun, rather than conjoining the meaning of the adjective and noun.
This talk is based on the author's ACL 2009 paper.
Time/Date: Friday May 29, 2009, noon.
Place: NSH 1507 (Note the room is not the usual NSH 3002)
Title: Quantitative modeling of the neural representation of adjective-noun phrases to account for fMRI activation
Abstract: Recent advances in functional Magnetic Resonance Imaging (fMRI) offer a significant new approach to studying semantic representations in humans by making it possible to directly observe brain activity while people comprehend words and sentences. In this study, we investigate how humans comprehend adjective-noun phrases (e.g. strong dog) while their neural activity is recorded. Classification analysis shows that the distributed pattern of neural activity contains sufficient signal to decode differences among phrases. Furthermore, vector-based semantic models can explain a significant portion of systematic variance in the observed neural activity. Multiplicative composition models of the two-word phrase outperform additive models, consistent with the assumption that people use adjectives to modify the meaning of the noun, rather than conjoining the meaning of the adjective and noun.
This talk is based on the author's ACL 2009 paper.
Monday, May 11, 2009
Hua Ai -- Friday May 15, 2009, noon
Speaker: Hua Ai (Intelligent Systems Program at University of Pittsburgh)
Date/Time: Friday May 15, 2009, noon
Location: 3002 Newell-Simon Hall (NSH)
User Simulation for Spoken Dialog System Development
Abstract:
In this talk, I will present my thesis study on investigating how to
evaluate and how to build user simulations to help dialog system
development. When evaluating user simulations, I use both human judges and
automatic evaluation measures to assess the simulation model qualities.
When building user simulations, I examine three factors that impact
simulation models in the tasks of dialog strategy learning and dialog
system development.
The talk is based on the author's ACL 2009 paper.
Date/Time: Friday May 15, 2009, noon
Location: 3002 Newell-Simon Hall (NSH)
User Simulation for Spoken Dialog System Development
Abstract:
In this talk, I will present my thesis study on investigating how to
evaluate and how to build user simulations to help dialog system
development. When evaluating user simulations, I use both human judges and
automatic evaluation measures to assess the simulation model qualities.
When building user simulations, I examine three factors that impact
simulation models in the tasks of dialog strategy learning and dialog
system development.
The talk is based on the author's ACL 2009 paper.
Subscribe to:
Posts (Atom)
