We gratefully acknowledge support from
the Simons Foundation
and member institutions

Computation and Language

New submissions

[ total of 18 entries: 1-18 ]
[ showing up to 2000 entries per page: fewer | more ]

New submissions for Fri, 17 Nov 17

[1]  arXiv:1711.05780 [pdf, ps, other]
Title: Detecting Egregious Conversations between Customers and Virtual Agents
Subjects: Computation and Language (cs.CL)

Virtual agents are becoming a prominent channel of interaction in customer service. Not all customer interactions are smooth, however, and some can become almost comically bad. In such instances, a human agent might need to step in and salvage the conversation. Detecting bad conversations is important since disappointing customer service may threaten customer loyalty and impact revenue. In this paper, we outline an approach to detecting such egregious conversations, using behavioral cues from the user, patterns in agent responses, and user-agent interaction. Using logs of two commercial systems, we show that using these features improves the detection F1-score by around 20% over using textual features alone. In addition, we show that those features are common across two quite different domains and, arguably, universal.

[2]  arXiv:1711.05789 [pdf, other]
Title: CMU LiveMedQA at TREC 2017 LiveQA: A Consumer Health Question Answering System
Comments: To appear in Proceedings of TREC 2017
Subjects: Computation and Language (cs.CL); Information Retrieval (cs.IR)

In this paper, we present LiveMedQA, a question answering system that is optimized for consumer health question. On top of the general QA system pipeline, we introduce several new features that aim to exploit domain-specific knowledge and entity structures for better performance. This includes a question type/focus analyzer based on deep text classification model, a tree-based knowledge graph for answer generation and a complementary structure-aware searcher for answer retrieval. LiveMedQA system is evaluated in the TREC 2017 LiveQA medical subtask, where it received an average score of 0.356 on a 3 point scale. Evaluation results revealed 3 substantial drawbacks in current LiveMedQA system, based on which we provide a detailed discussion and propose a few solutions that constitute the main focus of our subsequent work.

[3]  arXiv:1711.05795 [pdf, other]
Title: Finer Grained Entity Typing with TypeNet
Comments: Accepted at 6th Workshop on Automated Knowledge Base Construction (AKBC) at NIPS 2017
Subjects: Computation and Language (cs.CL); Neural and Evolutionary Computing (cs.NE)

We consider the challenging problem of entity typing over an extremely fine grained set of types, wherein a single mention or entity can have many simultaneous and often hierarchically-structured types. Despite the importance of the problem, there is a relative lack of resources in the form of fine-grained, deep type hierarchies aligned to existing knowledge bases. In response, we introduce TypeNet, a dataset of entity types consisting of over 1941 types organized in a hierarchy, obtained by manually annotating a mapping from 1081 Freebase types to WordNet. We also experiment with several models comparable to state-of-the-art systems and explore techniques to incorporate a structure loss on the hierarchy with the standard mention typing loss, as a first step towards future research on this dataset.

[4]  arXiv:1711.05851 [pdf, other]
Title: Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning
Comments: ICLR 2018 submission
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Knowledge bases (KB), both automatically and manually constructed, are often incomplete --- many valid facts can be inferred from the KB by synthesizing existing information. A popular approach to KB completion is to infer new relations by combinatory reasoning over the information found along other paths connecting a pair of entities. Given the enormous size of KBs and the exponential number of paths, previous path-based models have considered only the problem of predicting a missing relation given two entities or evaluating the truth of a proposed triple. Additionally, these methods have traditionally used random paths between fixed entity pairs or more recently learned to pick paths between them. We propose a new algorithm MINERVA, which addresses the much more difficult and practical task of answering questions where the relation is known, but only one entity. Since random walks are impractical in a setting with combinatorially many destinations from a start node, we present a neural reinforcement learning approach which learns how to navigate the graph conditioned on the input query to find predictive paths. Empirically, this approach obtains state-of-the-art results on several datasets, significantly outperforming prior methods.

[5]  arXiv:1711.05885 [pdf, other]
Title: Crowdsourcing Question-Answer Meaning Representations
Comments: 8 pages, 6 figures, 2 tables
Subjects: Computation and Language (cs.CL)

We introduce Question-Answer Meaning Representations (QAMRs), which represent the predicate-argument structure of a sentence as a set of question-answer pairs. We also develop a crowdsourcing scheme to show that QAMRs can be labeled with very little training, and gather a dataset with over 5,000 sentences and 100,000 questions. A detailed qualitative analysis demonstrates that the crowd-generated question-answer pairs cover the vast majority of predicate-argument relationships in existing datasets (including PropBank, NomBank, QA-SRL, and AMR) along with many previously under-resourced ones, including implicit arguments and relations. The QAMR data and annotation code is made publicly available to enable future work on how best to model these complex phenomena.

[6]  arXiv:1711.06061 [pdf, other]
Title: An Encoder-Decoder Framework Translating Natural Language to Database Queries
Subjects: Computation and Language (cs.CL)

Machine translation is going through a radical revolution, driven by the explosive development of deep learning techniques using Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). In this paper, we consider a special case in machine translation problems, targeting to translate natural language into Structural Query Language (SQL) for data retrieval over relational database. Although generic CNN and RNN learn the grammar structure of SQL when trained with sufficient samples, the accuracy and training efficiency of the model could be dramatically improved, when the translation model is deeply integrated with the grammar rules of SQL. We present a new encoder-decoder framework, with a suite of new approaches, including new semantic features fed into the encoder as well as new grammar-aware states injected into the memory of decoder. These techniques help the neural network focus on understanding semantics of the operations in natural language and save the efforts on SQL grammar learning. The empirical evaluation on real world database and queries show that our approach outperform state-of-the-art solution by a significant margin.

[7]  arXiv:1711.06141 [pdf, other]
Title: ConvAMR: Abstract meaning representation parsing
Comments: SCIDOCA2017, Japan
Subjects: Computation and Language (cs.CL)

Convolutional neural networks (CNN) have recently achieved remarkable performance in a wide range of applications. In this research, we equip convolutional sequence-to-sequence (seq2seq) model with an efficient graph linearization technique for abstract meaning representation parsing. Our linearization method is better than the prior method at signaling the turn of graph traveling. Additionally, convolutional seq2seq model is more appropriate and considerably faster than the recurrent neural network models in this task. Our method outperforms previous methods by a large margin on both the standard dataset LDC2014T12. Our result indicates that future works still have a room for improving parsing model using graph linearization approach.

[8]  arXiv:1711.06196 [pdf, other]
Title: Addressing Cross-Lingual Word Sense Disambiguation on Low-Density Languages: Application to Persian
Subjects: Computation and Language (cs.CL); Information Retrieval (cs.IR)

We explore the use of unsupervised methods in Cross-Lingual Word Sense Disambiguation (CL-WSD) with the application of English to Persian. Our proposed approach targets the languages with scarce resources (low-density) by exploiting word embedding and semantic similarity of the words in context. We evaluate the approach on a recent evaluation benchmark and compare it with the state-of-the-art unsupervised system (CO-Graph). The results show that our approach outperforms both the standard baseline and the CO-Graph system in both of the task evaluation metrics (Out-Of-Five and Best result).

[9]  arXiv:1711.06238 [pdf, other]
Title: An Abstractive approach to Question Answering
Authors: Rajarshee Mitra
Subjects: Computation and Language (cs.CL)

Question Answering has come a long way from answer sentence selection, relational QA to reading and comprehension. We move our attention to abstractive question answering by which we facilitate machine to read passages and answer questions by generating them. We frame the problem as a sequence to sequence learning where the encoder being a network that models the relation between question and passage, thereby relying solely on passage and question content to form an abstraction of the answer. Not being able to retain facts and making repetitions are common mistakes that affect the overall legibility of answers. To counter these issues, we employ copying mechanism and maintenance of coverage vector in our model respectively. Our results on MS-MARCO demonstrates it's superiority over baselines and we also show qualitative examples where we improved in terms of correctness and readability.

Cross-lists for Fri, 17 Nov 17

[10]  arXiv:1711.06004 (cross-list from cs.IR) [pdf, other]
Title: Remedies against the Vocabulary Gap in Information Retrieval
Comments: PhD thesis
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

Search engines rely heavily on term-based approaches that represent queries and documents as bags of words. Text---a document or a query---is represented by a bag of its words that ignores grammar and word order, but retains word frequency counts. When presented with a search query, the engine then ranks documents according to their relevance scores by computing, among other things, the matching degrees between query and document terms. While term-based approaches are intuitive and effective in practice, they are based on the hypothesis that documents that exactly contain the query terms are highly relevant regardless of query semantics. Inversely, term-based approaches assume documents that do not contain query terms as irrelevant. However, it is known that a high matching degree at the term level does not necessarily mean high relevance and, vice versa, documents that match null query terms may still be relevant. Consequently, there exists a vocabulary gap between queries and documents that occurs when both use different words to describe the same concepts. It is the alleviation of the effect brought forward by this vocabulary gap that is the topic of this dissertation. More specifically, we propose (1) methods to formulate an effective query from complex textual structures and (2) latent vector space models that circumvent the vocabulary gap in information retrieval.

[11]  arXiv:1711.06095 (cross-list from cs.CV) [pdf, other]
Title: Depression Severity Estimation from Multiple Modalities
Comments: 8 pages, 1 figure
Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

Depression is a major debilitating disorder which can affect people from all ages. With a continuous increase in the number of annual cases of depression, there is a need to develop automatic techniques for the detection of the presence and extent of depression. In this AVEC challenge we explore different modalities (speech, language and visual features extracted from face) to design and develop automatic methods for the detection of depression. In psychology literature, the PHQ-8 questionnaire is well established as a tool for measuring the severity of depression. In this paper we aim to automatically predict the PHQ-8 scores from features extracted from the different modalities. We show that visual features extracted from facial landmarks obtain the best performance in terms of estimating the PHQ-8 results with a mean absolute error (MAE) of 4.66 on the development set. Behavioral characteristics from speech provide an MAE of 4.73. Language features yield a slightly higher MAE of 5.17. When switching to the test set, our Turn Features derived from audio transcriptions achieve the best performance, scoring an MAE of 4.11 (corresponding to an RMSE of 4.94), which makes our system the winner of the AVEC 2017 depression sub-challenge.

[12]  arXiv:1711.06232 (cross-list from cs.CV) [pdf, other]
Title: A Novel Framework for Robustness Analysis of Visual QA Models
Comments: Submitted to IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018
Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

Deep neural networks have been playing an essential role in many computer vision tasks including Visual Question Answering (VQA). Until recently, the study of their accuracy has been the main focus of research and now there is a huge trend toward assessing the robustness of these models against adversarial attacks by evaluating the accuracy of these models under increasing levels of noisiness. In VQA, the attack can target the image and/or the proposed main question and yet there is a lack of proper analysis of this aspect of VQA. In this work, we propose a new framework that uses semantically relevant questions, dubbed basic questions, acting as noise to evaluate the robustness of VQA models. We hypothesize that as the similarity of a basic question to the main question decreases, the level of noise increases. So, to generate a reasonable noise level for a given main question, we rank a pool of basic questions based on their similarity with this main question. We cast this ranking problem as a LASSO optimization problem. We also propose a novel robustness measure R_score and two large-scale question datasets, General Basic Question Dataset and Yes/No Basic Question Dataset in order to standardize robustness analysis of VQA models. We analyze the robustness of several state-of-the-art VQA models and show that attention-based VQA models are more robust than other methods in general. The main goal of this framework is to serve as a benchmark to help the community in building more accurate and robust VQA models.

Replacements for Fri, 17 Nov 17

[13]  arXiv:1706.07179 (replaced) [pdf, other]
Title: RelNet: End-to-End Modeling of Entities & Relations
Comments: Accepted in AKBC 2017
Subjects: Computation and Language (cs.CL); Learning (cs.LG)
[14]  arXiv:1709.05475 (replaced) [pdf, other]
Title: Order-Preserving Abstractive Summarization for Spoken Content Based on Connectionist Temporal Classification
Comments: Accepted by Interspeech 2017
Subjects: Computation and Language (cs.CL)
[15]  arXiv:1710.08312 (replaced) [pdf, other]
Title: Attending to All Mention Pairs for Full Abstract Biological Relation Extraction
Comments: 6th Workshop on Automated Knowledge Base Construction (AKBC)
Subjects: Computation and Language (cs.CL)
[16]  arXiv:1711.04204 (replaced) [pdf, other]
Title: Commonsense LocatedNear Relation Extraction
Comments: 6 pages + 2 pages of reference. Accepted by AKBC@NIPS'17
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
[17]  arXiv:1711.04289 (replaced) [pdf, other]
Title: Natural Language Inference with External Knowledge
Comments: Submitted to ICLR 2018
Subjects: Computation and Language (cs.CL)
[18]  arXiv:1711.05380 (replaced) [pdf, other]
Title: Bridging Source and Target Word Embeddings for Neural Machine Translation
Comments: 8 pages, 6 figures
Subjects: Computation and Language (cs.CL)
[ total of 18 entries: 1-18 ]
[ showing up to 2000 entries per page: fewer | more ]

Disable MathJax (What is MathJax?)