QA
Intelligence 7 min read

All you need to know about Question-Answering (QA) systems built using NLP

Explore various solutions for building a QA Utility

Blog Overview

The goal is to explore various solutions for building a Question-Answering(QA) Utility. Presenting you a quick glance at all the information I gathered by going through various research papers, blogs, and articles, whether it be the different classifications of QA Systems, Frameworks, and Components of QA Systems, or various ways by which we can implement the same. 

Introduction to QA System

A Question-Answering (QA) system is a natural language processing (NLP) application that is designed to automatically answer questions posed in natural language. QA systems use a variety of techniques, including natural language understanding, information retrieval, and machine learning, to extract the relevant information from a given text corpus and generate an answer to a user’s question. Some QA systems are able to answer questions based on a specific domain or knowledge base, while others are more general and can answer a wide range of questions. They are widely used in fields like customer service, knowledge management, and e-commerce.

Classification of QA System

The framework of QA System

The Question-Answering Framework is built using Natural Language Processing and Information Retrieval Technique. The framework is divided into 4 modules as follows:

This can also be understood by the figure below:

Components of the QA System

Retriever-Generator Model

The retriever-generator architecture is used to generate answers to user questions based on information stored in a large database. The retriever component searches the database for relevant information, and the generator component uses that information to generate a natural language answer.

The generator component is typically implemented using a machine learning model, such as a transformer or a BERT model, which is trained to generate human-like text. The retriever component may be implemented using a search engine or other information retrieval system.

The retriever-generator architecture is useful because it allows the QA system to generate answers based on a wide range of information sources, while also providing the flexibility to generate answers in a variety of different formats (e.g., text, tables, graphs). It is a popular choice for building QA systems that need to handle a large volume of queries and provide accurate and relevant answers in real time.

Rule-based approaches 

One way to build a question-answering (QA) system is through a rule-based approach, where a set of rules is created for the model to follow in order to determine the correct answer to a given question. This approach can be effective for generating responses to certain types of non-factoid questions, but it also has its limitations.

Keyword Matching

One example of a rule-based approach is a “keyword matching” method. This approach involves looking for specific keywords in the question and then searching for those keywords in a database of information to find the answer. For example, if the question is “What is the capital of France?”, the model would look for the keywords “capital” and “France” and then find the answer “Paris” in the database. This method can be especially useful for simple factoid questions.

Syntax Parsing

Another example of a rule-based approach is a “syntax parsing” method. This approach involves analyzing the structure of the question to determine what information is being asked for, and then searching for that information in a database. For example, if the question is “Which country is known for its cheese?”, the model would understand that the question is looking for a country and that the answer should be related to cheese. By parsing the syntax, the model can then search for the answer “France” in the database.

Limitations

While a rule-based approach can be effective for generating responses to certain types of non-factoid questions, it can be difficult to cover all possible variations and nuances of language with a fixed set of rules. Additionally, rule-based systems are not as flexible as machine learning-based approaches, which can learn to generate responses based on patterns in large amounts of training data. For example, a rule-based system would struggle to understand a question like “What is the best way to make macaroni and cheese?”, which requires understanding the context and making a subjective judgment.

Linguistic Approach to QA

When it comes to understanding natural language text, one approach is to use linguistic techniques such as tokenization, POS tagging, and parsing. These techniques can help to formulate a user’s question into a precise query that can extract the relevant information from a structured database. This approach is known as a linguistic approach to question answering (QA).

Effectiveness

Linguistic approaches can be quite effective for generating abstractive responses to non-factoid questions, particularly when combined with machine learning techniques such as deep learning. However, the performance of these approaches will depend on the quality and coverage of the linguistic resources and knowledge bases being used, as well as the specific task being performed.

Limitations

Linguistic approaches are heavily dependent on the quality of the data, and knowledge bases that are being used. If the data is not accurate or is incomplete, the answer generated will not be accurate. Additionally, this approach may not be good for factoid questions as it heavily depends on structured data and the answers to factoid questions are usually short and precise.

Statistical approach

The rise of big data and the internet has made statistical approaches more important in building question-answering (QA) systems. Statistical learning methods can provide better results than other approaches. These methods use online text repositories and can accept natural language queries. Examples of statistical techniques used in QA systems include support vector machine classifiers, Bayesian classifiers, and maximum entropy models.

Advantages of Statistical Approach

The availability of huge amounts of data on the internet has made statistical approaches more significant. These methods can give better results than other approaches. They can also use online text repositories and formulate queries in natural language form, rather than relying on structured query languages.

Techniques Used in Statistical QA Systems

Many statistical-based QA systems use techniques such as Support Vector Machine classifiers, Bayesian Classifiers, and maximum entropy models. These techniques can help the system understand the underlying patterns in the data and generate accurate answers.

Support Vector Machine Classifiers

One statistical technique commonly used in QA systems is support vector machine (SVM) classifiers. SVM classifiers are a type of machine-learning algorithm that can be used for both classification and regression tasks. They work by finding the best boundary or “hyperplane” that separates different classes of data.

Bayesian Classifiers and Maximum Entropy Models

Another statistical technique used in QA systems is Bayesian classifiers. These classifiers use Bayes’ theorem to make predictions based on prior knowledge and new evidence. Maximum entropy models are also used in QA systems, which use the principle of maximum entropy to make predictions about the probability distribution of a set of variables.

Neural Networks

Attention-over-Attention (AoA)

One example of a neural network used in QA systems is the Attention-over-Attention (AoA) model. AoA is a neural net model that uses multiple levels of attention mechanisms to focus on different parts of the input. For example, in a question-answering task, one level of attention might focus on the question and another on the context in which the question is being asked. During training, the model is fed input-output pairs and uses its attention mechanisms to generate an output that is compared to the correct answer.

r-NET

r-NET is an extractive reading comprehension model developed by Microsoft. It is designed to extract an answer span from the input text, rather than generating abstractive responses. It uses a gated attention-based recurrent network to match a question and passage and then predicts the start and end positions of the answer inside the passage.

Reasoning Network

Microsoft’s Reasoning Network, also known as QnA Maker, is a natural language processing tool that can be used to build and deploy QA systems. It is designed to generate answers to fact-based questions, using a combination of machine learning and knowledge base technologies.

FusionNet

FusionNet is a machine learning model that has been developed for text classification tasks. It is based on a fully-aware attention mechanism, which allows the model to focus on different parts of the input text when making predictions. It has been applied to machine comprehension, which involves understanding and answering questions about a given piece of text.

BiDAF

BiDAF is a closed-domain, extractive Q&A model that can only answer factoid questions. It requires a context to answer a query and the answer returned is always a substring of the provided context.

Seq2Seq-Attention

Seq2Seq models consist of an encoder network that processes the input sequence and a decoder network that generates the output sequence. The decoder can use an attention mechanism to selectively focus on different parts of the input sequence when generating the output.

Fig: seq2seq attention model

Q = question

P = paragraph(context)

H_P, H_Q = encoded question and paragraph matrices 

H_C = weighted context under the specific question

When it comes to building a question-answering (QA) system with a neural network approach, there are a few key factors to consider:

Transformer-Based QA system

One of the most popular and widely used approaches is transformer-based QA systems. These systems have achieved state-of-the-art performance on a number of QA benchmarks and are widely used in industry and research. Transformer-based QA systems have become increasingly popular in recent years, and have achieved state-of-the-art performance on a number of QA benchmarks. These systems are widely used in industry and research. There are a number of state-of-the-art QA models that are based on transformer architectures. 

BERT

One popular transformer-based QA model is BERT (Bidirectional Encoder Representations from Transformers). It is a bidirectional model, which means it can consider the full context of a word by looking at the words that come before and after it. This is particularly useful for understanding the intent behind a query. BERT has been shown to achieve state-of-the-art performance on a wide range of natural language processing tasks, including language translation, language modeling, and text classification. BERT has been shown to achieve state-of-the-art performance on a wide range of natural language processing tasks, including language translation, language modeling, and text classification.

RoBERTa: Robustly Optimized BERT

RoBERTa is an extension of BERT that was developed by researchers at Facebook. It is trained on a larger dataset and using a different training objective, which makes it more robust and more effective for a variety of natural language processing tasks.

DistilBERT

DistilBERT is a smaller and more efficient version of BERT that was developed by researchers at Hugging Face. It is trained using the same training objective as BERT but is able to achieve similar performance using fewer parameters and less computation.

XL-Net

XL-Net is a transformer-based language model developed by researchers at Google. It is trained on a large dataset of unstructured text using a novel training objective called “permutation language modeling,” which allows the model to better capture the dependencies between words and phrases in the input.

Graph-based approach 

One of the most popular and widely used approaches is a graph-based approach to building a question-answering (QA) system. This approach involves representing the relationships between entities in a graph structure and using that structure to reason about the relationships between entities and predict answers to questions.

Knowledge Graph

One example of a graph-based model for question answering is the “knowledge graph” approach. In this approach, the model represents entities and relationships in the form of nodes and edges in a graph. The structure of the graph is then used to reason about the correct answer to a question. For example, if the question is “Who is the president of the United States?”, the model would use the structure of the graph to find the node representing the president of the United States and return the answer “Joe Biden”.

Graph Convolutional Network

Another example of a graph-based approach is the “graph convolutional network” (GCN) approach. In this approach, the model uses convolutional neural networks to process the graph representation of the information and make predictions about the correct answer. This approach can be more effective than traditional convolutional networks in handling graph-structured data and can achieve state-of-the-art performance on various graph-based tasks.

RDF

RDF (Resource Description Framework) is a standardized model for representing data as a graph of relationships between resources. It can be used to represent a wide range of information, including facts and relationships between resources. By organizing information in this way, it may be possible to use RDF to represent the knowledge base that a question-answering system relies on, and to use graph-based algorithms to search for and retrieve relevant information in response to a user’s question.

Different Approaches, Different Datasets

The specific types of data you need will depend on the approach you are using to build the question-answering system. For example, if you are using a rule-based approach, you may only need a dataset of questions and answers. If you are using an information retrieval approach, you may need a dataset of documents and corresponding answers. And if you are using a machine learning approach, you may need a dataset of context, questions, and answers. It’s important to carefully consider the types of data you need and plan accordingly to ensure that you have a high-quality dataset that will enable us to build an accurate and reliable question-answering system.

Evaluating QA System Performance

When building a question-answering system, it’s important to have a way to evaluate its performance. There are several metrics that can be used, including:

These metrics can help to identify areas where the system needs improvement and guide the development process.

Challenges in Q&A

Question-answering systems come with their own set of challenges. Some of the major ones include:

Need help with technology
for your digital platform?

Get to know how technology can be leveraged to turn your idea into a reality.
Schedule a call with design expert