pingcap / autoflow

Understanding the Core Concept of the Project

Motivation

The autoflow project aims to build a knowledge graph on top of TiDB Vector, focusing on efficient information retrieval and question answering.

Sub-Topics:

1. Knowledge Graph Construction

  • Objective: Create a knowledge graph that represents the relationships and entities within the data.
  • Process:
    • Data Extraction: Extract relevant information from various sources.
    • Entity Recognition: Identify entities within the extracted data.
    • Relationship Extraction: Determine the relationships between the identified entities.
    • Knowledge Graph Population: Populate the graph with the extracted entities and relationships.
  • Reference: https://github.com/pingcap/autoflow

2. TiDB Vector Integration

  • Objective: Leverage TiDB Vector for efficient similarity search and retrieval.
  • Process:
    • Data Embedding: Embed entities and relationships into vector representations.
    • Index Creation: Create a vector index within TiDB Vector for fast search.
    • Query Processing: Utilize TiDB Vector’s similarity search capabilities to retrieve relevant information based on user queries.
  • Reference: https://github.com/pingcap/tidb-vector

3. Question Answering

  • Objective: Provide answers to user queries in a natural language format.
  • Process:
    • Query Understanding: Parse and interpret user queries to identify the relevant entities and relationships.
    • Knowledge Graph Search: Utilize TiDB Vector to retrieve related entities and relationships from the graph.
    • Answer Generation: Synthesize the retrieved information and generate a coherent and accurate answer.
  • Reference: https://github.com/pingcap/autoflow