The project “genai-stack” by Docker uses various open-source technologies and dependencies for creating a PDF reader that loads local PDFs into text chunks and embeds them into Neo4j. It allows asking questions about the PDF contents and having a large language model (LLM) answer them using vector similarity search.
Key Technologies and Dependencies:
Docker and Compose: Docker is an open-source platform that automates the deployment, scaling, and management of applications. Compose is a tool for defining and running multi-container Docker applications. Together, they enable a consistent environment for development, testing, and production. (Sources: https://opensource.com/alternatives/adobe-acrobat and https://www.docker.com/)
Python: A popular programming language known for its simplicity and readability. It is used in this project for scripting and automation tasks. (Source: https://opensource.com/alternatives/adobe-acrobat)
Neo4j: An open-source, graph database management system that enables the storage and retrieval of data in graph format. It is used in this project to embed and query PDF contents. (Source: https://neo4j.com/)
OpenAI: An artificial intelligence research laboratory consisting of the for-profit corporation OpenAI LP and its parent company, the non-profit OpenAI Inc. It is focused on promoting and developing friendly AI in a way that benefits humanity as a whole. (Source: https://en.wikipedia.org/wiki/OpenAI)
Boto3: The Amazon Web Services (AWS) Software Development Kit (SDK) for Python, which allows Python developers to write software that makes use of AWS services like Amazon S3, Amazon EC2, and others. (Source: https://boto3.amazonaws.com/v1/documentation/api/latest/index.html)
FastAPI: A modern, fast (high-performance), web framework for building APIs with Python 3.6+ based on standard Python type hints. (Source: https://fastapi.tiangolo.com/)
Torch and Sentence Transformers: Torch is an open-source machine learning library for Python. Sentence Transformers is a Python framework for state-of-the-art sentence, text and image embeddings. (Sources: https://pytorch.org/ and https://www.sbert.net/)
Langchain: A tool for training large language models. It is used in this project to create a model that can answer questions about PDF contents. (Source: https://github.com/langchain/langchain)
For working with PDFs, various open-source tools are available, such as pdftk-java, Evince, Okular, and SumatraPDF. These tools can be used for creating, editing, and viewing PDF files. (Sources: https://opensource.com/article/19/2/manipulating-pdfs-linux and https://opensource.com/alternatives/adobe-acrobat)
The project “genai-stack” combines these technologies to create a powerful PDF reader that allows users to ask questions about the contents of a PDF and receive answers using vector similarity search.
