How I Connect Ai To My Own Data
A lot of developers and business leaders tell me their biggest frustration with AI isn’t that it’s too weak, but that it feels too generic. The question I hear most often is, “How do I securely and efficiently connect AI to my own data?” In this article, I’ll walk you through exactly how I do it, step by step, using the same techniques I rely on in my own projects.
Here’s How I Connect AI to My Own Data
These are the main techniques I use to connect AI to my own data.
1. Feeding Files Directly (Prompt Stuffing)
When I first tried adding private context to LLMs, I did what most people do: I copied and pasted text into the prompt. For engineers, this means reading a file—like a text document, JSON, or CSV—and sending its contents to the model along with the user’s question.
This is the simplest method. You don’t need a complex setup, just basic Python file I/O. But I soon realized it has big drawbacks. LLMs have strict context limits, and even with today’s larger token windows, sending a 500-page PDF every time is slow and expensive.
Here’s an example:
import openai
openai.api_key = "your-api-key"
# Read your private data
with open("meeting_notes.txt", "r") as file:
private_data = file.read()
user_question = "What were the key deliverables mentioned?"
# Stuff the data into the prompt
response = openai.chat.completions.create(
model="gpt-4",
messages=[
{"role": "system", "content": "Answer the user's question using ONLY the provided text."},
{"role": "user", "content": f"Text: {private_data}\n\nQuestion: {user_question}"}
]
)
print(response.choices[0].message.content)import openai
openai.api_key = "your-api-key"
# Read your private data
with open("meeting_notes.txt", "r") as file:
private_data = file.read()
user_question = "What were the key deliverables mentioned?"
# Stuff the data into the prompt
response = openai.chat.completions.create(
model="gpt-4",
messages=[
{"role": "system", "content": "Answer the user's question using ONLY the provided text."},
{"role": "user", "content": f"Text: {private_data}\n\nQuestion: {user_question}"}
]
)
print(response.choices[0].message.content)I only use this method for small, one-time tasks where the data easily fits into memory, like summarizing a meeting transcript or parsing a short log file.
2. Chatting with Structured Data: Databases and SQL
As an AI engineer, most of the data I work with isn’t in flat files; it’s in relational databases. You can’t just pass a whole SQL database into a prompt. Instead, I show my mentees how to use the LLM as a query translator.
You give the AI the database schema (table names and column definitions) and ask it to write a SQL query based on the user’s question. Your Python code then runs that SQL on your database and gets the raw data. If you want, you can send that data back to the LLM to turn it into a conversational answer.
Here’s an example:
import sqlite3
import openai
# 1. Connect to your database
conn = sqlite3.connect('company_data.db')
cursor = conn.cursor()
# 2. Define your schema to show the LLM
schema = """
Table: employees
Columns: id (INT), name (TEXT), department (TEXT), salary (INT)
"""
user_question = "Who is the highest paid employee in the engineering department?"
# 3. Ask LLM to write the SQL
prompt = f"Given this schema:\n{schema}\nWrite a secure SQLite query to answer: {user_question}. Return ONLY the SQL query."
response = openai.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": prompt}]
)
sql_query = response.choices[0].message.content.strip()
# 4. Execute the query locally
cursor.execute(sql_query)
result = cursor.fetchall()
print(f"Query Result: {result}")import sqlite3
import openai
# 1. Connect to your database
conn = sqlite3.connect('company_data.db')
cursor = conn.cursor()
# 2. Define your schema to show the LLM
schema = """
Table: employees
Columns: id (INT), name (TEXT), department (TEXT), salary (INT)
"""
user_question = "Who is the highest paid employee in the engineering department?"
# 3. Ask LLM to write the SQL
prompt = f"Given this schema:\n{schema}\nWrite a secure SQLite query to answer: {user_question}. Return ONLY the SQL query."
response = openai.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": prompt}]
)
sql_query = response.choices[0].message.content.strip()
# 4. Execute the query locally
cursor.execute(sql_query)
result = cursor.fetchall()
print(f"Query Result: {result}")The biggest mistake I see beginners make is letting the AI write directly to the database. Don’t do this. Always set your database connection to read-only and carefully check the generated queries before running them to avoid data loss or SQL injection.
3. Embeddings, Vector Databases, and RAG
When my clients have huge amounts of unstructured data, like thousands of PDFs, internal wikis, or years of product manuals, prompt stuffing just doesn’t work. This is when Retrieval-Augmented Generation (RAG) is essential.
Here’s how it works: You split your large documents into smaller text chunks. Then, you use an embedding model to turn those chunks into arrays of numbers (vectors) that capture their meaning. You store these vectors in a special vector database.
Here’s an example:
import chromadb
from sentence_transformers import SentenceTransformer
# Initialize embedding model and vector database
embedder = SentenceTransformer('all-MiniLM-L6-v2')
chroma_client = chromadb.Client()
collection = chroma_client.create_collection(name="company_docs")
# 1. Chunk and embed your own data
documents = [
"Our refund policy allows returns within 30 days.",
"The new AI feature launches in Q4.",
"Employee health benefits are provided by BlueCross."
]
embeddings = embedder.encode(documents).tolist()
# 2. Store in Vector DB
collection.add(
embeddings=embeddings,
documents=documents,
ids=["doc1", "doc2", "doc3"]
)
# 3. Search based on user query
query = "How long do I have to return a product?"
query_embedding = embedder.encode([query]).tolist()
results = collection.query(
query_embeddings=query_embedding,
n_results=1
)
# 4. Pass the retrieved context to your LLM (as shown in Step 1)
print(f"Retrieved Context: {results['documents'][0][0]}")import chromadb
from sentence_transformers import SentenceTransformer
# Initialize embedding model and vector database
embedder = SentenceTransformer('all-MiniLM-L6-v2')
chroma_client = chromadb.Client()
collection = chroma_client.create_collection(name="company_docs")
# 1. Chunk and embed your own data
documents = [
"Our refund policy allows returns within 30 days.",
"The new AI feature launches in Q4.",
"Employee health benefits are provided by BlueCross."
]
embeddings = embedder.encode(documents).tolist()
# 2. Store in Vector DB
collection.add(
embeddings=embeddings,
documents=documents,
ids=["doc1", "doc2", "doc3"]
)
# 3. Search based on user query
query = "How long do I have to return a product?"
query_embedding = embedder.encode([query]).tolist()
results = collection.query(
query_embeddings=query_embedding,
n_results=1
)
# 4. Pass the retrieved context to your LLM (as shown in Step 1)
print(f"Retrieved Context: {results['documents'][0][0]}")When someone asks a question, you turn it into a vector using the same embedding model. You compare this vector to your database and pull out the top three or four most relevant chunks. Then, you send only those chunks to the LLM to answer the question.
I often use libraries like sentence-transformers with vector databases like ChromaDB or FAISS for this. RAG keeps your prompts small, saves a lot on API costs, and greatly reduces AI hallucinations because the model has to use the exact text you retrieved.
4. APIs and Tool Calling
RAG is great for reading data, but what if I want the AI to do something with my data? For example, maybe I need it to update a CRM, get live stock prices from a private API, or send an email.
This is where tool calling (also called function calling) changed how I build apps. It’s important to remember that the LLM doesn’t run code itself. Instead, it creates a structured JSON response that tells your Python app which local function to run and what arguments to use.
Here’s an example:
import openai
import json
# Your secure, local function
def fetch_customer_status(customer_id):
# Imagine this hits your private API or Database
database = {"C123": "Premium Member", "C456": "Free Tier"}
return database.get(customer_id, "Customer not found")
tools = [
{
"type": "function",
"function": {
"name": "fetch_customer_status",
"description": "Get the subscription tier of a customer by ID.",
"parameters": {
"type": "object",
"properties": {
"customer_id": {"type": "string", "description": "The unique customer ID"}
},
"required": ["customer_id"]
}
}
}
]
# The LLM decides to use the tool
response = openai.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "What is the status of customer C123?"}],
tools=tools,
tool_choice="auto"
)
# Your Python code executes the tool based on the LLM's instruction
tool_call = response.choices[0].message.tool_calls[0]
if tool_call.function.name == "fetch_customer_status":
arguments = json.loads(tool_call.function.arguments)
result = fetch_customer_status(arguments["customer_id"])
print(f"Action Output: {result}")import openai
import json
# Your secure, local function
def fetch_customer_status(customer_id):
# Imagine this hits your private API or Database
database = {"C123": "Premium Member", "C456": "Free Tier"}
return database.get(customer_id, "Customer not found")
tools = [
{
"type": "function",
"function": {
"name": "fetch_customer_status",
"description": "Get the subscription tier of a customer by ID.",
"parameters": {
"type": "object",
"properties": {
"customer_id": {"type": "string", "description": "The unique customer ID"}
},
"required": ["customer_id"]
}
}
}
]
# The LLM decides to use the tool
response = openai.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "What is the status of customer C123?"}],
tools=tools,
tool_choice="auto"
)
# Your Python code executes the tool based on the LLM's instruction
tool_call = response.choices[0].message.tool_calls[0]
if tool_call.function.name == "fetch_customer_status":
arguments = json.loads(tool_call.function.arguments)
result = fetch_customer_status(arguments["customer_id"])
print(f"Action Output: {result}")When I build AI agents, tool calling is at the core. You define a Python function, write a JSON description of what it does, and give that to the LLM. All your security, API keys, and database credentials stay safe in your Python environment. The LLM just acts as a routing engine.
Continue Learning
If you want to learn more about building real-world AI applications, my book, Hands-on GenAI, LLMs and AI Agents, is a great next step. It covers LLMs, RAG, AI agents, and practical Generative AI projects to help you turn these ideas into working apps.
For a more structured learning path, you can also check out the Generative AI Engineering Professional Certificate by IBM. It teaches practical skills for building Generative AI apps, including Python, LLMs, RAG, and working with AI models and data.
The Takeaway
If there’s one thing I hope you remember, it’s that building reliable AI systems is more about the data pipelines than the models themselves. When people first come to me, they’re often focused on picking the best model, like GPT-4, Claude 3, or Llama 3.
But from my experience, a simple, smaller model that’s connected to high-quality, well-organized internal data will always beat a more advanced, expensive model that uses messy or poorly retrieved data. AI is only as smart as the data you give it.
I hope you enjoyed this article on how I connect AI to my own data. For more tips on AI and machine learning, you’re welcome to follow me on Instagram.
The post How I Connect AI to My Own Data appeared first on AmanXai by Aman Kharwal.
Popular Products
-
Fake Pregnancy Test$61.56$30.78 -
Anti-Slip Safety Handle for Elderly S...$57.56$28.78 -
Toe Corrector Orthotics$41.56$20.78 -
Waterproof Trauma Medical First Aid Kit$169.56$84.78 -
Rescue Zip Stitch Kit$109.56$54.78