Build a RAG App with pgvector on Databricks Lakebase
Build a retrieval-augmented generation app over real SEC 10-K filings. It has hybrid search, company and year filters, cited answers, and a quality gate that blocks a bad deploy.
- Level
- Intermediate
- Duration
- About 45 min
- Cost
- Free tier
- Runs on a vendor free plan.
- Technologies
- RAGpgvectorDatabricks LakebaseDatabricks AppsPostgreSQL
What you will build
- A RAG app on Postgres with pgvector, deployed as a Databricks App
- A pinned snapshot of real 10-K filings, split by section and loaded with embeddings
- Hybrid retrieval that fuses vector search and full-text search
- Company and year filters, suggested questions, and numbered citations with the filing URL
- A retrieval check that must reach a 0.6 hit rate before the app deploys
Prerequisites
- A Databricks Free Edition account
- The Databricks CLI, logged in with databricks auth login
- Python 3.9 or newer with pip
- Basic comfort with SQL and the terminal
Run it
Install once, then launch the lab. Preview the steps first with the dry run. Remove everything when you finish.
pip install 'labagent[rag]'
databricks auth login
# preview the steps without changing anything
labagent launch rag-sec-filings --name demo --dry-run
# launch the lab (safe to re-run, it resumes where it stopped)
labagent launch rag-sec-filings --name demo
# remove only what the lab created
labagent destroy rag-sec-filings --name demoSteps
- 1
Preflight
Check your login, the serving endpoints, and the Free Edition project limit.
- 2
Create the Lakebase project
Start the Postgres project that holds the vectors.
- 3
Fetch, parse and chunk
Download the filings snapshot, verify its checksum, and split it by section.
- 4
Enable pgvector and load
Create the table, embed each chunk, and load it.
- 5
Build the indexes
Add vector, full-text and metadata indexes.
- 6
Evaluate retrieval
Score retrieval on a question set. A low score stops the deploy.
- 7
Deploy the app
Publish the Databricks App and open it.
FAQ
Does this lab cost money?+
What if a step fails?+
How do I remove everything?+
Where does the filings data come from?+
Why can the deploy stop before the app goes live?+
Pick how you want to use this lab
Run this lab for your team
Tell us the class size and the date. We reply within one business day.
Related labs
Smoke-Test a RAG Pipeline on Databricks Lakebase
Run the full RAG pipeline on a small demo corpus of 25 passages. Use it to check your Databricks setup before you build with real data.
Deploy a Text-to-SQL AI Agent on Databricks Apps
Deploy the open-source db-agent to Databricks Apps over a built-in sample dataset. The lab then asks a test question through the live app to confirm it works.