Embedbase Documentation

You can run Embedbase locally in Python like so:

Installation

virtualenv env
source env/bin/activate
pip install embedbase uvicorn

Run

This is the basic codebase to run embedbase in python:

main.py

# main.py
import os
import typing
import uvicorn
from embedbase import get_app
# we also support postgres, supabase, qdrant, or implement your own
# check an example https://github.com/different-ai/embedbase-qdrant
from embedbase.database.memory_db import MemoryDatabase 
from embedbase.embedding.openai import OpenAI
 
app = (
    get_app()
    .use_embedder(OpenAI(os.environ["OPENAI_API_KEY"]))
    .use_db(MemoryDatabase())
    .run()
)
 
if __name__ == "__main__":
    uvicorn.run(app)

Now start embedbase:

python3 main.py

Using the SDK with your self-hosted embedbase

To use the SDK with your self-hosted embedbase, you need to install one of the SDKs:

npm i embedbase-js

Create the client:

import { createClient } from 'embedbase-js'
 
const url = 'http://localhost:8000'
 
const embedbase = createClient(url)

Then you can add some things to your embedbase:

const data =
  await // embeddings are extremely good for retrieving unstructured data
  // in this example we store an unparsable html string
  embedbase.dataset('test-amazon-product-reviews').add(`
  <div>
    <span>Lightweight. Telescopic. Easy zipper case for storage. Didn't put in dishwasher. Still perfect after many uses.</span>
`)
 
console.log(data)
//
// {
//   "id": "eiew823",
//   "data": "Lightweight. Telescopic. Easy zipper case for storage.
//          Didn't put in dishwasher. Still perfect after many uses.",
//   "embedding": [0.1, 0.2, 0.3, ...]
// }

Now you can search by similarity:

// fetching data
const data = await embedbase
  .dataset('test-amazon-product-reviews')
  .search('best hot dogs accessories', { limit: 3 })

What's next

You can deploy your embedbase to render.com in a click (opens in a new tab).

Or use Google Cloud:

main.py

import os
import uvicorn
from embedbase import get_app
 
from embedbase.database.memory_db import MemoryDatabase
from embedbase.embedding.openai import OpenAI
 
 
def main():
    app = (
        get_app()
        .use_embedder(OpenAI(os.getenv("OPENAI_API_KEY")))
        .use_db(MemoryDatabase())
    ).run()
    uvicorn.run(app, host="0.0.0.0", port=8000)

Make sure to install gcloud (opens in a new tab), then:

OPENAI_API_KEY=your-openai-key
gcloud functions deploy embedbase --gen2 --region=us-central1 --runtime=python310 \
 --source=. --trigger-http --set-env-vars OPENAI_API_KEY=$OPENAI_API_KEY \
 --entry-point=main --allow-unauthenticated

🚚 Self-hosting Deployment