OpenAI CLIP Embedding Similarity
OpenAI CLIP Embeddings similarity feature is in the feature-experimental-clip branch. The embeddings were slowing down my test Elasticsearch instance, so I've taken down the Vercel deployment. See examples below.
Below are some screenshots of CLIP Embedding similarity results.
Embeddings were added to Elasticsearch as dense vectors:
image: {
...
embedding: {
type: 'dense_vector',
dims: 512,
index: true,
similarity: 'cosine',
},
...
}Example cosine similarity query:
const esQuery: T.SearchRequest = {
index: 'collections',
query: {
bool: {
must: [
{ exists: { field: 'image.embedding' } },
{
script_score: {
query: { match_all: {} },
script: {
source:
"cosineSimilarity(params.query_vector, 'image.embedding') + 1.0",
params: { query_vector: input_vector },
},
},
},
],
must_not: {
term: {
id: document.id,
},
},
},
},
from: 0,
size: SIMILAR_PAGE_SIZE,
};
