Discussion on Performance and Scalability for Multiple Term Search
TLDR Bill asks the best way for multi-term searches in a recommendation system they developed. Kishore Nallan suggested using embeddings and remote embedder or storing and averaging vectors. Despite testing several suggested solutions, Bill continued to face performance issues, leading to unresolved discussions about scalability and recommendation system performance.
3
Nov 16, 2023 (2 weeks ago)
Bill
03:40 PMBill
03:41 PMNov 17, 2023 (2 weeks ago)
Kishore Nallan
03:07 AMBill
10:18 AMKishore Nallan
10:18 AMid -> embedding
and look them up right.Bill
10:20 AMBill
10:21 AMBill
10:22 AMKishore Nallan
10:22 AMKishore Nallan
10:22 AMBill
10:23 AM1
Bill
10:27 AMKishore Nallan
10:33 AMBill
10:36 AMKishore Nallan
10:37 AMBill
10:38 AMKishore Nallan
10:38 AMBill
10:38 AMBill
10:38 AMKishore Nallan
10:39 AMBill
10:39 AMKishore Nallan
10:41 AMKishore Nallan
10:42 AMBill
10:43 AMKishore Nallan
10:43 AMBill
10:44 AMKishore Nallan
10:46 AMKishore Nallan
10:46 AMBill
10:48 AMKishore Nallan
10:53 AM[a1, a2, a3]
and [b1, b2, b3]
average vector is: [a1+b1/2, a2+b2/2, a3+b3/2]
1
Bill
10:54 AMBill
10:58 AMKishore Nallan
11:01 AMBill
11:41 AMBill
01:27 PMBill
01:28 PM{
"searches": [
{
"collection": "test",
"q": "*",
"per_page":200,
"include_fields": "test",
"vector_query": "embedding:([-0.1110641211271286,0.757041597738862,-0.098668213468045,0.46491856407374144,0.1574960257858038,.......-1.0089502930641174], k:50)"
}
]
}
Bill
01:31 PMKishore Nallan
01:41 PMBill
01:43 PMBill
01:43 PMBill
01:44 PMBill
01:44 PMKishore Nallan
01:44 PMBill
01:45 PMKishore Nallan
01:45 PMBill
01:46 PMKishore Nallan
01:46 PMper_page
will override k
btw so you are fetching 200 records which might be slow.Bill
01:46 PMKishore Nallan
01:46 PMBill
01:48 PMBill
01:48 PMBill
01:49 PMBill
05:32 PMNov 18, 2023 (2 weeks ago)
Bill
01:09 PMKishore Nallan
01:15 PMBill
01:28 PMKishore Nallan
01:30 PMKishore Nallan
01:31 PMKishore Nallan
01:32 PMBill
01:32 PMKishore Nallan
02:25 PMKishore Nallan
02:25 PMBill
02:34 PMKishore Nallan
02:54 PMBill
02:56 PMBill
02:57 PMKishore Nallan
03:06 PMKishore Nallan
03:07 PMBill
03:07 PMKishore Nallan
03:08 PMBill
03:14 PMKishore Nallan
03:27 PMI'll have to look.
Bill
03:32 PMBill
06:34 PMBill
06:46 PMBill
06:47 PMNov 19, 2023 (1 week ago)
Jason
02:36 PMBill
02:38 PMJason
02:39 PMBill
02:42 PMNov 22, 2023 (1 week ago)
Bill
12:21 PMKishore Nallan
12:58 PM{ "transactions": 22439,
"availability": 100.00,
"elapsed_time": 19.42,
"data_transferred": 17.44,
"response_time": 0.04,
"transaction_rate": 1155.46,
"throughput": 0.90,
"concurrency": 49.84,
"successful_transactions": 22439,
"failed_transactions": 0,
"longest_transaction": 0.14,
"shortest_transaction": 0.01
}
Command used: https://gist.github.com/kishorenc/4be95e9242bd4b183d5cf74f7eae202c
Can you give this a shot locally so we have a quicker common reference point?
Bill
01:01 PMKishore Nallan
01:02 PMKishore Nallan
01:03 PMBill
01:03 PMKishore Nallan
01:04 PMBill
01:04 PMKishore Nallan
01:05 PMBill
01:05 PMKishore Nallan
01:05 PMKishore Nallan
01:06 PMI don't know, but can you try with the siege command above? You can run it from another instance if you want.
Bill
01:06 PMBill
01:07 PMBill
01:07 PMKishore Nallan
01:07 PMKishore Nallan
01:08 PMBill
01:08 PMKishore Nallan
01:09 PMBill
01:09 PMTypesense
Indexed 3015 threads (79% resolved)
Similar Threads
Enhancing Vector Search Performance and Response Time using Multi-Search Feature
Bill faced performance issues with vector search using multi_search feature. Jason and Kishore Nallan suggested running models on a GPU and excluding large fields from the search. Through discussion, it was established that adding more CPUs and enabling server-side caching could enhance performance. The thread concluded with the user reaching a resolution.
Utilizing Vector Search and Word Embeddings for Comprehensive Search in Typesense
Bill sought clarification on using vector search with multiple word embeddings in Typesense and using them instead of OpenAI's embedding. Kishore Nallan and Jason informed him that their development version 0.25 supports open source embedding models. They also resolved Bill's concerns regarding search performance, language support, and limitations in the search parameters.
Integrating Semantic Search with Typesense
Krish wants to integrate a semantic search functionality with typesense but struggles with the limitations. Kishore Nallan provides resources, clarifications and workarounds to the raised issues.
Implementing Semantic Search with Typesense
Erik sought advice for semantic search implementation in Typesense and raised issues around slow document import and excessive latency. Upon implementing advice from Kishore Nallan to try different models, Erik reported faster times, ultimately deciding to rate-limit imports.
Optimizing Typesense Implementation for Large Collections
Oskar faced performance issues with his document collection in Typesense due to filter additions. Jason suggested trying a newer Typesense build and potentially partitioning the data into country-wise collections. They also discussed reducing network latency with CDN solutions.