Optimizing Schema and Reducing Memory Usage
TLDR Shaun faced issues with memory usage when adding a new field to a schema. Kishore Nallan advised optimizing the schema by disabling unnecessary facets and sorting options to reduce memory usage.
Jun 20, 2023 (5 months ago)
Shaun
07:27 AM
Kishore Nallan
07:28 AMShaun
07:33 AMShaun
07:33 AMShaun
07:33 AMShaun
07:34 AMShaun
07:34 AMKishore Nallan
07:34 AMShaun
07:35 AMKishore Nallan
07:38 AMShaun
07:40 AMShaun
07:41 AMShaun
07:41 AMShaun
07:41 AMShaun
07:41 AMKishore Nallan
07:41 AMShaun
07:42 AMShaun
07:42 AMuserId if we just want to do exact match on that ID being found, we would need to have facet:true on it? Or facet:false will still be ok ?Kishore Nallan
07:44 AMKishore Nallan
07:44 AMShaun
07:45 AMShaun
07:45 AMShaun
07:45 AMfacet:true for like verified and deleted flagsShaun
07:46 AMShaun
07:46 AMShaun
07:46 AMdeleted=false etcKishore Nallan
07:49 AMShaun
07:49 AMShaun
07:49 AMKishore Nallan
07:50 AMShaun
07:50 AMShaun
07:50 AMShaun
07:50 AMShaun
07:50 AMShaun
07:55 AM {
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "deleted",
"optional": false,
"sort": false,
"type": "bool"
}Shaun
07:56 AMShaun
07:56 AMShaun
07:56 AMKishore Nallan
07:56 AMShaun
07:56 AMShaun
07:57 AMShaun
08:03 AMsort:true make much difference to memory usage? I take it that if this is false you cant sort by that field, except it seems to make it true by default for all fieldsKishore Nallan
08:04 AMKishore Nallan
08:04 AMShaun
08:04 AMShaun
08:04 AMShaun
08:04 AMShaun
08:04 AMKishore Nallan
08:04 AMShaun
08:05 AMShaun
08:07 AMKishore Nallan
08:19 AMShaun
08:19 AMShaun
08:19 AMShaun
08:19 AMJun 21, 2023 (5 months ago)
Gustavo
02:05 AMfacet: true then?https://typesense-community.slack.com/archives/C01P749MET0/p1687247364672419?thread_ts=1687246059.709859&cid=C01P749MET0
Typesense
Indexed 3015 threads (79% resolved)
Similar Threads
Discussions on Typesense, Collections, and Dynamic Fields
Tugay shares plans to use Typesense for their SaaS platform and asks about collection sizes and sharding. Jason clarifies Typesense's capabilities and shares a beta feature. They discuss using unique collections per customer and new improvements. Kishore Nallan and Gabe comment on threading and data protection respectively.
Adding New Fields to Items in Typesense Without Wiping Data
Alex wanted to add more fields to their Typesense items without wiping data. Kishore Nallan explained they had to create a new collection instead, and suggested using automatic schema detection for future changes. They also discussed potential downsides of indexing every field.
Handling Kinesis Stream Event Batching with Typesense
Dui had questions about how to handle Kinesis stream events with Typesense. Kishore Nallan suggested using upsert mode for creation/update and differentiating with logical deletion. After various discussions including identifying and resolving a bug, they finalized to introduce an `emplace` action in Typesense v0.23.
Issue with Embedding Error in Version 0.25.0.rc63
Bill reported a bug in version 0.25.0.rc63 regarding a problem with updating or emplacing a document and receiving an embedding error. This was resolved in version 0.25.0.rc65, but further discussion ensued regarding the function of 'index' in the update feature.
Utilizing Vector Search and Word Embeddings for Comprehensive Search in Typesense
Bill sought clarification on using vector search with multiple word embeddings in Typesense and using them instead of OpenAI's embedding. Kishore Nallan and Jason informed him that their development version 0.25 supports open source embedding models. They also resolved Bill's concerns regarding search performance, language support, and limitations in the search parameters.