Document Weighting and Sorting Discussion
TLDR SamHendley asked how to weight a document based on age and offered a sorting method. Jason clarified the method and suggested an RC for use. SamHendley tested and confirmed the solution. User Kishore Nallan assisted when an error occurred with a new feature.
1
Dec 02, 2022 (13 months ago)
SamHendley
08:16 PMSamHendley
08:18 PMtext_match(buckets: 10):desc
or similar to make sure I’m only looking at relavent docs first then use a sorting based on a date fieldJason
08:18 PMJason
08:19 PMSamHendley
08:19 PMJason
08:20 PM0.24.0.rc37
0.24.0.rcn37
is the latest RC btwSamHendley
08:20 PMSamHendley
08:21 PMJason
08:21 PMSamHendley
08:22 PMSamHendley
08:24 PM0.24.0.rcn37
Jason
08:24 PMJason
08:24 PMSamHendley
08:25 PMJason
08:25 PMJason
08:26 PMSamHendley
08:27 PMJason
08:29 PMSamHendley
08:32 PM_eval(recent_activity>now-90),_text_match(buckets: 10):desc,recent_activity(buckets:5):desc,_text_match:desc
• First things that are within last 90 days
• then “relevant” (but not strict)
• then “recent” (but not strict)
• then by text best text match
SamHendley
08:34 PMbuckets
the only way to group? I can’t pass in a constant divider? (ie in my case I might do 86400 seconds to make all documents on same day have same score)Jason
08:36 PMJason
08:36 PMNo, you would have to do this at indexing time
SamHendley
08:36 PMJason
08:37 PMJason
08:38 PMSamHendley
08:38 PM1
Dec 30, 2022 (12 months ago)
SamHendley
02:25 PM_eval
feature fell out of the more recent RCs? I’m on rcn47
and making what I think is a valid sort_by
setting and getting an error that feels like it isn’t parsing correctly.Setting:
"sort_by":"_eval(recent_activity_raw:1664634102),_text_match:desc,recent_activity_raw:desc"
Error:
{"message": "Could not find a field named
_eval(recent_activity_raw in the schema for sorting."}
Kishore Nallan
03:12 PMSamHendley
03:17 PMSamHendley
04:01 PM_eval(recent_activity_raw:1664637399):desc
or :asc
From Kishore Nallan in DM. My response :man-facepalming:
Typesense
Indexed 3015 threads (79% resolved)
Similar Threads
Using Google's Embeddings for Document Embeddings
Bill inquired about using Google's embeddings for document embeddings, and Jason confirmed that it is possible.
Full Text Search Across Long Books: Chunking vs. Single Document
Epi asked about providing FTS for long books. Kishore Nallan suggested breaking the books into chunks for better performance and query results.
Overrides/Curation Query and Filtering Docs
Jameshwart sought for filtering assistance, and Kishore Nallan suggested checking the overrides/curation section in the docs.
Typesense Support for Embedding Documents
Jacob asked if Typesense supports embedding documents for their specific use case. Kishore Nallan confirmed that this feature is high on their priority list.
Using Joins Without Filtering Results in Document Retrieval
Ahmed inquired whether joins could be used without filtering results, but needs to provide the schema of collections and a sample query as suggested by Harpreet.