#community-help

Solving Searchable String ID Sorting Issue

TLDR Mubashirullah needed to sort by string id, which was not supported. Kishore Nallan offered a preview Docker build with the feature. Mubashirullah expressed satisfaction with the solution.

Powered by Struct AI

1

1

1

12
25mo
Solved
Join the chat
Dec 23, 2021 (25 months ago)
Mubashirullah
Photo of md5-cea7a558edb0c66a1c07dfdcf00bc141
Mubashirullah
07:58 AM
Hi everyone.
I have this situation where I need to add id as searchable_id string since the client needs to search by it. But then, I have to add the id again as sort_id int so I can sort by it. I wish I could use my normal id for searching as well
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
07:58 AM
Is your id integer or string type?
Mubashirullah
Photo of md5-cea7a558edb0c66a1c07dfdcf00bc141
Mubashirullah
08:46 AM
The original is a string
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
09:11 AM
We don't support sorting by string, but that feature is almost ready for preview.
Mubashirullah
Photo of md5-cea7a558edb0c66a1c07dfdcf00bc141
Mubashirullah
09:19 AM
🙂 Looking forward to it. Otherwise, I'd have to convert every string into an int and that would really bloat my memory.
Dec 24, 2021 (25 months ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:28 PM
Mubashirullah I've a preview Docker build for the string sorting feature.
Mubashirullah
Photo of md5-cea7a558edb0c66a1c07dfdcf00bc141
Mubashirullah
01:29 PM
Great news! My manager will be happy to hear this.
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:29 PM
You can check against typesense/typesense:0.23.0.rc10

You should explicitly enable string sorting at a field level like this:

{"name": "title", "type": "string", "sort": true }

You can now include the title field in the sort_by clause.

1

1

01:30
Kishore Nallan
01:30 PM
Caveat: enabling string sorting will require more memory because as Typesense is an in-memory engine, the strings have to be now stored in-memory.
01:31
Kishore Nallan
01:31 PM
Compared with the text index, which is an inverted index of word -> document IDs, a string sorting datastructure has to store the verbatim strings, so there is little de-duplication so will consume more memory. Best to be used for small strings (anyway sorting large strings don't make much sense).
Mubashirullah
Photo of md5-cea7a558edb0c66a1c07dfdcf00bc141
Mubashirullah
01:34 PM
🙂❤️🎉
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:40 PM
Let me know your feedback if you test it out 🙂

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Adjusting Text Match Score Calculation in TypeSense

Johannes wanted to modify the Text Match Score calculation in TypeSense to improve search results returns. With counsel from Jason and Kishore Nallan, various solutions were proposed, including creating a Github issue, attempting different parameters, and updating Docker to a new version to resolve the matter.

3

48
19mo

Improving Typesense Query Performance

Jonathan queried about slower than expected typesense query performance. Jason and Kishore Nallan offered solutions and explanations. After a series of tests, Jonathan found other queries returned results quickly, indicating the issue was specific to the original query.

3

26
13mo
Solved

Troubleshooting Typesense Document Import Error

Christopher had trouble importing 2.1M documents into Typesense due to memory errors. Jason clarified the system requirements, explaining the correlation between RAM and dataset size, and ways to tackle the issue. They both also discussed database-like query options.

3

30
10mo
Solved

Methods for Fetching, Querying, and Modifying Collections in Typesense

Bill inquired about performing OR queries, querying empty arrays and modifying collections in Typesense. Kishore Nallan explained the current limitations and provided workarounds and recommendations for each case. The conversation also touched upon the usage of cache in Typesense and the workings of the _eval function.

5

52
10mo
Solved

Discussing Ability to Sort by a Text Field

Aljosa asks about sorting by text fields and proposes a workaround. Kishore Nallan discusses database implications and possible workarounds, informing that changes may require reindexing. Andrew expresses interest in string sorting.

23
27mo
Solved