#community-help

Issue with Optional Filtering and Id Field

TLDR Stefan encounters a bug with optional filtering on the id field, where _eval sort criteria doesn't show expected results. Kishore Nallan confirms it might be a bug. An issue is created on GitHub.

9mo

Join the chat

Mar 13, 2023 (9 months ago)

Photo of md5-d6c265b4792dbf0a1d6ae378f39d8736

Stefan

01:51 PM

Hi there,
I am having an issue with the new optional filtering, I would expect that if I put the "_eval" sort criteria first that documents that match this expressions are shown first no matter the other sort criteria, but that doesn't seem to be the case:

---------- Search parameters
{'q': '*', 'query_by': 'name', 'sort_by': '_eval(id:1):asc,pop:desc'}
---------- Query output
{'facet_counts': [],
 'found': 2,
 'hits': [{'document': {'id': '0', 'name': 'a', 'pop': 1},
           'highlight': {},
           'highlights': []},
          {'document': {'id': '1', 'name': 'b', 'pop': 0},
           'highlight': {},
           'highlights': []}],
 'out_of': 2,
 'page': 1,
 'request_params': {'collection_name': 'test_collection',
                    'per_page': 10,
                    'q': '*'},
 'search_cutoff': False,
 'search_time_ms': 0}

So in the case above I'd expect name: 'b' to be the first document, because it matches the _eval() condition, and 'a' the second, but that doesn't seem to be the case.

Am I misunderstanding the feature? Changing _eval(xxx):asc to desc doesn't change the output.

I made a little reproduction repo so you can see the exact configuration: https://github.com/reinoldus/typesense-reproduction-repo

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

01:53 PM

Is this a problem only for id field? Or are other fields affected as well?

Stefan

01:56 PM

let me check

01:58

Stefan

01:58 PM

Seems to be only related to the id field:

---------- Search parameters
{'q': '*',
 'query_by': 'name',
 'sort_by': '_eval(other_field:test):desc,pop:desc'}
---------- Query output
{'facet_counts': [],
 'found': 2,
 'hits': [{'document': {'id': '1',
                        'name': 'b',
                        'other_field': 'test',
                        'pop': 0},
           'highlight': {},
           'highlights': []},
          {'document': {'id': '0',
                        'name': 'a',
                        'other_field': 'not_test',
                        'pop': 1},
           'highlight': {},
           'highlights': []}],
 'out_of': 2,
 'page': 1,
 'request_params': {'collection_name': 'test_collection',
                    'per_page': 10,
                    'q': '*'},
 'search_cutoff': False,
 'search_time_ms': 0}

Kishore Nallan

01:59 PM

Yup so there's probably a bug there specific to id

Stefan

02:06 PM

Shall I create an issue?

Kishore Nallan

02:19 PM

Yes please

Stefan

02:24 PM

done: https://github.com/typesense/typesense/issues/936

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Fixing Multiple Document Retrieval in Typesense

Phil needed an efficient way to retrieve multiple documents by id. Kishore Nallan proposed a solution available in a pre-release build. After some bug fixing regarding id matching by Jason and Kishore Nallan, Phil successfully tested the solution.

26mo

Methods for Fetching, Querying, and Modifying Collections in Typesense

Bill inquired about performing OR queries, querying empty arrays and modifying collections in Typesense. Kishore Nallan explained the current limitations and provided workarounds and recommendations for each case. The conversation also touched upon the usage of cache in Typesense and the workings of the _eval function.

10mo

Resolving Search Errors in RC Branch using Python Typesense

Aadarsh was encountering an error while testing a RC Branch using Python Typesense. With advice and debugging from Kishore Nallan and Harpreet, the issue was resolved by fixing malformed search parameters and ensuring reference fields match in referenced documents.

1mo

Threading Problem During Multiple Collection Creation and Batch Insertion in Typesense

Johan has a problem with creating multiple collections and batch-inserting documents into Typesense, which is returning results from different collections. Kishore Nallan helps troubleshoot the issue and suggests a potential local race condition, which is fixed in a later build.

17mo

Issue with Embedding Error in Version 0.25.0.rc63

Bill reported a bug in version 0.25.0.rc63 regarding a problem with updating or emplacing a document and receiving an embedding error. This was resolved in version 0.25.0.rc65, but further discussion ensued regarding the function of 'index' in the update feature.

4mo