#community-help

Solving Typesense Cloud Interface Issue in Python

TLDR Orion had an issue querying from Python in the Typesense Cloud interface. Kishore Nallan identified the problem within the array declaration. They then discussed improving error messages and Orion offered to make a PR.

Powered by Struct AI
raised_hands3
+12
1001
47
11mo
Solved
Join the chat
Oct 14, 2022 (11 months ago)
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
10:38 AM
Hey all, I’m encountering an issue querying from Python that I don’t understand. If I do a search through the Typesense Cloud interface with query_by: ['title', 'text'] I’ll get hits from both titles and text fields, but the same query from Python only gets me matches with 'text' (or 'title' if I switch the order).
Any idea what could be causing this?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
10:39 AM
This is likely because the fields must be specified as a comma separated string in your client and not as an array of strings.
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
10:40 AM
Oh my god
10:44
Orion
10:44 AM
That works, thank you so much! On closer reading I see it is actually documented:

> One or more string / string[] fields that should be queried against. Separate multiple fields with a comma: company_name, country
I think I got thrown by the string / string[] part and made a wrong assumption.

Is it expected behaviour to treat an array as a valid input?
raised_hands1
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
10:51 AM
Most of the Typesense clients are a light wrapper around the underlying HTTP API so the same conventions hold.
10:52
Kishore Nallan
10:52 AM
Maybe we should have some kind of error when an array is used, purely from a usability perspective.
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
10:53 AM
Yeah I think that would be useful for sure.
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
10:54 AM
Happy to accept a PR if that's simple enough to add
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
10:54 AM
Either that or stringify the array, but that feels less inline with the light wrapper design (which I like)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
10:54 AM
I was just about to type the same
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
10:54 AM
I’ll create a PR
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
10:54 AM
Thank you!
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
10:58 AM
Do you think an error is better than stringification?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
10:59 AM
Yes I think an error is probably better.
10:59
Kishore Nallan
10:59 AM
For e.g. num_typos can again be comma separated but here, if given as an array of integers, it cannot be joined as string without doing a conversion.
1001
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
11:06 AM
Should I create a new exception? There’s RequestMalformed but that doesn’t seem quite right… Looking around I don’t see anywhere doing query validation. Maybe I should create a dirt simple validation step for search_parameters ?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:16 AM
We can create a InvalidParameter error or something here: https://github.com/typesense/typesense-python/blob/master/typesense/exceptions.py
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
11:16 AM
Yeah that’s my thinking
+11
11:32
Orion
11:32 AM
Okay, I have something simple set up. Haven’t tested it, if I open the PR is that easy enough to do on your end?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:45 AM
Yes thanks
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
11:45 AM
11:45
Orion
11:45 AM
Happy to make tweaks.
11:46
Orion
11:46 AM
There may also be other sensible validation stuff worth adding, but nothing comes to mind off the top of my head.
11:50
Orion
11:50 AM
Probably worth generalising… As all fields are strings, should it just check/validate all parameters?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:51 AM
Typesense search API began as a GET + query param API. So when we moved to the /mutli_search POST API we just went with the same same format but as JSON body.
11:51
Kishore Nallan
11:51 AM
So yes, all fields should be strings
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
11:53 AM
Okay cool, I can make that change
12:00
Orion
12:00 PM
Done
12:10
Orion
12:10 PM
I imagine there’s an expectation for basic type casting (bools, ints, etc) but as I understand it this isn’t current behaviour.

I don’t think this PR interferes with any existing behaviour like that right?

Maybe a separate PR for basic casting would be nice, could improve ergonomics and match more peoples expectations?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
12:32 PM
Can you give me an example of what you mean by casting?
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
12:44 PM
E.g. "pre_segmented_query": True -> "pre_segmented_query":"true"
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
01:14 PM
^ yeah
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:17 PM
I think boolean can be processed as string. That would be a easy convenient fix.
01:17
Kishore Nallan
01:17 PM
Unlike with arrays, there are no gotchas there with multiple types
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
01:19 PM
Yeah booleans and ints are the main ones I was thinking of. Could just be a simple preprocessing step before validation, then the validator catches anything else.
01:20
Orion
01:20 PM
Would probably be enough to just convert any booleans and ints to lowercase strings. Should I add that to the existing PR or make a new one?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:26 PM
Yes, please that will be good
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
01:43 PM
Done
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:52 PM
Thank you! I will be reviewing it later in the evening or tomorrow.
raised_hands1
Oct 15, 2022 (11 months ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:43 PM
Merged, thank you!
raised_hands1
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
01:58 PM
Let me know if there’s other bits that need doing!
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
02:01 PM
On the subject of types, there was also an ask to treat None field values as null (I think) : https://github.com/typesense/typesense-python/issues/14
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
02:10 PM
Yeah I saw that, not entirely sure the intended use case. Wish the issue was more detailed
02:11
Orion
02:11 PM
I think your assumption makes sense
02:12
Orion
02:12 PM
Is that something we want?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
02:12 PM
I haven't checked it yet but seems like having None as a value is not being accepted? But I would think that it will be parsed as null during json conversion. I'll ask for further details.
+11