#community-help

Solving Typesense Cloud Interface Issue in Python

TLDR Orion had an issue querying from Python in the Typesense Cloud interface. Kishore Nallan identified the problem within the array declaration. They then discussed improving error messages and Orion offered to make a PR.

11mo

Join the chat

Oct 14, 2022 (11 months ago)

Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c

Orion

10:38 AM

Hey all, I’m encountering an issue querying from Python that I don’t understand. If I do a search through the Typesense Cloud interface with query_by: ['title', 'text'] I’ll get hits from both titles and text fields, but the same query from Python only gets me matches with 'text' (or 'title' if I switch the order).
Any idea what could be causing this?

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

10:39 AM

This is likely because the fields must be specified as a comma separated string in your client and not as an array of strings.

Orion

10:40 AM

Oh my god

10:44

Orion

10:44 AM

That works, thank you so much! On closer reading I see it is actually documented:

> One or more string / string[] fields that should be queried against. Separate multiple fields with a comma: company_name, country
I think I got thrown by the string / string[] part and made a wrong assumption.

Is it expected behaviour to treat an array as a valid input?

Kishore Nallan

10:51 AM

Most of the Typesense clients are a light wrapper around the underlying HTTP API so the same conventions hold.

10:52

Kishore Nallan

10:52 AM

Maybe we should have some kind of error when an array is used, purely from a usability perspective.

Orion

10:53 AM

Yeah I think that would be useful for sure.

Kishore Nallan

10:54 AM

Happy to accept a PR if that's simple enough to add

Orion

10:54 AM

Either that or stringify the array, but that feels less inline with the light wrapper design (which I like)

Kishore Nallan

10:54 AM

I was just about to type the same

Orion

10:54 AM

I’ll create a PR

Kishore Nallan

10:54 AM

Thank you!

Orion

10:58 AM

Do you think an error is better than stringification?

Kishore Nallan

10:59 AM

Yes I think an error is probably better.

10:59

Kishore Nallan

10:59 AM

For e.g. num_typos can again be comma separated but here, if given as an array of integers, it cannot be joined as string without doing a conversion.

Orion

11:06 AM

Should I create a new exception? There’s RequestMalformed but that doesn’t seem quite right… Looking around I don’t see anywhere doing query validation. Maybe I should create a dirt simple validation step for search_parameters ?

Kishore Nallan

11:16 AM

We can create a InvalidParameter error or something here: https://github.com/typesense/typesense-python/blob/master/typesense/exceptions.py

Orion

11:16 AM

Yeah that’s my thinking

11:32

Orion

11:32 AM

Okay, I have something simple set up. Haven’t tested it, if I open the PR is that easy enough to do on your end?

Kishore Nallan

11:45 AM

Yes thanks

Orion

11:45 AM

Here’s the PR

11:45

Orion

11:45 AM

Happy to make tweaks.

11:46

Orion

11:46 AM

There may also be other sensible validation stuff worth adding, but nothing comes to mind off the top of my head.

11:50

Orion

11:50 AM

Probably worth generalising… As all fields are strings, should it just check/validate all parameters?

Kishore Nallan

11:51 AM

Typesense search API began as a GET + query param API. So when we moved to the /mutli_search POST API we just went with the same same format but as JSON body.

11:51

Kishore Nallan

11:51 AM

So yes, all fields should be strings

Orion

11:53 AM

Okay cool, I can make that change

12:00

Orion

12:00 PM

Done

12:10

Orion

12:10 PM

I imagine there’s an expectation for basic type casting (bools, ints, etc) but as I understand it this isn’t current behaviour.

I don’t think this PR interferes with any existing behaviour like that right?

Maybe a separate PR for basic casting would be nice, could improve ergonomics and match more peoples expectations?

Kishore Nallan

12:32 PM

Can you give me an example of what you mean by casting?

Orion

12:44 PM

E.g. "pre_segmented_query": True -> "pre_segmented_query":"true"

Kishore Nallan

12:57 PM

Ah, same as: https://github.com/typesense/typesense-python/issues/20

Orion

01:14 PM

^ yeah

Kishore Nallan

01:17 PM

I think boolean can be processed as string. That would be a easy convenient fix.

01:17

Kishore Nallan

01:17 PM

Unlike with arrays, there are no gotchas there with multiple types

Orion

01:19 PM

Yeah booleans and ints are the main ones I was thinking of. Could just be a simple preprocessing step before validation, then the validator catches anything else.

01:20

Orion

01:20 PM

Would probably be enough to just convert any booleans and ints to lowercase strings. Should I add that to the existing PR or make a new one?

Kishore Nallan

01:26 PM

Yes, please that will be good

Orion

01:43 PM

Done

Kishore Nallan

01:52 PM

Thank you! I will be reviewing it later in the evening or tomorrow.

Oct 15, 2022 (11 months ago)

Kishore Nallan

01:43 PM

Merged, thank you!

Orion

01:58 PM

Let me know if there’s other bits that need doing!

Kishore Nallan

02:01 PM

On the subject of types, there was also an ask to treat None field values as null (I think) : https://github.com/typesense/typesense-python/issues/14

Orion

02:10 PM

Yeah I saw that, not entirely sure the intended use case. Wish the issue was more detailed

02:11

Orion

02:11 PM

I think your assumption makes sense

02:12

Orion

02:12 PM

Is that something we want?

Kishore Nallan

02:12 PM

I haven't checked it yet but seems like having None as a value is not being accepted? But I would think that it will be parsed as null during json conversion. I'll ask for further details.

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2764 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Sorting Results in Typesense and Handling Errors

Ramees asks about sorting results by distance in Typesense, entering a list of strings as a field, and resolving an error with `fields` format. Kishore Nallan assists with these issues and advises on storing timestamps and proper authentication.

26mo

Querying with Typesense-Js and Handling Null Values

michtio was querying using typesense-js and receiving fewer results than expected. Kishore Nallan suggested using different query parameters. Further discussion led to the handling of 'null' values and filtering syntax in the search queries. The thread ended with Jason offering migration support from Algolia to Typesense.

14mo

Understanding and Implementing Typesense Dart Library with Flutter

Alexandro sought help with the Typesense Dart library. Jason explained that the library is in progress, discussed utilizing other HTTP libraries, and provided detailed instructions on utilizing Typesense with Flutter. Alexandro provided feedback on the Typesense UI and expressed interest in creating a tutorial video.

28mo

Cold Start Problem with Dynamic Collections

Adrian reported cold start issues with dynamic collections. Jason suggested using wildcard `*` for query_by parameters, upgrading to `0.25.0.rc34`, and clarified conventions. Adrian's issues were resolved but they reported a limitation that will potentially be addressed.

4mo

Issue with Typesense Schema and Ruby Client

Mateo faces issues while creating a schema and using Ruby client for Typesense. Jason suggests using a new field instead of 'id' and provides assistance for Ruby client errors.

7mo