#community-help

Complex Query System and Collections Schema for Typesense

TLDR Orion asks about the grammar for 'filter_by' search parameters and where to find a collections schema. Jason provides the needed documentation and encourages Orion to share back.

Powered by Struct AI

1

1

1

17
5mo
Solved
Join the chat
Jun 30, 2023 (5 months ago)
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
04:01 PM
I have two related questions:
Is there a formal grammar for filter_by or any of the search parameters? I am working on a complex query system which takes a query language, validates and parses it, then constructs typesense queries from the syntax tree. Doing this without knowing the precise filtering grammar (or other params) is quite a challenge. At the moment I've taken all the documented examples and tried to formalise them into PEG and BNF grammars but I would love to be more confident that it matches.

Second (much more mundane) question, is there any way to programmatically get a collections schema? I'm using the schema to inject some more intelligent behaviour into the query system so that things like filtering tags[] > 5 throws a useful syntax error in the frontend. I can copy-paste the schema when it changes but I would really love to avoid that.
04:12
Orion
04:12 PM
I would also love to know if there's some list of syntactically valid filters somewhere, at least that way I could build out a test suite for the filtering grammar
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:14 PM
We don’t have a formal grammar documented anywhere. But here’s all the possible options written out here under filter_by in this table
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
04:14 PM
Yeah that's the reference I've been using
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:15 PM
> is there any way to programmatically get a collections schema?
Yup. Using this GET endpoint

1

Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
04:15 PM
Any insight into how the filters are parsed? I couldn't find much searching the repo
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:16 PM
We tried a grammar-based codegen approach, but it became to unwieldy to manage the generated code, so we built our own light-weight parser
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
04:17 PM
Gotcha, is that parser public?
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
04:22 PM
Awesome, this is great reference, thanks!

1

Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
04:23 PM
Sweet
04:23
Orion
04:23 PM
For client libraries is there any existing support (or interest in) clientside query validation?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:23 PM
No, we validate only the server-side
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
04:26 PM
Cool, if I end up producing something interesting or useful I'll share it here, having realtime feedback for complex typesense queries has been really enjoyable for the project I'm working on, might be for others too.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:30 PM
That would be great! I’m really curious to see what you’re building!
Orion
Photo of md5-8e802b48c0369226a7b50a22ab6e9e0c
Orion
04:31 PM
Yeah I look forward to sharing! If deadlines are hit it will be a month, so expect me back here in 2!

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3005 threads (79% resolved)

Join Our Community

Similar Threads