#community-help

Querying and Indexing Multiple Elements Issues

TLDR Krish queried fields with multiple elements, which Kishore Nallan suggested checking drop_tokens_threshold. Krish wished to force OR mode for token, but Kishore Nallan admitted the feature was missing. Krish was able to resolve the issue with url encoding.

12mo

Join the chat

Dec 15, 2022 (12 months ago)

Photo of md5-7dff5651efcba5c9cdfac93918dc7eeb

Krish

06:49 AM

Hey a question: My query field is something like: fieldName: ["one", "two", "three"]. How could I query this by sending a query like so "one two"? Not sure if I explained it properly.

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

07:19 AM

"one two" will not match 2 distinct elements in the array unless dropping of tokens kick in (check on drop_tokens_threshold)

Krish

08:09 AM

Thank you. What should be the value of drop tokens threshold? I mean how do I decide the value?

Kishore Nallan

08:19 AM

It depends on your dataset. Ideally we should have a way to force an OR mode for the tokens which we don't have now. It's on our backlog though.

08:20

Kishore Nallan

08:20 AM

If you set a very high value for drop_tokens_threshold then it will more or less try to drop tokens for 2 token queries.

08:21

Kishore Nallan

08:21 AM

But it won't really work for 3+ token queries. For that we need OR mode.

Krish

08:23 AM

OK... May be you could help. My use case is I have a bunch of records aka documents indexed. It has a field that's an array of words. And when the user types in a bunch of words I expect it to search the words in that field and return appropriately.

08:23

Krish

08:23 AM

How could I achieve something like this?

Kishore Nallan

08:25 AM

Squash the array into a single string and index it.

Krish

08:26 AM

string separetd by space or just one string with all elements concatenated?

Kishore Nallan

08:26 AM

["foo bar", "baz"] -> "foo bar baz"

Krish

08:27 AM

OK...

08:27

Krish

08:27 AM

But I thought this wouldn't work because it looks only the prefix of the string..

08:28

Krish

08:28 AM

If I look for baz would it still find it?

Kishore Nallan

08:28 AM

Yes.

Krish

08:28 AM

OK... Let me give that a try.

08:30

Krish

08:30 AM

Btw, a different question, is there a way I could do the instant search on the browser without having to use server side code?

Kishore Nallan

08:47 AM

Yes: https://github.com/typesense/typesense-instantsearch-adapter

Krish

12:11 PM

Hey Kishore Nallan I have setup the schema the way you mentioned above, but the call doesn't fetch anything when I query for something like "foo bar". Could you assist?

Kishore Nallan

12:17 PM

Need a reproduceable example to see what's happening.

Krish

12:20 PM

roger@W102TS3793:~/searchapp/typesense-server$ curl -H "X-TYPESENSE-API-KEY: RnVjazFuZzRzc2gwbDMh" "<http://localhost:3000/search?q=avi>"

{"facet_counts":[],"found":1,"hits":[{"document":{"end":193626,"id":"3124498306","keywords":"avio community driven monetization platform open source projects connect maintainers developers businesses tons typesense hope scotch give shot","start":112130,"title":"Build React Full-Text Instant Search App with Typesense - 10,000 Movies database","url":"<https://example.com>"},"highlights":[{"field":"keywords","matched_tokens":["avi"],"snippet":"&lt;mark&gt;avi&lt;/mark&gt;o community driven monetization platform open source projects connect maintainers developers businesses tons typesense hope scotch give shot"}],"text_match":72341261125681150}],"out_of":6,"page":1,"request_params":{"collection_name":"videos","per_page":10,"q":"avi"},"search_cutoff":false,"search_time_ms":0}

12:20

Krish

12:20 PM

Example that works, and below is the example that doesn't

12:20

Krish

12:20 PM

roger@W102TS3793:~/searchapp/typesense-server$ curl -H "X-TYPESENSE-API-KEY: RnVjazFuZzRzc2gwbDMh" "<http://localhost:3000/search?q=avio> typesense"

roger@W102TS3793:~/searchapp/typesense-server$

12:22

Krish

12:22 PM

the query by field is Keywords. If you look at the example that worked th efield keywords has both the words searched for

"keywords":"avio community driven monetization platform open source projects connect maintainers developers businesses tons typesense hope scotch give shot"

Kishore Nallan

12:26 PM

<http://localhost:3000/search?q=avi> is proxying Typesense? I need the actual query made to Typesense.

12:28

Kishore Nallan

12:28 PM

Indexed a single document and tried querying directly (check the URL) and it works fine.

Krish

12:29 PM

i have a node client written to make the calls...like so

const videoSchema = {
'name': 'videos',
'fields': [
{ 'name': 'id', 'type': 'string', 'facet': false },
{ 'name': 'title', 'type': 'string', 'facet': true },
{ 'name': 'keywords', 'type': 'string', 'facet': false },
{ 'name': 'url', 'type': 'string', 'facet': true },
{ 'name': 'start', 'type': 'int32', 'facet': true },
{ 'name': 'end', 'type': 'int32', 'facet': true }
],
//'default_sorting_field': 'title'
}
if (typesense.collections('videos').exists()) {
const c = typesense.collections('videos').retrieve()
console.log('Exisitng collection found')
console.log('No. of records : ', (await c).num_documents)
return await typesense.collections('videos').documents().create(item)

}

And th eapi is invoked like so

app.get('/search', async(req, res) => {

const { q } = req.query;

try {
const searchResults = await lookup(q)
console.log('Search Results : ', searchResults)
res.send(searchResults)

} catch (err) {
console.error(err)
res.send(err)
}
});

12:30

Krish

12:30 PM

interesting

Kishore Nallan

12:31 PM

See this example and see if you can reproduce via curl: https://gist.github.com/kishorenc/0cf9b6b71e2993cb8cd10f44dea3e944

Krish

12:31 PM

12:41

Krish

12:41 PM

That worked.. Also figured that url encoding the separator between words resolves the issue

12:43

Krish

12:43 PM

so urlencoding special chars is needed, isn't it?

01:06

Krish

01:06 PM

Also, I am guessing their isn't a way to do an OR here right?

Kishore Nallan

01:09 PM

Yeah OR is missing feature.

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3005 threads (79% resolved)

Join Our Community

Similar Threads

Phrase Search Relevancy and Weights Fix

Jan reported an issue with phrase search relevancy using Typesense Instantsearch Adapter. The problem occurred when searching phrases with double quotes. The team identified the issue to be related to weights and implemented a fix, improving the search results.

111

8mo

Troubleshooting Typesense Setup and Understanding Facets and Keywords

Demitri encountered errors when exploring Typesense for the first time. Jason guided them through troubleshooting and discussed facets, keyword settings, and widget configurations. Helin shared a Python demo app and its source code to help Demitri with their project.

21mo

Querying with Not-in in Typesense

Masahiro inquired about using not-in queries in Typesense. Kishore Nallan explained how to conduct such queries by using the "-" operator in the query string, and assisted Masahiro with issues stemming from a high number of exclusion tokens. The problem was eventually resolved by switching to the `multi_search` endpoint.

33mo

Troubleshooting Issues with DocSearch Hits and Scraper Configuration

Rubai encountered issues with search result priorities and ellipsis. Jason helped debug the issue and suggested using different versions of typesense-docsearch.js, updating initialization parameters, and running the scraper on a Linux-based environment. The issues related to hits structure and scraper configuration were resolved.

131

8mo

Sorting Results in Typesense and Handling Errors

Ramees asks about sorting results by distance in Typesense, entering a list of strings as a field, and resolving an error with `fields` format. Kishore Nallan assists with these issues and advises on storing timestamps and proper authentication.

30mo