#community-help

Handling Field Variations in Typesense for Different Clients

TLDR Yoshi asked for a solution to handle varying contact_date fields for different clients in typesense. Jason recommended using nested fields for the short-term and mentioned future support for joining data across collections.

Powered by Struct AI

1

18
7mo
Solved
Join the chat
May 04, 2023 (7 months ago)
Yoshi
Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d
Yoshi
11:46 PM
Hi everyone,
We have a collection of companies that we have included in typesense, specifically dentists. Each client has a different contact_date for each dentist.

I’m curious if there is any best practice for handling this kind of field that is different for each company, but wanting to filter/sort by it for each user.

One way I could potentially do this is by adding a different schema field per client:
{
  "name": "ABC Company",
  "location": "San Francisco",
  "latest_contact_date_client_1": "2023-05-04T10:30:00Z",
  "latest_contact_date_client_2": "2023-05-04T10:30:00Z",
  ...
}

But that seems terribly inefficient.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
11:51 PM
Is the end goal to let clients search for dentists and allow them to filter based on their last contact date for each dentist?
May 05, 2023 (7 months ago)
Yoshi
Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d
Yoshi
12:06 AM
exactly
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
12:08 AM
I would recommend creating one main collection with all dentists… And then when a client contacts a dentist, create a new user-specific collection called say dentists_contacted_by_client_1 and add that dentist record into that collection.

Then at search time, you can do a request and fetch results from the user-specific collection sorted by contact time, then do another request to the main collection excluding the IDs for in the first result set
Yoshi
Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d
Yoshi
12:15 AM
Thanks Jason. How would that work though if we are then filtering using other fields as well?

I also worry a little a bit about making that search, since we would need to pass in an extremely large filter_by string to exclude all of the ids, which can be hundreds of thousands of Documents.
12:17
Yoshi
12:17 AM
I saw that it was released relatively recently, but would using an object with enable_nested_fieldsbe a potential solution?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
12:18 AM
> since we would need to pass in an extremely large filter_by string to exclude all of the ids, which can be hundreds of thousands of Documents.
A client can contact hundreds of thousands of dentists?
Yoshi
Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d
Yoshi
12:19 AM
A client represents an organization with a sales team, which as a team can contact hundreds of thousands of businesses.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
12:20 AM
Ah I see…
12:22
Jason
12:22 AM
Even with nested fields, you’d still essentially create one field per client. So for eg:

{latest_contact_dates: {client_1: 123, client_2: 234}
12:23
Jason
12:23 AM
So if the same dentist gets contacted by say 1000 organizations (clients), then you would have 1000 fields in that nested object, one for each client
12:25
Jason
12:25 AM
which is not too bad…
Yoshi
Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d
Yoshi
12:26 AM
yea, i’m not sure what is considered reasonable…

I’m happy to change our data structure in the future, trying to figure out what would be a good solution for the short to mid term (probably up to 1000 organizations or so)
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
12:27 AM
Nested fields should be fine for now…
12:27
Jason
12:27 AM
In the near future, we plan to launch support for JOINing data across collections, which should help you use-case
12:28
Jason
12:28 AM
So you’d store all dentist info in one collection, and all client contact dates in another collection, and join this data at run-time
Yoshi
Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d
Yoshi
12:28 AM
oh awesome, i love that
12:31
Yoshi
12:31 AM
ok, so maybe for the short term i can implement with nested_fields and then can pivot to using collection relationships when it’s ready

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Discussions on Typesense, Collections, and Dynamic Fields

Tugay shares plans to use Typesense for their SaaS platform and asks about collection sizes and sharding. Jason clarifies Typesense's capabilities and shares a beta feature. They discuss using unique collections per customer and new improvements. Kishore Nallan and Gabe comment on threading and data protection respectively.

3

45
35mo
Solved

Updating Collections Strategy and Faceting New Field

Nithin asked about strategies for updating collections and faceting new fields. Kishore Nallan suggested creating another collection, indexing in the background and using aliases to switch live traffic over, and shared details about the upcoming release.

10
34mo

Performance Characteristics of Filtering Search Results

Oskar queries the performance difference in filtering search results. Jason clarifies how filters work and provides performance improvement suggestions like increasing vCPUs and sharding the collection. Kishore Nallan explains filter IDs and document ID matching. The thread concludes with discussions on performance tradeoffs in filter implementation.

33
15mo
Solved

Issues with Schema Creation and Nested Fields

Sean encountered a problem with schema creation involving auto nested fields. Kishore Nallan suggested checking the API response for errors and adding problematic fields to the schema as optional. Sean confirmed the advice.

1

9
2mo
Solved

Issue with Typesense Schema and Ruby Client

Mateo faces issues while creating a schema and using Ruby client for Typesense. Jason suggests using a new field instead of 'id' and provides assistance for Ruby client errors.

33
10mo