#community-help

Handling Field Variations in Typesense for Different Clients

TLDR Yoshi asked for a solution to handle varying contact_date fields for different clients in typesense. Jason recommended using nested fields for the short-term and mentioned future support for joining data across collections.

Powered by Struct AI

1

18
5mo
Solved
Join the chat
May 04, 2023 (5 months ago)
Yoshi
Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d
Yoshi
11:46 PM
Hi everyone,
We have a collection of companies that we have included in typesense, specifically dentists. Each client has a different contact_date for each dentist.

I’m curious if there is any best practice for handling this kind of field that is different for each company, but wanting to filter/sort by it for each user.

One way I could potentially do this is by adding a different schema field per client:
{
  "name": "ABC Company",
  "location": "San Francisco",
  "latest_contact_date_client_1": "2023-05-04T10:30:00Z",
  "latest_contact_date_client_2": "2023-05-04T10:30:00Z",
  ...
}

But that seems terribly inefficient.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
11:51 PM
Is the end goal to let clients search for dentists and allow them to filter based on their last contact date for each dentist?
May 05, 2023 (5 months ago)
Yoshi
Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d
Yoshi
12:06 AM
exactly
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
12:08 AM
I would recommend creating one main collection with all dentists… And then when a client contacts a dentist, create a new user-specific collection called say dentists_contacted_by_client_1 and add that dentist record into that collection.

Then at search time, you can do a request and fetch results from the user-specific collection sorted by contact time, then do another request to the main collection excluding the IDs for in the first result set
Yoshi
Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d
Yoshi
12:15 AM
Thanks Jason. How would that work though if we are then filtering using other fields as well?

I also worry a little a bit about making that search, since we would need to pass in an extremely large filter_by string to exclude all of the ids, which can be hundreds of thousands of Documents.
12:17
Yoshi
12:17 AM
I saw that it was released relatively recently, but would using an object with enable_nested_fieldsbe a potential solution?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
12:18 AM
> since we would need to pass in an extremely large filter_by string to exclude all of the ids, which can be hundreds of thousands of Documents.
A client can contact hundreds of thousands of dentists?
Yoshi
Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d
Yoshi
12:19 AM
A client represents an organization with a sales team, which as a team can contact hundreds of thousands of businesses.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
12:20 AM
Ah I see…
12:22
Jason
12:22 AM
Even with nested fields, you’d still essentially create one field per client. So for eg:

{latest_contact_dates: {client_1: 123, client_2: 234}
12:23
Jason
12:23 AM
So if the same dentist gets contacted by say 1000 organizations (clients), then you would have 1000 fields in that nested object, one for each client
12:25
Jason
12:25 AM
which is not too bad…
Yoshi
Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d
Yoshi
12:26 AM
yea, i’m not sure what is considered reasonable…

I’m happy to change our data structure in the future, trying to figure out what would be a good solution for the short to mid term (probably up to 1000 organizations or so)
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
12:27 AM
Nested fields should be fine for now…
12:27
Jason
12:27 AM
In the near future, we plan to launch support for JOINing data across collections, which should help you use-case
12:28
Jason
12:28 AM
So you’d store all dentist info in one collection, and all client contact dates in another collection, and join this data at run-time
Yoshi
Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d
Yoshi
12:28 AM
oh awesome, i love that
12:31
Yoshi
12:31 AM
ok, so maybe for the short term i can implement with nested_fields and then can pivot to using collection relationships when it’s ready

1