#community-help

Handling Field Variations in Typesense for Different Clients

TLDR Yoshi asked for a solution to handle varying contact_date fields for different clients in typesense. Jason recommended using nested fields for the short-term and mentioned future support for joining data across collections.

7mo

Join the chat

May 04, 2023 (7 months ago)

Photo of md5-9e27fed7af6568f2c8abff36e7e9da4d

Yoshi

11:46 PM

Hi everyone,
We have a collection of companies that we have included in typesense, specifically dentists. Each client has a different contact_date for each dentist.

I’m curious if there is any best practice for handling this kind of field that is different for each company, but wanting to filter/sort by it for each user.

One way I could potentially do this is by adding a different schema field per client:

{
  "name": "ABC Company",
  "location": "San Francisco",
  "latest_contact_date_client_1": "2023-05-04T10:30:00Z",
  "latest_contact_date_client_2": "2023-05-04T10:30:00Z",
  ...
}

But that seems terribly inefficient.

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

11:51 PM

Is the end goal to let clients search for dentists and allow them to filter based on their last contact date for each dentist?

May 05, 2023 (7 months ago)

Yoshi

12:06 AM

exactly

Jason

12:08 AM

I would recommend creating one main collection with all dentists… And then when a client contacts a dentist, create a new user-specific collection called say dentists_contacted_by_client_1 and add that dentist record into that collection.

Then at search time, you can do a request and fetch results from the user-specific collection sorted by contact time, then do another request to the main collection excluding the IDs for in the first result set

Yoshi

12:15 AM

Thanks Jason. How would that work though if we are then filtering using other fields as well?

I also worry a little a bit about making that search, since we would need to pass in an extremely large filter_by string to exclude all of the ids, which can be hundreds of thousands of Documents.

12:17

Yoshi

12:17 AM

I saw that it was released relatively recently, but would using an object with enable_nested_fieldsbe a potential solution?

Jason

12:18 AM

> since we would need to pass in an extremely large filter_by string to exclude all of the ids, which can be hundreds of thousands of Documents.
A client can contact hundreds of thousands of dentists?

Yoshi

12:19 AM

A client represents an organization with a sales team, which as a team can contact hundreds of thousands of businesses.

Jason

12:20 AM

Ah I see…

12:22

Jason

12:22 AM

Even with nested fields, you’d still essentially create one field per client. So for eg:

{latest_contact_dates: {client_1: 123, client_2: 234}

12:23

Jason

12:23 AM

So if the same dentist gets contacted by say 1000 organizations (clients), then you would have 1000 fields in that nested object, one for each client

12:25

Jason

12:25 AM

which is not too bad…

Yoshi

12:26 AM

yea, i’m not sure what is considered reasonable…

I’m happy to change our data structure in the future, trying to figure out what would be a good solution for the short to mid term (probably up to 1000 organizations or so)

Jason

12:27 AM

Nested fields should be fine for now…

12:27

Jason

12:27 AM

In the near future, we plan to launch support for JOINing data across collections, which should help you use-case

12:28

Jason

12:28 AM

So you’d store all dentist info in one collection, and all client contact dates in another collection, and join this data at run-time

Yoshi

12:28 AM

oh awesome, i love that

12:31

Yoshi

12:31 AM

ok, so maybe for the short term i can implement with nested_fields and then can pivot to using collection relationships when it’s ready

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Discussions on Typesense, Collections, and Dynamic Fields

Tugay shares plans to use Typesense for their SaaS platform and asks about collection sizes and sharding. Jason clarifies Typesense's capabilities and shares a beta feature. They discuss using unique collections per customer and new improvements. Kishore Nallan and Gabe comment on threading and data protection respectively.

35mo

Updating Collections Strategy and Faceting New Field

Nithin asked about strategies for updating collections and faceting new fields. Kishore Nallan suggested creating another collection, indexing in the background and using aliases to switch live traffic over, and shared details about the upcoming release.

34mo

Performance Characteristics of Filtering Search Results

Oskar queries the performance difference in filtering search results. Jason clarifies how filters work and provides performance improvement suggestions like increasing vCPUs and sharding the collection. Kishore Nallan explains filter IDs and document ID matching. The thread concludes with discussions on performance tradeoffs in filter implementation.

15mo

Issues with Schema Creation and Nested Fields

Sean encountered a problem with schema creation involving auto nested fields. Kishore Nallan suggested checking the API response for errors and adding problematic fields to the schema as optional. Sean confirmed the advice.

2mo

Issue with Typesense Schema and Ruby Client

Mateo faces issues while creating a schema and using Ruby client for Typesense. Jason suggests using a new field instead of 'id' and provides assistance for Ruby client errors.

10mo