#community-help

Resolving Issues with Reference Field in v0.26.0.rc11

TLDR Shaun reported problems applying the reference field in v0.26.0.rc11. The issue made the tested field disappear from UI, and an attempt to fix it resulted in an error. Harpreet suggested re-indexing data and promised to consider improving the feature in future updates. Harpreet is currently working on a related feature - sorting by reference field.

Powered by Struct AI

2

1

1

Aug 08, 2023 (4 months ago)
Shaun
Photo of md5-9dd01dbbef7bac5e85a472a52dc35647
Shaun
05:45 AM
Relating to new join/reference feature testing in v0.26.0.rc11
05:46
Shaun
05:46 AM
Posted here per Kishore Nallan request.

Applied this to food collection
{
  "fields": [
    {
      "name": "foodBrandId",
      "drop": true
    },
    {
      "name": "foodBrandId",
      "type": "string",
      "optional": true,
      "facet": false,
      "sort": false,
      "reference": "foodBrands.id"
    }
  ]
}

Which it said was saved successfully, but I don't think the web UI knows how to show it yet, as that field disappears from the list of the schema when this change was applied (still seems to be found on the actual data).

Trying to make subsequent changes shows: "Error: e: Request failed with HTTP code 400 | Server said: Field foodBrandId is not part of collection schema." so seems like all it ended up doing was dropping the field (I just added it back to schema).

Furthermore in the sole example given, can we do it this direction for the reference instead to accomplish the same thing:

Schema of Products collection:
{
  "name":  "Products",
  "fields": [
    {"name": "product_id", "type": "string", "reference": "CustomerProductPrices.product_id"},
    {"name": "product_name", "type": "string"},
    {"name": "product_description", "type": "string"}
  ]
}

CustomerProductPrices collectionโ€™s schema would store the reference to the product and its associated price:
{
  "name":  "CustomerProductPrices",
  "fields": [
    {"name": "customer_id", "type": "string"},
    {"name": "product_id", "type": "string"},
    {"name": "product_price", "type": "float"}
  ] 
}

Or that wont work?

As when I tried to add the reference in my case about, I get this when trying to query:

"RequestMalformed: Request failed with HTTP code 400 | Server said: Failed to apply reference filter on foodBrands collection: Could not find any field in foodBrands referencing the collection food."

However, I can't add the reference in the foodBrands needs to be foods, as many foods could ref/share the same brand, so it needs to be food.foodBrandId pointing to foodBrands.id
[12:26 PM] My query as follows: "filter_by": "$foodBrands(id:=foodBrandId)"
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
05:47 AM
Harpreet cc

1

Harpreet
Photo of md5-745d880d794220d9f0fb9ade17c6b861
Harpreet
06:27 AM
We don't support updating the fields with reference yet so you'll have to re-index the data. Regarding the Products collection having the reference field, it won't work since we would be searching for products.
If I understood your problem correctly, you have food and foodBrands collections and you would want to search on food collection and filter by specific foodBrands?
Shaun
Photo of md5-9dd01dbbef7bac5e85a472a52dc35647
Shaun
06:29 AM
What you mean, re-index? As in delete all the records in the TS collection, then reload them ?
Harpreet
Photo of md5-745d880d794220d9f0fb9ade17c6b861
Harpreet
06:29 AM
Yeah
Shaun
Photo of md5-9dd01dbbef7bac5e85a472a52dc35647
Shaun
06:30 AM
Ok understood, will that be changed later on, so we can update with a reference property and not have to delete everything ?
Harpreet
Photo of md5-745d880d794220d9f0fb9ade17c6b861
Harpreet
06:31 AM
Yeah, I think this would be nice feature to support.

1

Shaun
Photo of md5-9dd01dbbef7bac5e85a472a52dc35647
Shaun
06:32 AM
If I understood your problem correctly, you have food and foodBrands collections and you would want to search on food collection and filter by specific foodBrands?

Yes we have both collections. Both do get searched and currently food can filter by foodBrandId already. We want to add the reference to foodBrands so that when a food is returned (in any way) we can also grab the foodBrands.name for example from the other collection (currently food just has the foreign key).
Harpreet
Photo of md5-745d880d794220d9f0fb9ade17c6b861
Harpreet
07:09 AM
Give me some time to think about this.
Shaun
Photo of md5-9dd01dbbef7bac5e85a472a52dc35647
Shaun
07:09 AM
ok got it
11:22
Shaun
11:22 PM
Any ideas ?
Aug 09, 2023 (4 months ago)
Harpreet
Photo of md5-745d880d794220d9f0fb9ade17c6b861
Harpreet
06:13 AM
Hey Shaun we have a few ideas but working out what would be the most efficient way to implement this. I'll keep you posted.
Shaun
Photo of md5-9dd01dbbef7bac5e85a472a52dc35647
Shaun
06:13 AM
ok cool

1

06:14
Shaun
06:14 AM
Please come back to me when you know next
Aug 14, 2023 (3 months ago)
Shaun
Photo of md5-9dd01dbbef7bac5e85a472a52dc35647
Shaun
08:05 AM
any progress Harpreet
Harpreet
Photo of md5-745d880d794220d9f0fb9ade17c6b861
Harpreet
08:10 AM
Hey Shaun, I'm currently working on adding support for sorting by a reference field. I'll implement your feature request next.
Shaun
Photo of md5-9dd01dbbef7bac5e85a472a52dc35647
Shaun
08:10 AM
ah ok makes sense - the sort by ref field is also useful I think
08:10
Shaun
08:10 AM
Ok, I will await that being done before further testing

1

Aug 17, 2023 (3 months ago)
Harpreet
Photo of md5-745d880d794220d9f0fb9ade17c6b861
Harpreet
05:45 AM
> We want to add the reference to foodBrands so that when a food is returned (in any way) we can also grab the foodBrands.name for example from the other collection (currently food just has the foreign key).
This should work by adding include_fields: $foodBrands(name) when searching on food collection.
https://github.com/typesense/typesense/pull/1156
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
05:46 AM
Let me build that change.
Shaun
Photo of md5-9dd01dbbef7bac5e85a472a52dc35647
Shaun
05:47 AM
sweet ๐Ÿ˜„

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Troubleshooting Typesense API Sorting Error in Item Collections

Nikhil encountered a sorting error when using Typesense API due to an optional reference field in the items collection schema. Harpreet and Jason helped debug and identified a bug with sorting in general for 2 fields with the first one being the join field, promising a fix in a forthcoming build.

2

45
2w
Solved

Issue with Query Hangs Identifying Different Collection Field Names

Kanak experiences query hangs when collection field names differ. Ahmed had an issue adding documents to collections, which was self-resolved. Kishore Nallan resolved Kanak's issue with the updated build.

5
4mo
Solved

Resolving Typesense Analytics "top_queries" Collection Issue

Zain faced issues with typesense analytics and an empty "top_queries" collection. Jason and Kishore Nallan assisted in troubleshooting, identified a bug, and suggested a fix in `0.25.0.rc66`. The fix worked for Zain.

3

29
3mo
Solved

Issue with Join Operation in `CustomerProductPrices` Collection

Nikhil had concerns about adding product prices before the corresponding product and reported a crash when using a filter clause. Harpreet provided a workaround for fetching related products and also suggested a reason for the need of product presence enforcement. The crash could not be reproduced at Harpreet's end.

1

17
3mo

Trouble in Implementing Deeply Nested Search

Anirudh is struggling to implement a two-level nested search. Jason asked for some specific examples to study the issue. Anirudh provided some material, realizing that adding top fields helped but might over-index. Jason then suggested reporting this issue on GitHub.

2

21
today