Slackbot
08/08/2023, 5:45 AMShaun Byrnes
08/08/2023, 5:46 AMfood collection
{
"fields": [
{
"name": "foodBrandId",
"drop": true
},
{
"name": "foodBrandId",
"type": "string",
"optional": true,
"facet": false,
"sort": false,
"reference": "foodBrands.id"
}
]
}
Which it said was saved successfully, but I don't think the web UI knows how to show it yet, as that field disappears from the list of the schema when this change was applied (still seems to be found on the actual data).
Trying to make subsequent changes shows: "Error: e: Request failed with HTTP code 400 | Server said: Field foodBrandId is not part of collection schema." so seems like all it ended up doing was dropping the field (I just added it back to schema).
Furthermore in the sole example given, can we do it this direction for the reference instead to accomplish the same thing:
Schema of Products collection:
{
"name": "Products",
"fields": [
{
"name": "product_id",
"type": "string",
"reference": "CustomerProductPrices.product_id"
},
{
"name": "product_name",
"type": "string"
},
{
"name": "product_description",
"type": "string"
}
]
}
CustomerProductPrices collectionโs schema would store the reference to the product and its associated price:
{
"name": "CustomerProductPrices",
"fields": [
{
"name": "customer_id",
"type": "string"
},
{
"name": "product_id",
"type": "string"
},
{
"name": "product_price",
"type": "float"
}
]
}
Or that wont work?
As when I tried to add the reference in my case about, I get this when trying to query:
"RequestMalformed: Request failed with HTTP code 400 | Server said: Failed to apply reference filter on foodBrands collection: Could not find any field in foodBrands referencing the collection food."
However, I can't add the reference in the foodBrands needs to be foods, as many foods could ref/share the same brand, so it needs to be food.foodBrandId pointing to foodBrands.id
[12:26 PM] My query as follows: "filter_by": "$foodBrands(id:=foodBrandId)"Kishore Nallan
08/08/2023, 5:47 AMHarpreet Sangar
08/08/2023, 6:27 AMreference yet so you'll have to re-index the data. Regarding the Products collection having the reference field, it won't work since we would be searching for products.
If I understood your problem correctly, you have food and foodBrands collections and you would want to search on food collection and filter by specific foodBrands?Shaun Byrnes
08/08/2023, 6:29 AMHarpreet Sangar
08/08/2023, 6:29 AMShaun Byrnes
08/08/2023, 6:30 AMreference property and not have to delete everything ?Harpreet Sangar
08/08/2023, 6:31 AMShaun Byrnes
08/08/2023, 6:32 AMIf I understood your problem correctly, you have food and foodBrands collections and you would want to search on food collection and filter by specific foodBrands?
Yes we have both collections. Both do get searched and currently food can filter by foodBrandId already. We want to add the reference to foodBrands so that when a food is returned (in any way) we can also grab the foodBrands.name for example from the other collection (currently food just has the foreign key).Harpreet Sangar
08/08/2023, 7:09 AMShaun Byrnes
08/08/2023, 7:09 AMShaun Byrnes
08/08/2023, 11:22 PMHarpreet Sangar
08/09/2023, 6:13 AMShaun Byrnes
08/09/2023, 6:13 AMShaun Byrnes
08/09/2023, 6:14 AMShaun Byrnes
08/14/2023, 8:05 AMHarpreet Sangar
08/14/2023, 8:10 AMShaun Byrnes
08/14/2023, 8:10 AMShaun Byrnes
08/14/2023, 8:10 AMHarpreet Sangar
08/17/2023, 5:45 AMWe want to add the reference toThis should work by addingso that when a food is returned (in any way) we can also grab thefoodBrandsfor example from the other collection (currently food just has the foreign key).foodBrands.name
include_fields: $foodBrands(name) when searching on food collection.
https://github.com/typesense/typesense/pull/1156Kishore Nallan
08/17/2023, 5:46 AMShaun Byrnes
08/17/2023, 5:47 AMKishore Nallan
08/18/2023, 1:56 AM0.26.0.rc14 where you can try this out.Shaun Byrnes
08/18/2023, 7:05 AMShaun Byrnes
08/18/2023, 7:06 AMKishore Nallan
08/18/2023, 7:07 AMShaun Byrnes
08/18/2023, 7:07 AMShaun Byrnes
08/18/2023, 7:07 AMsh376gjtdxq8b9mop-1.a1.typesense.net [https: 443]Shaun Byrnes
08/18/2023, 7:08 AMKishore Nallan
08/18/2023, 7:08 AMKishore Nallan
08/18/2023, 7:09 AMShaun Byrnes
08/18/2023, 7:09 AMShaun Byrnes
08/18/2023, 7:09 AMShaun Byrnes
08/18/2023, 7:09 AMShaun Byrnes
08/18/2023, 7:10 AMHarpreet Sangar
08/18/2023, 7:16 AM{
"name": "foods",
"fields": [
{
"name": "name",
"type": "string"
},
{
"name": "food_brand_id",
"type": "string",
"reference": "foodBrands.id"
},
{
"name": "calories",
"type": "int32"
}
]
}
{
"name": "foodBrands",
"fields": [
{
"name": "brand_name",
"type": "string"
},
{
"name": "brand_category",
"type": "string"
}
]
}
Multi-search Query:
{
"collection": "foods",
"query_by": "name",
"q": "apple",
"include_fields": "name,calories,$foodBrands(brandName)"
}Shaun Byrnes
08/18/2023, 8:01 AMShaun Byrnes
08/19/2023, 12:04 AMfood collection:
{
"fields": [
{
"name": "foodBrandId",
"drop": true
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "foodBrandId",
"optional": true,
"sort": false,
"type": "string",
"reference": "foodBrands.id"
}
]
}Shaun Byrnes
08/19/2023, 12:04 AMError: e: Request failed with HTTP code 400 | Server said: There are duplicate field names in the schema.Shaun Byrnes
08/19/2023, 12:04 AMShaun Byrnes
08/19/2023, 12:06 AMfoodBrandId in the schema.
When trying to even run the searchShaun Byrnes
08/19/2023, 12:07 AMfoodBrandId from searching field, results are returned, I think it's because the drop operation worked, but adding the field back with reference failed.Shaun Byrnes
08/19/2023, 12:08 AMfoodBrandId still exists ...Shaun Byrnes
08/19/2023, 12:09 AMKishore Nallan
08/19/2023, 3:21 AMShaun Byrnes
08/19/2023, 10:37 AMKishore Nallan
08/19/2023, 10:41 AMShaun Byrnes
08/19/2023, 10:41 AMShaun Byrnes
08/19/2023, 10:41 AMShaun Byrnes
08/19/2023, 10:41 AMShaun Byrnes
09/05/2023, 5:49 AMlet searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "$brands1(uid:=brandId)"
};
let results = await client.collections("food1").documents().search(searchParameters);Shaun Byrnes
09/05/2023, 5:49 AMbrands1 collection: Could not find any field in brands1 referencing the collection food1.Shaun Byrnes
09/05/2023, 5:50 AMsh376gjtdxq8b9mop-1.a1.typesense.net [https: 443]
food1 and brands1Shaun Byrnes
09/05/2023, 5:50 AMKishore Nallan
09/05/2023, 6:26 AMKishore Nallan
09/05/2023, 4:09 PMShaun Byrnes
09/06/2023, 3:50 AMShaun Byrnes
09/06/2023, 3:51 AMShaun Byrnes
09/06/2023, 3:51 AMKishore Nallan
09/06/2023, 4:52 AM0.26.0.rc17. It should complete shortly (you'll get an email).
1. Typesense Cloud UI still does not support references.
2. Schema alter still does not support adding references.
We will get to both of those as soon as we iron out these issues we are finding. Sorry and thank you for the support and patience!Shaun Byrnes
09/06/2023, 5:03 AMShaun Byrnes
09/06/2023, 5:03 AMShaun Byrnes
09/06/2023, 5:03 AMShaun Byrnes
09/06/2023, 5:04 AMfood1 in brands1 collection.Shaun Byrnes
09/06/2023, 5:04 AMKishore Nallan
09/06/2023, 5:06 AMShaun Byrnes
09/06/2023, 5:06 AMKishore Nallan
09/06/2023, 5:07 AMShaun Byrnes
09/06/2023, 5:07 AMShaun Byrnes
09/06/2023, 5:10 AMShaun Byrnes
09/06/2023, 5:10 AMShaun Byrnes
09/06/2023, 5:10 AMbrands1 and food1Kishore Nallan
09/06/2023, 5:10 AMShaun Byrnes
09/06/2023, 5:10 AMHarpreet Sangar
09/06/2023, 5:24 AMShaun Byrnes
09/06/2023, 5:25 AMShaun Byrnes
09/06/2023, 5:26 AMShaun Byrnes
09/06/2023, 5:26 AMShaun Byrnes
09/06/2023, 5:27 AMlet searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "$brands1(uid:=brandId)"
};
let results = await client.collections("food1").documents().search(searchParameters);
console.dir(results, { depth: null });Harpreet Sangar
09/06/2023, 5:29 AMShaun Byrnes
09/06/2023, 5:31 AMShaun Byrnes
09/06/2023, 5:31 AMHarpreet Sangar
09/06/2023, 8:56 AMbrandId already present in food1 collection? you can do
let searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "brandId:=foo"
};Harpreet Sangar
09/06/2023, 8:57 AMbrands1.name, you can do
let searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "brandId:=foo",
"include_fields": "$brands1(name)"
};Harpreet Sangar
09/06/2023, 9:04 AMinclude_fields for this scenario, you'll have to send *, $brands1(name) in the meantime.Shaun Byrnes
09/07/2023, 6:25 AMShaun Byrnes
09/07/2023, 6:26 AMShaun Byrnes
09/07/2023, 6:42 AMlet searchParameters = {
"q": "",
"query_by": "name",
"include_fields": "*, $brands1(name)"
};
let results = await client.collections("food1").documents().search(searchParameters);
the issue seems to be the food1.name is overwritten by brands1.name ๐Shaun Byrnes
09/07/2023, 6:42 AM{
facet_counts: [],
found: 2,
hits: [
{
document: {
brandId: '1',
brandId_sequence_id: 0,
name: 'Brand one',
uid: '2'
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
},
{
document: {
brandId: '1',
brandId_sequence_id: 0,
name: 'Brand one',
uid: '1'
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
}
],
out_of: 2,
page: 1,
request_params: { collection_name: 'food1', per_page: 10, q: '' },
search_cutoff: false,
search_time_ms: 23
}Kishore Nallan
09/07/2023, 6:43 AMShaun Byrnes
09/07/2023, 6:44 AMname: 'Brand one' coming back for each is correct, but the food1.name values are not returnedShaun Byrnes
09/07/2023, 6:44 AMShaun Byrnes
09/07/2023, 6:46 AM$collectionName(prop1, prop2) we should have it come back as collectionName.propN in the document obj? This way it prevents same name properties from being clobberedShaun Byrnes
09/07/2023, 6:47 AM"include_fields": "*, $brands1(name, uid)" both the name and uid field in food1 are overwritten with the brands1 name+uid valuesKishore Nallan
09/07/2023, 6:49 AMShaun Byrnes
09/07/2023, 7:11 AMShaun Byrnes
09/07/2023, 7:13 AM"include_fields": "* as f, $brands1(name, uid) as b" maybeShaun Byrnes
09/07/2023, 7:18 AM"include_fields": "* as food, $foodBrands(name, uid) as brands, $userFoods(count) as userFoods"Shaun Byrnes
09/11/2023, 5:05 AMKishore Nallan
09/11/2023, 1:21 PMShaun Byrnes
09/11/2023, 11:23 PMKishore Nallan
09/15/2023, 2:56 AM0.26.0.rc18 that will address the issues you faced. Can I upgrade the sh376gjtdxq8b9mop cluster?Shaun Byrnes
09/19/2023, 4:22 AMShaun Byrnes
09/19/2023, 4:22 AMShaun Byrnes
09/19/2023, 4:22 AMHarpreet Sangar
09/19/2023, 5:03 AM"include_fields": "$foodBrands(name, uid) as brands, $userFoods(count) as userFoods"
Also, you'll be able to join on foodBrands collection when searching on foodsKishore Nallan
09/19/2023, 5:15 AMsh376gjtdxq8b9mop has now been upgraded.Shaun Byrnes
09/19/2023, 5:15 AMShaun Byrnes
09/19/2023, 5:15 AMShaun Byrnes
10/24/2023, 4:28 AMlet searchParameters = {
"q": "",
"query_by": "name",
"include_fields": "$brands1(name) as brands, $userFood1(count, userId) as userFood"
};
let results = await client.collections("food1").documents().search(searchParameters);Shaun Byrnes
10/24/2023, 4:30 AM{
facet_counts: [],
found: 2,
hits: [
{
document: {
brandId: '1',
brandId_sequence_id: 0,
'brands.name': 'Brand one',
id: '1',
name: 'Food 2',
uid: '2'
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
},
{
document: {
brandId: '1',
brandId_sequence_id: 0,
'brands.name': 'Brand one',
id: '0',
name: 'Food 1',
uid: '1'
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
}
],
out_of: 2,
page: 1,
request_params: { collection_name: 'food1', per_page: 10, q: '' },
search_cutoff: false,
search_time_ms: 0
}Shaun Byrnes
10/24/2023, 4:34 AMuserFood data being returned - doesnt error out either ๐Shaun Byrnes
10/24/2023, 4:35 AMuserFood1 collection and loaded some dummy docs into it with a reference to food1.uidHarpreet Sangar
10/24/2023, 4:37 AM*_sequence_id field like brandId_sequence_id in the response. My guess is for some reason the reference to food1 is not created.Harpreet Sangar
10/24/2023, 4:39 AMShaun Byrnes
10/24/2023, 4:39 AMShaun Byrnes
10/24/2023, 4:40 AMShaun Byrnes
10/24/2023, 4:40 AMHarpreet Sangar
10/24/2023, 4:41 AMShaun Byrnes
10/24/2023, 4:43 AMShaun Byrnes
10/24/2023, 4:43 AMShaun Byrnes
10/24/2023, 4:43 AMcount field with some int data in itShaun Byrnes
10/24/2023, 4:44 AMHarpreet Sangar
10/24/2023, 4:48 AMfood1 collection doesn't have the reference field to userFood1, you'll have to specify a "join" in the filter_by like:
"filter_by": "$userFood1(userId:foo)",
to get the data of a particular user
or
"filter_by": "$userFood1(id:*)",
to get all the related docs.Harpreet Sangar
10/24/2023, 4:49 AMv0.26.0.rc25. I'm not sure if v0.26.0.rc11 supports the id: * syntax.Shaun Byrnes
10/24/2023, 4:50 AMShaun Byrnes
10/24/2023, 4:50 AMShaun Byrnes
10/24/2023, 4:50 AM$collectionName() syntax which is where the reference has to be ?Shaun Byrnes
10/24/2023, 4:51 AMShaun Byrnes
10/24/2023, 4:51 AMShaun Byrnes
10/24/2023, 4:52 AM"reference": "food1.uid",
from userFoods1Shaun Byrnes
10/24/2023, 4:52 AMfood1 collection instead ?Harpreet Sangar
10/24/2023, 4:55 AMShaun Byrnes
10/24/2023, 4:57 AMShaun Byrnes
10/24/2023, 4:57 AMShaun Byrnes
10/24/2023, 4:57 AMShaun Byrnes
10/24/2023, 4:57 AMShaun Byrnes
10/24/2023, 4:58 AMShaun Byrnes
10/24/2023, 4:59 AMShaun Byrnes
10/24/2023, 5:02 AM"include_fields": "$brands1(name) as brands, $userFood1(foodId:*) as userFood"Shaun Byrnes
10/24/2023, 5:02 AMHarpreet Sangar
10/24/2023, 5:05 AMfilter_by. Try this out:
let searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "$userFood1(id:*)",
"include_fields": "$brands1(name) as brands, $userFood1(count, userId) as userFood"
};
let results = await client.collections("food1").documents().search(searchParameters);Kishore Nallan
10/24/2023, 5:05 AMShaun Byrnes
10/24/2023, 5:05 AMShaun Byrnes
10/24/2023, 5:05 AMKishore Nallan
10/24/2023, 5:05 AM0.26.0.rc25Shaun Byrnes
10/24/2023, 5:06 AMShaun Byrnes
10/24/2023, 5:06 AMShaun Byrnes
10/24/2023, 5:11 AMShaun Byrnes
10/24/2023, 5:21 AM"include_fields": "$brands1(name) as brands, $userFood1(foodId:*) as userFood"Shaun Byrnes
10/24/2023, 5:22 AMHarpreet Sangar
10/24/2023, 5:23 AMfilter_by. Try this out:
let searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "$userFood1(id:*)",
"include_fields": "$brands1(name) as brands, $userFood1(count, userId) as userFood"
};
let results = await client.collections("food1").documents().search(seShaun Byrnes
10/24/2023, 5:23 AMShaun Byrnes
10/24/2023, 5:23 AMShaun Byrnes
10/24/2023, 5:25 AM{
facet_counts: [],
found: 2,
hits: [
{
document: {
brandId: '1',
brandId_sequence_id: 0,
'brands.name': 'Brand one',
id: '1',
name: 'Food 2',
uid: '2',
'userFood.count': [ 10, 1 ],
'userFood.userId': [ 's', 'b' ]
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
},
{
document: {
brandId: '1',
brandId_sequence_id: 0,
'brands.name': 'Brand one',
id: '0',
name: 'Food 1',
uid: '1',
'userFood.count': 5,
'userFood.userId': 's'
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
}
],
out_of: 2,
page: 1,
request_params: { collection_name: 'food1', per_page: 10, q: '' },
search_cutoff: false,
search_time_ms: 0
}Shaun Byrnes
10/24/2023, 5:25 AMShaun Byrnes
10/24/2023, 5:25 AMlet searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "$userFood1(id:*)",
"include_fields": "$brands1(name) as brands, $userFood1(count, userId) as userFood"
};
let results = await client.collections("food1").documents().search(searchParameters);Shaun Byrnes
10/24/2023, 5:26 AM"$userFood1(id:*)",Shaun Byrnes
10/24/2023, 5:26 AMHarpreet Sangar
10/24/2023, 5:27 AMfood1 collection doesn't have the reference field to userFood1, we have to specify a "join" in the filter_by like:
"filter_by": "$userFood1(userId:foo)",
to get the data of a particular user
or
"filter_by": "$userFood1(id:*)",
to get all the related docs.Shaun Byrnes
10/24/2023, 5:27 AMShaun Byrnes
10/24/2023, 5:28 AMShaun Byrnes
10/24/2023, 5:28 AMHarpreet Sangar
10/24/2023, 5:28 AM"filter_by": "$userFood1(count:>100)",Shaun Byrnes
10/24/2023, 5:28 AMShaun Byrnes
10/24/2023, 5:28 AMShaun Byrnes
10/24/2023, 5:28 AMHarpreet Sangar
10/24/2023, 5:28 AMShaun Byrnes
10/24/2023, 5:29 AMShaun Byrnes
10/24/2023, 5:29 AMShaun Byrnes
10/24/2023, 5:29 AMShaun Byrnes
10/24/2023, 5:29 AMShaun Byrnes
10/24/2023, 5:29 AMHarpreet Sangar
10/24/2023, 5:32 AMwhat if i remove the reference in the userFood collection to food
and put it in food collectionThen the "join" won't be necessary in the
filter_by but I think in your scenario it wouldn't make sense to do that.Shaun Byrnes
10/24/2023, 5:33 AMuserFood1 collection:
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "foodId",
"optional": false,
"reference": "food1.uid",
"sort": false,
"type": "string"
},Shaun Byrnes
10/24/2023, 5:33 AMHarpreet Sangar
10/24/2023, 5:36 AMlet searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "$userFood1(id:*)",
"include_fields": "$brands1(name) as brands, $userFood1(*) as userFood"
};
let results = await client.collections("food1").documents().search(searchParameters);
You'll see userFood.foodId_sequence_id field that's used internally to link the referenced docs.Shaun Byrnes
10/24/2023, 5:38 AMShaun Byrnes
10/24/2023, 5:38 AMdocument: {
brandId: '1',
brandId_sequence_id: 0,
'brands.name': 'Brand one',
id: '0',
name: 'Food 1',
uid: '1',
'userFood.count': 5,
'userFood.userId': 's'
},Shaun Byrnes
10/24/2023, 5:38 AMbrandId_sequence_id but no foodId_sequence_idHarpreet Sangar
10/24/2023, 5:40 AMShaun Byrnes
10/24/2023, 5:40 AMShaun Byrnes
10/24/2023, 5:40 AMShaun Byrnes
10/24/2023, 5:40 AMHarpreet Sangar
10/24/2023, 5:42 AMfoodId_sequence_id field is still present in the userFood docs that have already been created. Try indexing new docs in userFood, they won't be returned in the response..Shaun Byrnes
10/24/2023, 5:44 AMHarpreet Sangar
10/24/2023, 5:46 AMuserFoodShaun Byrnes
10/24/2023, 5:46 AMShaun Byrnes
10/24/2023, 5:58 AMShaun Byrnes
10/24/2023, 5:58 AMuserFood.count in the schema for sorting.Shaun Byrnes
10/24/2023, 5:58 AMHarpreet Sangar
10/24/2023, 5:59 AM$userFood(count:desc)Shaun Byrnes
10/24/2023, 5:59 AMShaun Byrnes
10/24/2023, 6:00 AMShaun Byrnes
10/24/2023, 6:00 AMShaun Byrnes
10/24/2023, 6:00 AMHarpreet Sangar
10/24/2023, 6:01 AMKishore Nallan
10/24/2023, 6:01 AM$ syntax. Like $ReferenceCollection(field_name) .Shaun Byrnes
10/24/2023, 6:03 AMShaun Byrnes
10/24/2023, 6:03 AMShaun Byrnes
10/24/2023, 6:03 AMuserFood1.count.Shaun Byrnes
10/24/2023, 6:04 AMShaun Byrnes
10/24/2023, 6:05 AM"q": "",
"query_by": "name",
"filter_by": "$userFood1(id:*)",
"include_fields": "$brands1(name) as brands, $userFood1(count, userId) as userFood",
"sort_by": "$userFood1(count:desc)"Shaun Byrnes
10/24/2023, 6:05 AMShaun Byrnes
10/24/2023, 6:05 AMShaun Byrnes
10/24/2023, 6:06 AMShaun Byrnes
10/24/2023, 6:06 AM{
document: {
brandId: '2',
brandId_sequence_id: 1,
'brands.name': 'Brand two',
id: '2',
name: 'Food 3',
uid: '3',
'userFood.count': 11,
'userFood.userId': 'b'
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
},
{
document: {
brandId: '1',
brandId_sequence_id: 0,
'brands.name': 'Brand one',
id: '1',
name: 'Food 2',
uid: '2',
'userFood.count': [ 5, 3 ],
'userFood.userId': [ 's', 'b' ]
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
},
{
document: {
brandId: '1',
brandId_sequence_id: 0,
'brands.name': 'Brand one',
id: '0',
name: 'Food 1',
uid: '1',
'userFood.count': [ 10, 0 ],
'userFood.userId': [ 's', 'b' ]
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
}Shaun Byrnes
10/24/2023, 6:06 AMHarpreet Sangar
10/24/2023, 6:20 AM'userFood.count': [ 10, 0 ], and 'userFood.count': [ 5, 3 ],Harpreet Sangar
10/24/2023, 6:40 AMBascially trying to sort by all userFood counts highest to lowestYou can try
"q": "*",
"filter_by": "$food1(id:*)",
"include_fields": "$brands1(name) as brands, $food1(brandId, id, name, uid) as foods",
"sort_by": "count:desc"Harpreet Sangar
10/24/2023, 6:44 AMlet results = await client.collections("userFood1").documents().search(searchParameters);Shaun Byrnes
10/24/2023, 8:29 AM{
facet_counts: [],
found: 5,
hits: [
{
document: {
count: 11,
'food.brandId': '2',
'food.id': '2',
'food.name': 'Food 3',
'food.uid': '3',
foodId: '3',
foodId_sequence_id: 2,
id: '4',
userId: 'b'
},
highlight: {},
highlights: []
},
{
document: {
count: 10,
'food.brandId': '1',
'food.id': '0',
'food.name': 'Food 1',
'food.uid': '1',
foodId: '1',
foodId_sequence_id: 0,
id: '0',
userId: 's'
},
highlight: {},
highlights: []
},
{
document: {
count: 5,
'food.brandId': '1',
'food.id': '1',
'food.name': 'Food 2',
'food.uid': '2',
foodId: '2',
foodId_sequence_id: 1,
id: '1',
userId: 's'
},
highlight: {},
highlights: []
},
{
document: {
count: 3,
'food.brandId': '1',
'food.id': '1',
'food.name': 'Food 2',
'food.uid': '2',
foodId: '2',
foodId_sequence_id: 1,
id: '3',
userId: 'b'
},
highlight: {},
highlights: []
},
{
document: {
count: 0,
'food.brandId': '1',
'food.id': '0',
'food.name': 'Food 1',
'food.uid': '1',
foodId: '1',
foodId_sequence_id: 0,
id: '2',
userId: 'b'
},
highlight: {},
highlights: []
}
],
out_of: 5,
page: 1,
request_params: { collection_name: 'userFood1', per_page: 10, q: '*' },
search_cutoff: false,
search_time_ms: 0
}Harpreet Sangar
10/24/2023, 8:32 AMShaun Byrnes
10/24/2023, 8:32 AMShaun Byrnes
10/24/2023, 8:33 AMShaun Byrnes
10/24/2023, 8:34 AMlet searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "$userFood1(userId:s)",
"include_fields": "$brands1(name) as brands, $userFood1(count, userId) as userFood",
"sort_by": "$userFood1(count:desc)"
};
let results = await client.collections("food1").documents().search(searchParameters);Shaun Byrnes
10/24/2023, 8:34 AMShaun Byrnes
10/24/2023, 8:35 AM$brands1(name)Shaun Byrnes
10/24/2023, 8:35 AMShaun Byrnes
10/24/2023, 8:36 AMbrands.name in the schema.Shaun Byrnes
10/24/2023, 8:37 AMHarpreet Sangar
10/24/2023, 8:37 AMfilter_by: $brands(name: foo)Shaun Byrnes
10/24/2023, 8:39 AMShaun Byrnes
10/24/2023, 8:39 AMShaun Byrnes
10/24/2023, 8:41 AMmin() max() count() sum() avg() etc right ?Shaun Byrnes
10/24/2023, 8:41 AMHarpreet Sangar
10/24/2023, 8:42 AMShaun Byrnes
10/24/2023, 8:42 AMShaun Byrnes
10/24/2023, 8:42 AMShaun Byrnes
10/24/2023, 8:43 AMShaun Byrnes
10/24/2023, 8:43 AMHarpreet Sangar
10/24/2023, 8:45 AMfacet_by reference field and then other features.Shaun Byrnes
10/24/2023, 8:45 AMShaun Byrnes
10/24/2023, 9:15 AMfood collection schema:
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "portions",
"optional": true,
"sort": false,
"type": "object[]"
}
which produces this type of output when returning results
{
facet_counts: [],
found: 2,
hits: [
{
document: {
brandId: '1',
brandId_sequence_id: 0,
'brands.name': 'Brand one',
id: '0',
name: 'Food 1',
portions: [ { portionId: 'a' } ],
uid: '1',
'userFood.count': 10,
'userFood.userId': 's'
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
},
{
document: {
brandId: '1',
brandId_sequence_id: 0,
'brands.name': 'Brand one',
id: '1',
name: 'Food 2',
portions: [ { portionId: 'c' }, { portionId: 'b' } ],
uid: '2',
'userFood.count': 5,
'userFood.userId': 's'
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
}
],
out_of: 3,
page: 1,
request_params: { collection_name: 'food1', per_page: 10, q: '' },
search_cutoff: false,
search_time_ms: 0
}
Is it currently possible to have portions[].portionId match the portions collection via reference field ?Harpreet Sangar
10/24/2023, 9:46 AMhow will the facet_by reference is supposed to work ?For example, faceting on brand name when searching on foods.
Is it currently possible to haveYou'll have to include this field in the schema, like:match theportions[].portionIdcollection via reference field ?portions
"name": "portions.portionId", "type": "string[]", "reference": "foo.bar"
Is there a reason for having an array of objects though?Shaun Byrnes
10/24/2023, 9:47 AMShaun Byrnes
10/24/2023, 9:49 AMShaun Byrnes
10/24/2023, 9:50 AMfood.portions.portionId = portions.idShaun Byrnes
10/24/2023, 9:50 AMportions.nameHarpreet Sangar
10/24/2023, 9:51 AMstring[] field of just portion ids work?Shaun Byrnes
10/24/2023, 9:54 AMShaun Byrnes
10/24/2023, 9:54 AM"portionSize": {
"displayFormat": 0,
"portionsPerItem": null,
"selectedPortionIndex": 1,
"unit": "g",
"value": 1
},
"portions": [
{
"amount": 100,
"measureUnitId": "REzO2pZ0K1J6MFrCnWAi",
"multiple": 1,
"portionSizeId": "GEMDwd7SBYFZ6T9Ui0s2",
"type": 0
},
{
"amount": 604,
"measureUnitId": "GmwdaGtivG85KDyjohtj",
"multiple": 6.04,
"portionSizeId": "9UvEjhgOXKTITezpsPGu",
"type": 0
}
],Harpreet Sangar
10/24/2023, 9:58 AMportionId in each object. So I suggested this
portionIds: [ 'c', 'b' ],Shaun Byrnes
10/24/2023, 10:01 AMShaun Byrnes
10/24/2023, 10:02 AMShaun Byrnes
10/24/2023, 10:02 AM{
facet_counts: [],
found: 2,
hits: [
{
document: {
brandId: '1',
brandId_sequence_id: 0,
'brands.name': 'Brand one',
id: '0',
name: 'Food 1',
'portionSizes.name': [ '1 cup', '1 glass' ],
'portions.portionId': [ 'a', 'b' ],
'portions.portionId_sequence_id': [ 0, 1 ],
uid: '1',
'userFood.count': 10,
'userFood.userId': 's'
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
},
{
document: {
brandId: '1',
brandId_sequence_id: 0,
'brands.name': 'Brand one',
id: '1',
name: 'Food 2',
'portionSizes.name': [ '1 glass', '100ml' ],
'portions.portionId': [ 'b', 'c' ],
'portions.portionId_sequence_id': [ 1, 2 ],
uid: '1',
'userFood.count': 5,
'userFood.userId': 's'
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
}
],
out_of: 3,
page: 1,
request_params: { collection_name: 'food1', per_page: 10, q: '' },
search_cutoff: false,
search_time_ms: 0
}Shaun Byrnes
10/24/2023, 10:02 AMlet searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "$userFood1(userId:s)",
"include_fields": "$brands1(name) as brands, $userFood1(count, userId) as userFood, $portionSizes1(name) as portionSizes",
"sort_by": "$userFood1(count:desc)"
};Shaun Byrnes
10/24/2023, 10:03 AM[
{
"uid": "String Value",
"name": "String Value",
"brandId": "String Value",
"portions.portionId": [
"String Value 1 ",
"String Value 2"
]
},
{
"uid": "String Value",
"name": "String Value",
"brandId": "String Value",
"portions.portionId": [
"String Value 1 ",
"String Value 2"
]
}
]Shaun Byrnes
10/24/2023, 10:03 AMShaun Byrnes
10/24/2023, 10:05 AM[
{
"uid": "String Value",
"name": "String Value",
"brandId": "String Value",
"portions": [
{ "portionId": "String Value 1 ", "other": 1},
{ "portionId": "String Value 2", "other": 2}
]
},
{
"uid": "String Value",
"name": "String Value",
"brandId": "String Value",
"portions": [
{ "portionId": "String Value 1 ", "other": 1},
{ "portionId": "String Value 2", "other": 2}
]
}
]Shaun Byrnes
10/24/2023, 10:05 AMHarpreet Sangar
10/24/2023, 10:09 AMinclude_by syntax to specify this.Shaun Byrnes
10/24/2023, 10:10 AMShaun Byrnes
10/24/2023, 10:10 AM{
"created_at": 1698141520,
"default_sorting_field": "",
"enable_nested_fields": true,
"fields": [
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "uid",
"optional": false,
"sort": false,
"type": "string"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "name",
"optional": false,
"sort": false,
"type": "string"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "brandId",
"optional": false,
"reference": "brands1.uid",
"sort": false,
"type": "string"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "portions.portionId",
"optional": false,
"reference": "portionSizes1.uid",
"sort": false,
"type": "string[]"
}
],
"name": "food1",
"num_documents": 3,
"symbols_to_index": [],
"token_separators": []
}Shaun Byrnes
10/24/2023, 10:11 AMlet searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "$userFood1(userId:s)",
"include_fields": "$brands1(name) as brands, $userFood1(count, userId) as userFood, $portionSizes1(name) as portionSizes",
"sort_by": "$userFood1(count:desc)"
};
let results = await client.collections("food1").documents().search(searchParameters);Harpreet Sangar
10/24/2023, 10:11 AMinclude_fields syntax.Shaun Byrnes
10/24/2023, 10:14 AMHarpreet Sangar
10/24/2023, 10:16 AMtype[] internally with the name object_name.field_name.Shaun Byrnes
10/24/2023, 10:32 AM"portionSize": {
"displayFormat": 0,
"portionsPerItem": null,
"selectedPortionIndex": 1,
"unit": "g",
"value": 1
},
"portions": [
{
"amount": 100,
"measureUnitId": "REzO2pZ0K1J6MFrCnWAi",
"multiple": 1,
"portionSizeId": "GEMDwd7SBYFZ6T9Ui0s2",
"type": 0
},
{
"amount": 604,
"measureUnitId": "GmwdaGtivG85KDyjohtj",
"multiple": 6.04,
"portionSizeId": "9UvEjhgOXKTITezpsPGu",
"type": 0
}
],Kishore Nallan
10/24/2023, 10:41 AMShaun Byrnes
10/24/2023, 10:42 AMShaun Byrnes
10/24/2023, 10:42 AMKishore Nallan
10/24/2023, 10:45 AMShaun Byrnes
10/24/2023, 10:45 AMShaun Byrnes
10/24/2023, 10:47 AMShaun Byrnes
10/24/2023, 10:48 AMShaun Byrnes
10/24/2023, 10:48 AMShaun Byrnes
10/24/2023, 10:49 AMShaun Byrnes
10/24/2023, 10:50 AMKishore Nallan
10/24/2023, 11:01 AMShaun Byrnes
10/24/2023, 11:09 AMHarpreet Sangar
10/24/2023, 2:29 PMHarpreet Sangar
10/24/2023, 2:33 PMShaun Byrnes
10/24/2023, 10:08 PMShaun Byrnes
10/24/2023, 10:08 PMShaun Byrnes
10/24/2023, 10:42 PM...
,
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "portions.portionId",
"optional": false,
"reference": "portionSizes1.uid",
"sort": false,
"type": "string[]"
}
],
"name": "food1",
...
which produces this in TS:
[
{
"uid": "String Value",
"name": "String Value",
"brandId": "String Value",
"portions.portionId": [
"String Value 1 ",
"String Value 2"
]
},
...
]
With the above, I am able to reference the portion size names based on "reference": "portionSizes1.uid", as shown here in my results:
document: {
brandId: '1',
brandId_sequence_id: 0,
'brands.name': 'Brand one',
id: '1',
name: 'Food 2',
'portionSizes.name': [ '1 glass', '100ml' ],
'portions.portionId': [ 'b', 'c' ],
'portions.portionId_sequence_id': [ 1, 2 ],
uid: '1',
'userFood.count': 5,
'userFood.userId': 's'
},
However, with that schema, TS is altering my source format of an array of objects to:
"portions.portionId": [
"portionSizeId1",
"portionSizeId2"
]
Whereas my source data looks like this:
"portions": [
{
"amount": 100,
"measureUnitId": "REzO2pZ0K1J6MFrCnWAi",
"multiple": 1,
"portionSizeId": "GEMDwd7SBYFZ6T9Ui0s2",
"type": 0
},
{
"amount": 604,
"measureUnitId": "GmwdaGtivG85KDyjohtj",
"multiple": 6.04,
"portionSizeId": "9UvEjhgOXKTITezpsPGu",
"type": 0
}
],
As you can see our data being saved to TS from the source DB is an array of objects not an array of strings. We need to maintain the other values in each "portion" object under portions[] with the intent being to create reference to portionSizeId in each object, as that is matched to our TS portions collection, so we can obtain the name for each portion size.
As you can see in the results returned we are able to get the name matched (per Harpreet suggested query), but the schema is incorrect. Wondering how to keep the array of objects with each portionSizeId referenced and matched to the portions collection?
The query we are using is as follows:
let searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "$userFood1(userId:s)",
"include_fields": "$brands1(name) as brands, $userFood1(count, userId) as userFood, $portionSizes1(name) as portionSizes",
"sort_by": "$userFood1(count:desc)"
};
let results = await client.collections("food1").documents().search(searchParameters);
So not sure if this is normal behaviour etc, but looking at the docs on https://typesense.org/docs/0.25.1/api/collections.html#notes-on-indexing-common-types-of-data isn't super clear how exactly to structure the schema for type object or object[]
We actually originally has object[] for the portions.portionId thinking it should be that way given an array of objects which each contain a portionSizeId but Harpreet said to write it as above.
Hopefully this is all clear now what we are asking?Harpreet Sangar
10/25/2023, 3:53 AMportions.portionId in your food1 schema but your source data does not contain portionId field in it.
"portions": [
{
"amount": 100,
"measureUnitId": "REzO2pZ0K1J6MFrCnWAi",
"multiple": 1,
"portionSizeId": "GEMDwd7SBYFZ6T9Ui0s2",
"type": 0
},
{
"amount": 604,
"measureUnitId": "GmwdaGtivG85KDyjohtj",
"multiple": 6.04,
"portionSizeId": "9UvEjhgOXKTITezpsPGu",
"type": 0
}
],Harpreet Sangar
10/25/2023, 3:55 AM"portions.portionId": [
"portionSizeId1",
"portionSizeId2"
]Harpreet Sangar
10/25/2023, 4:01 AMcurl "http://localhost:8108/collections" \
-X POST \
-H "Content-Type: application/json" \
-H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
-d '{
"name": "food1",
"fields": [
{
"name": "portions.portionId",
"reference": "portionSizes1.uid",
"type": "string[]"
}
],
"enable_nested_fields": true
}'
curl "http://localhost:8108/collections/food1/documents/import?action=create" \
-H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
-H "Content-Type: text/plain" \
-X POST \
-d '{"portions": [{"amount": 100,"measureUnitId": "REzO2pZ0K1J6MFrCnWAi","multiple": 1,"portionSizeId": "GEMDwd7SBYFZ6T9Ui0s2","type": 0},{"amount": 604,"measureUnitId": "GmwdaGtivG85KDyjohtj","multiple": 6.04,"portionSizeId": "9UvEjhgOXKTITezpsPGu","type": 0}]}'Harpreet Sangar
10/25/2023, 4:03 AM"error":"Missing the required reference field `portions.portionId` in the document."Shaun Byrnes
10/25/2023, 6:00 AMShaun Byrnes
10/25/2023, 6:01 AMShaun Byrnes
10/25/2023, 6:03 AM'portionSizes.name': [ '1 glass', '100ml' ], array... Why would that have happened then?
Furthermore, because the TS cloud currently doesnt support changes with the reference fields to my knowledge, I expect to fix the portions.portionId > portions.portionSizeId change, we would need to delete the test food1 collection and re-create with correct field name, correct?Kishore Nallan
10/25/2023, 6:13 AMKishore Nallan
10/25/2023, 6:13 AMShaun Byrnes
10/25/2023, 6:13 AMShaun Byrnes
10/25/2023, 6:14 AMShaun Byrnes
10/25/2023, 6:23 AMShaun Byrnes
10/25/2023, 6:23 AMKishore Nallan
10/25/2023, 6:27 AMShaun Byrnes
10/25/2023, 6:31 AMsh376gjtdxq8b9mop-1.a1.typesense.net [https: 443]Shaun Byrnes
10/25/2023, 6:32 AMShaun Byrnes
10/25/2023, 6:33 AMfood1 brands1 userFood1 portionSizes1Kishore Nallan
10/25/2023, 6:34 AMShaun Byrnes
10/25/2023, 6:34 AMShaun Byrnes
10/25/2023, 6:35 AMHarpreet Sangar
10/25/2023, 8:17 AM{
"brandId": "1",
"brandId_sequence_id": 0,
"brands.name": "Brand one",
"id": "1",
"name": "Food 2",
"portionSizes": [
{
"name": "1 glass",
"uid": "b"
},
{
"name": "100ml",
"uid": "c"
}
],
"portions.portionId_sequence_id": [
1,
2
],
"uid": "2",
"userFood.count": 5,
"userFood.userId": "s"
}Shaun Byrnes
10/25/2023, 8:36 AMHarpreet Sangar
10/25/2023, 8:57 AMShaun Byrnes
10/25/2023, 9:00 AMShaun Byrnes
10/25/2023, 9:00 AMShaun Byrnes
10/26/2023, 10:21 PMHarpreet Sangar
10/27/2023, 8:52 AMcurl 'http://localhost:8108/multi_search' -X POST -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '
{"searches":[{
"collection": "food1",
"q": "",
"query_by": "name",
"filter_by": "$userFood1(userId:s)",
"include_fields": "$brands1(name :merge) as brands, $userFood1(count, userId :merge) as userFood, $portionSizes1(*:nest) as portionSizes",
"sort_by": "$userFood1(count:desc)"
}]}' | jqShaun Byrnes
10/27/2023, 9:24 AMShaun Byrnes
10/27/2023, 9:24 AMShaun Byrnes
10/27/2023, 9:25 AMHarpreet Sangar
10/27/2023, 10:26 AMShaun Byrnes
10/27/2023, 10:27 AMKishore Nallan
10/27/2023, 10:54 AM0.26.0.rc26 which contains these changes.Shaun Byrnes
10/27/2023, 10:55 AMKishore Nallan
10/27/2023, 10:55 AMShaun Byrnes
10/27/2023, 10:26 PMKishore Nallan
10/28/2023, 6:01 AMKishore Nallan
10/30/2023, 5:17 AMShaun Byrnes
10/30/2023, 5:18 AMShaun Byrnes
10/30/2023, 5:18 AMKishore Nallan
10/30/2023, 5:19 AMShaun Byrnes
10/30/2023, 8:53 AMShaun Byrnes
10/30/2023, 9:56 AM{
"created_at": 1698659635,
"default_sorting_field": "",
"enable_nested_fields": true,
"fields": [
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "uid",
"optional": false,
"sort": false,
"type": "string"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "name",
"optional": false,
"sort": false,
"type": "string"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "brandId",
"optional": false,
"reference": "brands1.uid",
"sort": false,
"type": "string"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "portions.portionSizeId",
"optional": false,
"reference": "portionSizes1.uid",
"sort": false,
"type": "string[]"
}
],
"name": "food1",
"num_documents": 0,
"symbols_to_index": [],
"token_separators": []
}
Then try and add this:
[
{
"uid": "1",
"name": "Food 1",
"brandId": "1",
"portions": [
{
"portionSizeId": "a",
"amount": 10
},
{
"portionSizeId": "b",
"amount": 590
}
]
}
]
And get this error:
There were some errors importing your documents.
[
{
"code": 400,
"document": "{\"uid\":\"1\",\"name\":\"Food 1\",\"brandId\":\"1\",\"portions\":[{\"portionSizeId\":\"1\",\"amount\":10},{\"portionSizeId\":\"2\",\"amount\":590}]}",
"error": "Missing the required reference field `portions.portionSizeId` in the document.",
"success": false
}
]Shaun Byrnes
10/30/2023, 9:58 AMShaun Byrnes
10/30/2023, 9:59 AMHarpreet Sangar
10/30/2023, 10:10 AMfoodPortionPrice collection that has reference to both food and portion. That collection would contain the price of a certain food's portion.
food foodPortionPrice portion
uid food_uid_reference uid
portion_uid_reference
price
following will get you the portions and their prices for the foods that match the query.
collection: food
q: *
filter_by: $foodPortionPrice(*)
include_fields: $foodPortionPrice(price), $portion(*)Shaun Byrnes
10/30/2023, 10:12 AMShaun Byrnes
10/30/2023, 10:12 AMShaun Byrnes
10/30/2023, 10:13 AMHarpreet Sangar
10/30/2023, 10:13 AMHarpreet Sangar
10/30/2023, 10:15 AM{
"portionSizeId": "a",
"amount": 10
}Shaun Byrnes
10/30/2023, 10:15 AMShaun Byrnes
10/30/2023, 10:16 AMShaun Byrnes
10/30/2023, 10:17 AMShaun Byrnes
10/30/2023, 10:17 AM"portionSize": {
"displayFormat": 0,
"portionsPerItem": null,
"selectedPortionIndex": 1,
"unit": "g",
"value": 1
},
"portions": [
{
"amount": 100,
"measureUnitId": "REzO2pZ0K1J6MFrCnWAi",
"multiple": 1,
"portionSizeId": "GEMDwd7SBYFZ6T9Ui0s2",
"type": 0
},
{
"amount": 152,
"measureUnitId": "GmwdaGtivG85KDyjohtj",
"multiple": 1.52,
"portionSizeId": "9UvEjhgOXKTITezpsPGu",
"type": 0
}
],Shaun Byrnes
10/30/2023, 10:18 AMportionSizeId we would wanna match the reference field to the portionSizes collection and merge in the portion name etcShaun Byrnes
10/30/2023, 10:18 AM"nutrients": [
{
"amount": 6.69,
"nutrientId": "ejXlabfZ7hM3N9oHF0KJ"
},
{
"amount": 1.25,
"nutrientId": "qlRUwsJ69Oyyy0DWfS07"
},
{
"amount": 42.8,
"nutrientId": "7eCtoVIpp0bVDtpi0aSG"
},
...
]Shaun Byrnes
10/30/2023, 10:19 AMnutrientId to get its name from the nutrients collection, merged in, as they are not just string[]Harpreet Sangar
10/30/2023, 10:22 AMShaun Byrnes
10/30/2023, 10:25 AMShaun Byrnes
10/30/2023, 10:25 AMShaun Byrnes
10/30/2023, 10:27 AMShaun Byrnes
10/30/2023, 10:28 AM{
"name": "portions.portionSizeId",
"reference": "portionSizes1.uid",
"type": "object[]"
}Shaun Byrnes
10/30/2023, 10:31 AM{
"name": "portions[].portionSizeId",
"reference": "portionSizes1.uid",
"type": "string[]"
}Shaun Byrnes
10/30/2023, 10:32 AMKishore Nallan
10/30/2023, 10:56 AMShaun Byrnes
10/30/2023, 11:23 AMShaun Byrnes
11/02/2023, 1:23 AMKishore Nallan
11/02/2023, 3:52 AMShaun Byrnes
11/02/2023, 5:55 AMShaun Byrnes
11/06/2023, 5:01 AMKishore Nallan
11/06/2023, 5:07 AMShaun Byrnes
11/06/2023, 5:15 AMShaun Byrnes
11/06/2023, 5:15 AMShaun Byrnes
11/15/2023, 2:48 AMKishore Nallan
11/15/2023, 2:48 AMKishore Nallan
11/15/2023, 4:42 PM0.26.0.rc34
We have updated instruction for using objects as references here: https://gist.github.com/kishorenc/037314619c3104970ae703b25aff2f51#reference-field-inside-an-objectShaun Byrnes
11/15/2023, 9:38 PMShaun Byrnes
11/15/2023, 11:50 PMlet searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "$userFood1(userId:s)",
"include_fields": "$brands1(name) as brands, $portionSizes1(name) as portionSizes, $userFood1(count, userId) as userFood",
"sort_by": "$userFood1(count:desc)"
};
let results = await client.collections("food1").documents().search(searchParameters);
Produces this result:
{
facet_counts: [],
found: 2,
hits: [
{
document: {
brandId: '1',
brands: { name: 'Brand one' },
id: '0',
name: 'Food 1',
portions: [
{
amount: 10,
portionSizeId: 'a',
portionSizes: { name: '1 cup' }
},
{
amount: 590.508,
portionSizeId: 'b',
portionSizes: { name: '1 glass' }
}
],
uid: '1',
userFood: { count: 20, userId: 's' }
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
},
{
document: {
brandId: '2',
brands: { name: 'Brand two' },
id: '2',
name: 'Food 2',
portions: [
{
amount: 1,
portionSizeId: 'c',
portionSizes: { name: '100ml' }
},
{
amount: 5,
portionSizeId: 'a',
portionSizes: { name: '1 cup' }
},
{
amount: 3,
portionSizeId: 'b',
portionSizes: { name: '1 glass' }
}
],
uid: '2',
userFood: { count: 4, userId: 's' }
},
highlight: {},
highlights: [],
text_match: 100,
text_match_info: {
best_field_score: '0',
best_field_weight: 12,
fields_matched: 4,
score: '100',
tokens_matched: 0
}
}
],
out_of: 2,
page: 1,
request_params: { collection_name: 'food1', per_page: 10, q: '' },
search_cutoff: false,
search_time_ms: 0
}
Pretty cool!Shaun Byrnes
11/16/2023, 12:48 AMKishore Nallan
11/16/2023, 1:10 AMShaun Byrnes
11/16/2023, 1:39 AMShaun Byrnes
11/16/2023, 2:20 AM{
"created_at": 1700091091,
"default_sorting_field": "count",
"enable_nested_fields": true,
"fields": [
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "userId",
"optional": false,
"sort": false,
"type": "string"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "count",
"optional": false,
"sort": true,
"type": "int32"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "foodId",
"optional": false,
"reference": "food1.uid",
"sort": false,
"type": "string"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "dateLastUsed",
"optional": false,
"sort": true,
"type": "int64"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "addedToMeals",
"optional": false,
"sort": false,
"type": "string[]"
}
],
"name": "userFood1",
"num_documents": 4
}Shaun Byrnes
11/16/2023, 2:24 AMSUM(count) GROUP BY foodId ? In that way we group all the foods which have been added and sum up the added count of each food, which would allow to see most added food's irrespective of userKishore Nallan
11/16/2023, 2:58 AMgroup_by=foodId along with facet_by=count ?
There will be a stats field returned as part of the facet response, which will have a sum.Shaun Byrnes
11/16/2023, 3:13 AM"group_by": "$userFood1(foodId)",Shaun Byrnes
11/16/2023, 3:13 AMRequest #1700104343962: Request to Node 0 failed due to "ECONNABORTED timeout of 3000ms exceeded"
Request #1700104343962: Sleeping for 0.1s and then retrying request...
Request #1700104343962: Request to Node 0 failed due to "ECONNABORTED timeout of 3000ms exceeded"
Request #1700104343962: Sleeping for 0.1s and then retrying request...
/Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/axios/lib/core/createError.js:16
var error = new Error(message);
^
Error: timeout of 3000ms exceeded
at createError (/Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/axios/lib/core/createError.js:16:15)
at RedirectableRequest.handleRequestTimeout (/Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/axios/lib/adapters/http.js:369:16)
at RedirectableRequest.emit (node:events:513:28)
at Timeout.<anonymous> (/Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/follow-redirects/index.js:169:12)
at listOnTimeout (node:internal/timers:564:17)
at process.processTimers (node:internal/timers:507:7) {
config: {
transitional: {
silentJSONParsing: true,
forcedJSONParsing: true,
clarifyTimeoutError: false
},
adapter: [Function: httpAdapter],
transformRequest: [ [Function: transformRequest] ],
transformResponse: [ [Function (anonymous)] ],
timeout: 3000,
xsrfCookieName: 'XSRF-TOKEN',
xsrfHeaderName: 'X-XSRF-TOKEN',
maxContentLength: Infinity,
maxBodyLength: Infinity,Kishore Nallan
11/16/2023, 3:14 AMShaun Byrnes
11/16/2023, 3:14 AMKishore Nallan
11/16/2023, 3:15 AMShaun Byrnes
11/16/2023, 3:15 AMShaun Byrnes
11/16/2023, 3:15 AMKishore Nallan
11/16/2023, 3:30 AMShaun Byrnes
11/16/2023, 3:32 AMShaun Byrnes
11/16/2023, 11:58 PMfood (docs in orange) and meals (docs in purple). Documents in both can optionally have a userId value, which would associate them with an end user who created them.
When we want to present a list of aggregated results that match the user such as userId=2 from both collections, which would be best approach?
At first glance federated search seems useful, but as its simple 2 seperate searches, with their own pagination, its not really useful for this. We need the results grouped into a single result set, so they can be sorted accordingly and paginated correctly. As it's simply a single list with "scroll down to load more" used to paginate.Shaun Byrnes
11/16/2023, 11:59 PMKishore Nallan
11/17/2023, 10:49 AMShaun Byrnes
11/17/2023, 10:50 AMKishore Nallan
11/17/2023, 10:54 AMShaun Byrnes
11/17/2023, 10:55 AMKishore Nallan
11/17/2023, 11:02 AMShaun Byrnes
11/17/2023, 12:11 PMKishore Nallan
11/17/2023, 12:12 PMShaun Byrnes
11/18/2023, 2:03 AMShaun Byrnes
11/29/2023, 1:00 PMKishore Nallan
11/30/2023, 3:50 AMShaun Byrnes
12/01/2023, 1:38 AMShaun Byrnes
12/01/2023, 1:38 AMShaun Byrnes
12/11/2023, 11:46 PMKishore Nallan
12/12/2023, 1:19 AMShaun Byrnes
12/12/2023, 2:25 AMShaun Byrnes
12/12/2023, 2:25 AMKishore Nallan
12/12/2023, 3:19 AMShaun Byrnes
12/12/2023, 3:20 AMShaun Byrnes
01/03/2024, 1:42 AMKishore Nallan
01/03/2024, 2:42 AMShaun Byrnes
01/03/2024, 11:23 PMShaun Byrnes
01/03/2024, 11:23 PMHarpreet Sangar
01/04/2024, 11:01 AMBascially trying to sort by all userFood counts highest to lowest@Shaun Byrnes In the latest RC build you can try this query now and it will return the brand names as well.
curl 'http://localhost:8108/multi_search' -X POST -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '
{"searches":[{
"collection": "userFood1",
"q": "*",
"filter_by": "$food1(id:*)",
"include_fields": "$brands1(*), $food1(*)",
"sort_by": "count:desc"
}]}' | jqShaun Byrnes
01/04/2024, 9:59 PMShaun Byrnes
01/23/2024, 4:59 AMShaun Byrnes
01/25/2024, 3:19 AMKishore Nallan
01/25/2024, 3:26 AMShaun Byrnes
01/29/2024, 6:01 AMKishore Nallan
01/29/2024, 6:05 AMShaun Byrnes
01/29/2024, 7:49 AMShaun Byrnes
01/30/2024, 3:01 AMper_page / limit alias). As in we still want pagination - such as 10 per page. But max results of 50. Then we can do max of 5 pages - sorted as we want, but limit the amount of results returned from server ?Kishore Nallan
01/30/2024, 3:07 AMlimit_hits
Maximum number of hits that can be fetched from the collection. Eg:200
should be less than this number for the search request to return results.page * per_page
Default: no limit
You'd typically want to generate a scoped API key with this parameter embedded and use that API key to perform the search, so it's automatically applied and can't be changed at search time.
Shaun Byrnes
01/30/2024, 3:07 AMShaun Byrnes
01/30/2024, 3:08 AMKishore Nallan
01/30/2024, 3:08 AMShaun Byrnes
01/30/2024, 3:10 AMShaun Byrnes
01/30/2024, 3:11 AMShaun Byrnes
01/30/2024, 3:11 AMShaun Byrnes
01/30/2024, 3:14 AM"limit_hits": 1,
"page": 2,
"per_page": 1Shaun Byrnes
01/30/2024, 3:14 AMObjectUnprocessable: Request failed with HTTP code 422 | Server said: Only upto 1 hits can be fetched. Ensure that `page` and `per_page` parameters are within this range.Shaun Byrnes
01/30/2024, 3:15 AMpage: 1 is happy, but changing to page: 2 throws that error - correct me if im wrong, but it should go to the result on the second page - if working as suggested ?Kishore Nallan
01/30/2024, 3:16 AMShaun Byrnes
01/30/2024, 3:17 AMShaun Byrnes
01/30/2024, 3:17 AMShaun Byrnes
01/30/2024, 3:17 AM"limit_hits": 10,
"page": 2,
"per_page": 1Shaun Byrnes
01/30/2024, 3:17 AMShaun Byrnes
01/30/2024, 3:17 AMShaun Byrnes
01/30/2024, 3:17 AMShaun Byrnes
01/30/2024, 3:20 AMJason Bosco
01/30/2024, 3:49 AMShaun Byrnes
01/30/2024, 3:57 AMShaun Byrnes
01/30/2024, 3:57 AMShaun Byrnes
01/30/2024, 3:57 AMJason Bosco
01/30/2024, 6:46 PMJason Bosco
01/30/2024, 6:46 PMShaun Byrnes
01/30/2024, 10:13 PMShaun Byrnes
01/30/2024, 10:50 PMlimit_hits and your suggestion for scoped API key. Quite a few questions or thoughts below as to how this could apply in our case ๐
Initially I wasn't going to bother on adding scoped key's as a further complexity if we didn't really need it (such as the limit fields is just for UX), but we also have some user related information - joined in to other collections - which gives the data for recently added/usage counts etc per user.
Currently we don't have any custom backend reliance for Typesense, as the full search API key (and marketplace extensions) have been sufficient to use on the client side till now as all TS collections were full public. However, from what I understand from the docs, if you have reliance on scoped keys for some kind of security purpose, you should not use the default full search API key on client anywhere - as this would simply bypass the point of scoping keys. Correct?
In which case, this key would be relocated to the backend to generate scoped keys from it correct? Given that 90% of all the collections are still completely public, I presume the scoped key would have the array of all collections listed except the one(s) which have userId etc. In this case it would mean that all the other collections which have no user related content and should be completely public would function as normal with scoped key returned, with the exception of those which have the reliance of the userId ? My concern was will those public collections get messed up with something like this returned for each user:
return client.keys().generateScopedSearchKey(searchOnlyAPIKey, {
"filter_by": `userId:=${userId}`,
// 1 month from now
"expires_at": Utils.getUnixTimestamp() + 2592000
});
Given `"filter_by": userId:=${userId},` the filter_by would bake in the userId for all requests. Or will collections that don't have a userId in schema be unaffected? I presume they will try and query by that if included and either throw an error or return no results? How will this work with queries that join collections (which is what we are doing). Consider the following example on client side at present:
const searchParameters = {
"q": "",
"query_by": "name",
"filter_by": "$userFood(userId:Mb2L5bAc1AbaZ6xPEkIjl60UDbr2)",
"include_fields": "$foodBrands(name) as foodBrand, $portionSizes(name) as portionSize, $userFood(count, dateLastUsed, addedToMeals)",
"sort_by": "$userFood(count:desc)",
"limit_hits": 50
};
const results = await client.collections("food").documents().search(searchParameters);
In the query above, only the userFood collection has a userId field - as it contains info about publicly accessible foods related to each user's use. Currently the userId is passed on client side - so technically anyone could just replace with another user's ID - to get their count for a food (which doesn't really matter, but I am thinking ahead for scenarios where this could have actual private data). If we were to go the way of scoped key, I assume we would set this `"filter_by": $userFood(userId:${userId}),` on the backend? I.e. Using the same syntax for reference collections, but just interpolating the user's ID server side, and will work ok?
In that case I expected maybe we need to send back 2 keys - one with full public collections listed in it (excluding any which have userId) and every user can fetch that scoped from backend for public access (to replace the current full search API key). Followed by another key they can use for user scoped collections - where their userId is included in the filter_by ?
Additionally, for expires_at it simply states in docs to pass a Unix timestamp - from what I could deduce, it should be in seconds - or it doesn't matter if its down to nanoseconds? Can we simply pass 0 for no expiry? Or must it be some time in the future always?Kishore Nallan
01/31/2024, 5:37 AM&& condition to other filter clauses you add in the query.
Additionally, forAll the keys already have a default expires_at set far into the future, so you don't need to specify anything if you don't want your keys to expire.it simply states in docs to pass a Unix timestamp - from what I could deduce, it should be in seconds - or it doesn't matter if its down to nanoseconds? Can we simply passexpires_atfor no expiry? Or must it be some time in the future always?0
Shaun Byrnes
02/01/2024, 3:43 AM&& condition on client? I.e. we can use the same syntax for reference collections like $collectionName(conditions) in the scoped key ?Shaun Byrnes
02/01/2024, 3:45 AM$userFood(userId:${userId}),` to ensure that the referenced userFood collection matches only on userId set on backend to generate scoped keyKishore Nallan
02/01/2024, 3:52 AMShaun Byrnes
02/01/2024, 4:55 AMShaun Byrnes
02/01/2024, 4:58 AM* explicitly lists all the full public collections - then use that in place on client side for all the full public access. Then send another scoped key with the userId baked in for the specific collections which need that ?Kishore Nallan
02/01/2024, 4:59 AMKishore Nallan
02/01/2024, 5:00 AMShaun Byrnes
02/01/2024, 5:33 AMShaun Byrnes
02/01/2024, 5:34 AMShaun Byrnes
02/01/2024, 5:36 AMKishore Nallan
02/01/2024, 5:49 AMKishore Nallan
02/01/2024, 5:49 AMShaun Byrnes
02/01/2024, 5:50 AMShaun Byrnes
02/01/2024, 5:50 AMShaun Byrnes
02/01/2024, 5:59 AMShaun Byrnes
02/12/2024, 5:38 AMclient.keys().create({
"description": "Key for searching org collections.",
"actions": [
"documents:search"
],
"collections": [
"org_.*"
]
})
Are we free to use any regex? Like can we do:
"collections": [
"^(?!userFood).*$"
]
Ie any collection not starting with userFood ?Shaun Byrnes
02/12/2024, 5:42 AMpublic_ except there seems to be no rename function for existing collections in TS Cloud, meaning we would have to delete each collection and re-create it with the new name like public_collectionName? Furthermore, each env has marketplace extensions setup for each collection, each of which would need to be re-configured with the new TS collection name, followed by backfilling all the docs, am I correct on this?Kishore Nallan
02/12/2024, 6:26 AMKishore Nallan
02/12/2024, 6:27 AMShaun Byrnes
02/12/2024, 9:39 PMfood aliased to pub_food can we do
"collections": [
"pub_.*"
]Kishore Nallan
02/13/2024, 1:27 AMShaun Byrnes
02/13/2024, 10:30 AMShaun Byrnes
02/13/2024, 10:30 AMShaun Byrnes
02/13/2024, 10:31 AMKishore Nallan
02/13/2024, 10:33 AMKishore Nallan
02/13/2024, 10:35 AMfoo to real collection bar and then you make a query with /collections/foo -> the API key associated with the request MUST have foo in allowed collection name. (it could be a regexp like f.* also)Shaun Byrnes
02/13/2024, 10:35 AMShaun Byrnes
02/13/2024, 10:36 AMf.* matching that alias would be good - provided that was added to the collections[] of the key used for the queryKishore Nallan
02/13/2024, 10:38 AMShaun Byrnes
02/15/2024, 2:21 PMShaun Byrnes
02/15/2024, 2:21 PMShaun Byrnes
03/05/2024, 8:14 AMnullShaun Byrnes
03/05/2024, 8:15 AM{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "foodBrandId",
"optional": true,
"reference": "foodBrands.uid",
"sort": false,
"type": "string"
},Shaun Byrnes
03/05/2024, 8:15 AMoptional: true then null should be OK? doesn't need to be saved as ""Shaun Byrnes
03/05/2024, 8:16 AMShaun Byrnes
03/05/2024, 8:16 AMerror: 'Field `foodBrandId` must have `string` value.',
success: falseShaun Byrnes
03/05/2024, 8:16 AMKishore Nallan
03/05/2024, 8:35 AMKishore Nallan
03/05/2024, 8:36 AMShaun Byrnes
03/05/2024, 8:36 AMShaun Byrnes
03/05/2024, 8:36 AMShaun Byrnes
03/05/2024, 8:36 AMShaun Byrnes
03/05/2024, 9:48 AMShaun Byrnes
03/05/2024, 9:48 AMShaun Byrnes
03/05/2024, 9:55 AMKishore Nallan
03/05/2024, 9:56 AMShaun Byrnes
03/05/2024, 9:56 AMShaun Byrnes
03/05/2024, 9:56 AMShaun Byrnes
03/05/2024, 10:10 AMTypesense v0.26.0.rc59Shaun Byrnes
03/05/2024, 10:10 AMShaun Byrnes
03/05/2024, 10:11 AMImport error ImportError: 3 documents imported successfully, 2 documents failed during import. Use `error.importResults` from the raised exception to get a detailed error reason for each document.
at ImportError.TypesenseError [as constructor] (/workspace/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23:28)
at new ImportError (/workspace/node_modules/typesense/lib/Typesense/Errors/ImportError.js:25:28)
at Documents.<anonymous> (/workspace/node_modules/typesense/lib/Typesense/Documents.js:160:39)
at step (/workspace/node_modules/typesense/lib/Typesense/Documents.js:48:23)
at Object.next (/workspace/node_modules/typesense/lib/Typesense/Documents.js:29:53)
at fulfilled (/workspace/node_modules/typesense/lib/Typesense/Documents.js:20:58)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
importResults: [
{ success: true },
{ success: true },
{
code: 400,
document: '{"foodBrandId":null,"notes":null,"preparationTime":null,"verified":false,"userId":"kPKKgXOyiGXnv21I5DL4cHNXiFu2","cookingTime":null,"uid":"M8KWqkMWPttyxb9yN5Wv","baseUid":"M8KWqkMWPttyxb9yN5Wv","createdDate":1708911965,"deleted":true,"directions":[],"parts":[],"name":"Meal for good","foodCategories":[],"dietaryTypes":null,"portionSize":{"unit":"g","selectedPortionIndex":0,"displayFormat":0,"value":1,"portionsPerItem":null},"itemWeight":null,"portions":[{"amount":null,"multiple":1,"type":0,"measureUnitId":"REzO2pZ0K1J6MFrCnWAi","portionSizeId":"qO6fez6gHhO8SQ8riQ89"}],"id":"M8KWqkMWPttyxb9yN5Wv"}',
error: 'Field `foodBrandId` must have `string` value.',
success: false
},
{ success: true },
{
code: 400,
document: '{"foodBrandId":null,"verified":false,"preparationTime":null,"userId":"tS1RgdIiRYY5VFgWL5P1I7EI9UN2","cookingTime":null,"uid":"qP9oFHBbgBNPAEFnvw8d","directions":[],"name":"meal1","foodCategories":[],"dietaryTypes":null,"portionSize":{"unit":"g","selectedPortionIndex":0,"displayFormat":0,"value":1,"portionsPerItem":null},"itemWeight":null,"portions":[{"amount":null,"multiple":1,"type":0,"measureUnitId":"GmwdaGtivG85KDyjohtj","portionSizeId":"9L8ZLOBNTw4jUUxOhmQY"}],"parts":[{"uid":"R2EswNcBNwRtuVcQleuH","name":"Main","items":[{"uid":"HSOmGn34fwr0viVYlaPj","type":0},{"uid":"N9oRfnPxD1ECLuE4f7Xa","type":0}]}],"createdDate":1708336556,"deleted":false,"id":"qP9oFHBbgBNPAEFnvw8d"}',
error: 'Field `foodBrandId` must have `string` value.',
success: false
}
]
}Kishore Nallan
03/05/2024, 10:12 AMShaun Byrnes
03/05/2024, 10:12 AMShaun Byrnes
03/05/2024, 10:12 AMShaun Byrnes
03/05/2024, 10:12 AMKishore Nallan
03/05/2024, 10:13 AMShaun Byrnes
03/05/2024, 10:13 AMShaun Byrnes
03/05/2024, 10:13 AMShaun Byrnes
03/05/2024, 10:13 AMKishore Nallan
03/05/2024, 10:13 AMShaun Byrnes
03/06/2024, 12:54 AMKishore Nallan
03/06/2024, 1:23 AMShaun Byrnes
03/06/2024, 5:40 AMMalik
03/07/2024, 5:49 AMKishore Nallan
03/07/2024, 6:14 AMKishore Nallan
03/07/2024, 6:14 AMShaun Byrnes
03/07/2024, 7:48 AMShaun Byrnes
03/07/2024, 9:47 AMpub_portionSizes collection. Expected portionSize to be an array. Try renaming the alias.Shaun Byrnes
03/07/2024, 9:47 AMHarpreet Sangar
03/07/2024, 9:53 AMShaun Byrnes
03/07/2024, 10:44 AMHarpreet Sangar
03/07/2024, 10:46 AMinclude_fields like as foo.Shaun Byrnes
03/07/2024, 10:47 AMShaun Byrnes
03/07/2024, 10:49 AMlet searchParametersAll = {
"q": "",
"query_by": "name",
"filter_by": "deleted:false",
"include_fields": "$foodBrands(name) as foodBrand, $portionSizes(name) as portionSize"
};
let results = await client.collections("food").documents().search(searchParametersAll);Shaun Byrnes
03/07/2024, 10:49 AMRequestMalformed: Request failed with HTTP code 400 | Server said: Could not include the reference document of `portionSizes` collection. Expected `portionSize` to be an array. Try renaming the alias.
at RequestMalformed.TypesenseError [as constructor] (/Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23:28)
at new RequestMalformed (/Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/typesense/lib/Typesense/Errors/RequestMalformed.js:25:42)
at ApiCall.customErrorForResponse (/Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/typesense/lib/Typesense/ApiCall.js:332:21)
at /Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/typesense/lib/Typesense/ApiCall.js:199:98
at step (/Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/typesense/lib/Typesense/ApiCall.js:33:23)
at Object.next (/Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/typesense/lib/Typesense/ApiCall.js:14:53)
at step (/Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/typesense/lib/Typesense/ApiCall.js:18:139)
at Object.next (/Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/typesense/lib/Typesense/ApiCall.js:14:53)
at fulfilled (/Users/shaunbyrnes/Software/shredded-platform/shredded-firebase/utils/temp/node_modules/typesense/lib/Typesense/ApiCall.js:5:58)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
httpStatus: 400
}
Node.js v18.12.1Shaun Byrnes
03/07/2024, 10:50 AMTypesense v0.26.0.rc62 from rc58Shaun Byrnes
03/07/2024, 10:50 AMShaun Byrnes
03/07/2024, 10:54 AM$portionSizes(name) as portionSize > $portionSizes(name)Shaun Byrnes
03/07/2024, 10:54 AMShaun Byrnes
03/07/2024, 10:54 AM$pub_portionSizes(name)
But this does not $pub_portionSizes(name) as portionSizeKishore Nallan
03/07/2024, 10:55 AMShaun Byrnes
03/07/2024, 10:55 AMShaun Byrnes
03/07/2024, 10:55 AMShaun Byrnes
03/07/2024, 10:55 AMShaun Byrnes
03/07/2024, 10:55 AMShaun Byrnes
03/07/2024, 10:55 AM$foodBrands(name) as foodBrand is still working fineHarpreet Sangar
03/07/2024, 10:57 AM$pub_portionSizes(name) as portionSize to $pub_portionSizes(name) as p or even $portionSizes(name) as p. Trying to determine if the alias name is actually causing the issue here.Shaun Byrnes
03/07/2024, 10:57 AMShaun Byrnes
03/07/2024, 10:58 AM$pub_portionSizes(name) as p and $portionSizes(name) as p workHarpreet Sangar
03/07/2024, 11:01 AMfoods document must have a field named portionSize and it must be referencing multiple portion sizes. Renaming the alias in include_fields should fix this. I'm surprized this was working in previous RC build.Shaun Byrnes
03/07/2024, 11:02 AMShaun Byrnes
03/07/2024, 11:02 AMShaun Byrnes
03/07/2024, 11:02 AMShaun Byrnes
03/07/2024, 11:02 AMportionSize
{ "displayFormat": 0, "portionsPerItem": 25.4, "selectedPortionIndex": 0, "unit": "g", "value": 1 }Shaun Byrnes
03/07/2024, 11:03 AMShaun Byrnes
03/07/2024, 11:03 AMHarpreet Sangar
03/07/2024, 11:03 AMShaun Byrnes
03/07/2024, 11:03 AMShaun Byrnes
03/07/2024, 11:04 AMShaun Byrnes
03/07/2024, 11:04 AMHarpreet Sangar
03/07/2024, 11:08 AMnest as the default. Earlier it was merge. So I think $portionSizes(name :merge) as portionSize will be a better fix for you.Shaun Byrnes
03/07/2024, 11:18 AMShaun Byrnes
03/07/2024, 11:18 AMShaun Byrnes
03/07/2024, 11:19 AM"include_fields": "$pub_foodBrands(name, deleted, verified, userId) as foodBrand, $pub_portionSizes(name, verified, deleted, userId) as portionSize"Shaun Byrnes
03/07/2024, 11:19 AMportions: [
{
amount: 100,
measureUnitId: 'zDyQbolmfqHvru7I8YHs',
multiple: 1,
portionSize: { deleted: false, name: '100ml', verified: true },
portionSizeId: 'y7Y7sWBoCUJeVMj95RcC',
type: 0
},
{
amount: 450,
measureUnitId: 'zDyQbolmfqHvru7I8YHs',
multiple: 4.5,
portionSize: { deleted: false, name: '15.2oz', verified: true },
portionSizeId: 'pOVPgpOOtYclVT0u2Ocf',
type: 0
}
]Shaun Byrnes
03/07/2024, 11:19 AMHarpreet Sangar
03/07/2024, 11:22 AMShaun Byrnes
03/07/2024, 11:22 AMShaun Byrnes
03/07/2024, 11:23 AMShaun Byrnes
03/07/2024, 2:21 PMKishore Nallan
03/07/2024, 2:22 PMShaun Byrnes
03/07/2024, 2:23 PMShaun Byrnes
03/07/2024, 2:23 PMKishore Nallan
03/07/2024, 2:23 PMShaun Byrnes
03/07/2024, 2:24 PMKishore Nallan
03/07/2024, 2:24 PMShaun Byrnes
03/07/2024, 2:24 PMShaun Byrnes
03/07/2024, 2:24 PMKishore Nallan
03/07/2024, 2:24 PMKishore Nallan
03/07/2024, 2:25 PMShaun Byrnes
03/07/2024, 2:26 PMShaun Byrnes
03/07/2024, 2:26 PMShaun Byrnes
03/07/2024, 2:26 PMShaun Byrnes
03/07/2024, 2:26 PMShaun Byrnes
03/07/2024, 2:26 PMShaun Byrnes
03/07/2024, 2:26 PMShaun Byrnes
03/07/2024, 2:35 PMKishore Nallan
03/07/2024, 2:39 PMShaun Byrnes
03/07/2024, 2:40 PMJason Bosco
03/07/2024, 2:55 PMJason Bosco
03/07/2024, 2:56 PMShaun Byrnes
03/07/2024, 3:11 PMShaun Byrnes
03/09/2024, 10:38 AMJason Bosco
03/09/2024, 8:46 PMJason Bosco
03/12/2024, 4:04 AMShaun Byrnes
03/12/2024, 1:10 PMShaun Byrnes
03/12/2024, 1:11 PMShaun Byrnes
03/12/2024, 1:12 PMx-typesense-api-key header must be sent.Shaun Byrnes
03/12/2024, 1:12 PMJason Bosco
03/12/2024, 6:57 PMIs that for the extensions ?Correct
Jason Bosco
03/12/2024, 6:58 PMJason Bosco
03/12/2024, 6:58 PMJason Bosco
03/12/2024, 6:59 PMAlso, when using the scoped key for TS request (swapped out the default public search key ) we get this
Request failed with status code: 401.Server said: Forbidden - a validYou want to check if the parent API key has sufficient permissions to access the collectionheader must be sent.x-typesense-api-key
Shaun Byrnes
03/12/2024, 10:52 PMShaun Byrnes
03/12/2024, 10:59 PMShaun Byrnes
03/12/2024, 11:03 PMDescription Key Prefix Actions Collections
Search only key for all public collections zHAV [ "documents:search" ] [ "pub_.+" ]Shaun Byrnes
03/12/2024, 11:03 PMShaun Byrnes
03/12/2024, 11:26 PMconst publicSearchAPIKey = typesensePublicSearchAPIKey.value(); // zHAV...
result.public = client.keys().generateScopedSearchKey(publicSearchAPIKey, paramaters);Kishore Nallan
03/13/2024, 4:52 AMShaun Byrnes
03/13/2024, 6:01 AMShaun Byrnes
03/13/2024, 6:01 AMShaun Byrnes
03/13/2024, 8:22 AM//config
String publicKey = "aEdySm************************"
Configuration configuration = new Configuration(nodes, Duration.ofSeconds(2), publicKey);
client = new Client(configuration);
//search query - search portionSize with given uid
SearchParameters searchParameters = new SearchParameters()
.q("fH6H6i8g8Dplf4kPUf3b")
.queryBy("uid")
.filterBy("deleted:=false")
.sortBy("name: asc")
.page(1)
.perPage(20);
return client.collections("portionSizes").documents().search(searchParameters);
//error code:
org.typesense.api.exceptions.RequestUnauthorized:
Request failed with status code: 401.Server said: Forbidden - a valid `x-typesense-api-key` header must be sent.
//note
this same request works with default API key โNtWM***โShaun Byrnes
03/13/2024, 8:23 AMKishore Nallan
03/13/2024, 8:26 AMShaun Byrnes
03/13/2024, 8:28 AMShaun Byrnes
03/13/2024, 8:28 AM//config
String publicKey = "aEdySm************************"
Configuration configuration = new Configuration(nodes, Duration.ofSeconds(2), publicKey);
client = new Client(configuration);
//search query with join
SearchParameters searchParameters = new SearchParameters()
.q("")
.queryBy("name")
.filterBy("deleted:=false")
.includeFields("$pub_foodBrands(deleted, name, uid, verified, userId) as foodBrand, $pub_portionSizes(uid, name, verified, deleted, measureUnitId, userId) as portionSizes")
.page(1)
.perPage(20);
return client.collections("food").documents().search(searchParameters);
//error code:
org.typesense.api.exceptions.RequestUnauthorized:
Request failed with status code: 401.Server said: Forbidden - a valid `x-typesense-api-key` header must be sent.
//note
this same request works with default API key โNtWM***โShaun Byrnes
03/22/2024, 12:28 AM"include_fields": "$pub_foodBrands(name, deleted, verified, userId) as foodBrand, $pub_portionSizes(name, verified, deleted, userId) as portionSize"
which gave this output (which was perfect):
...
portions: [
{
amount: 100,
measureUnitId: 'zDyQbolmfqHvru7I8YHs',
multiple: 1,
portionSize: { deleted: false, name: '100ml', verified: true },
portionSizeId: 'y7Y7sWBoCUJeVMj95RcC',
type: 0
},
{
amount: 450,
measureUnitId: 'zDyQbolmfqHvru7I8YHs',
multiple: 4.5,
portionSize: { deleted: false, name: '15.2oz', verified: true },
portionSizeId: 'pOVPgpOOtYclVT0u2Ocf',
type: 0
}
]
...Shaun Byrnes
03/22/2024, 12:42 AMShaun Byrnes
03/22/2024, 12:44 AMportionSizess to avoid conflict I now get this:
portionSizess: [
{ deleted: false, name: '1 cup, pieces', verified: true },
{ deleted: false, name: '100g', verified: true },
{ deleted: false, name: '1 slice, NFS', verified: true },
{
deleted: false,
name: '1 Guideline amount on regular sandwich',
verified: true
},
{
deleted: false,
name: '1 Guideline amount on large sandwich',
verified: true
},
{
deleted: false,
name: '1 Quantity not specified',
verified: true
}
],
portions: [
{
amount: 100,
measureUnitId: 'REzO2pZ0K1J6MFrCnWAi',
multiple: 1,
portionSizeId: 'GEMDwd7SBYFZ6T9Ui0s2',
type: 0
},
{
amount: 56,
measureUnitId: 'GmwdaGtivG85KDyjohtj',
multiple: 0.56,
portionSizeId: 'Z14eBJrHfEDxqwUMfUhX',
type: 0
},
{
amount: 28,
measureUnitId: 'GmwdaGtivG85KDyjohtj',
multiple: 0.28,
portionSizeId: 'HCU3MoLa3YFsCZsos9MD',
type: 0
},
{
amount: 84,
measureUnitId: 'GmwdaGtivG85KDyjohtj',
multiple: 0.84,
portionSizeId: 'eoE3YE153eqJKmiSRSvL',
type: 0
},
{
amount: 140,
measureUnitId: 'GmwdaGtivG85KDyjohtj',
multiple: 1.4,
portionSizeId: '68tKa9ViNktfxpKL4fh0',
type: 0
},
{
amount: 56,
measureUnitId: 'GmwdaGtivG85KDyjohtj',
multiple: 0.56,
portionSizeId: 'mTrUr1NUxN8rfo3iTiOG',
type: 0
}
],
They are now 2 separate arrays (instead of as before)...Shaun Byrnes
03/22/2024, 12:47 AMShaun Byrnes
03/22/2024, 4:01 AMKishore Nallan
03/22/2024, 4:03 AMKishore Nallan
03/22/2024, 4:05 AMShaun Byrnes
03/22/2024, 4:05 AMShaun Byrnes
03/22/2024, 4:05 AMKishore Nallan
03/22/2024, 5:01 AMShaun Byrnes
03/22/2024, 5:02 AMShaun Byrnes
03/22/2024, 5:04 AMHarpreet Sangar
03/22/2024, 5:20 AMShaun Byrnes
03/22/2024, 6:21 AMKishore Nallan
03/22/2024, 6:21 AMinclude_fields that you have pasted above.Shaun Byrnes
03/22/2024, 6:21 AMKishore Nallan
03/22/2024, 6:22 AMShaun Byrnes
03/22/2024, 6:22 AMShaun Byrnes
03/22/2024, 6:23 AMKishore Nallan
03/22/2024, 6:24 AMShaun Byrnes
03/22/2024, 7:13 AMlet searchParametersAll = {
"q": "",
"query_by": "name",
"filter_by": "deleted:false",
"include_fields": "$foodBrands(name, deleted, verified, userId) as foodBrand, $portionSizes(name, verified, deleted, userId) as portionSizess"
};
let results = await client.collections("food").documents().search(searchParametersAll);Shaun Byrnes
03/22/2024, 7:17 AMfood.portions[] array called portionSizess we get an array of objects food.portionSizes[]Harpreet Sangar
03/22/2024, 7:29 AM"include_fields": "$foodBrands(name, deleted, verified, userId) as foodBrand, $portionSizes(name, verified, deleted, userId : merge)"Shaun Byrnes
03/22/2024, 7:29 AMShaun Byrnes
03/22/2024, 7:31 AMShaun Byrnes
03/22/2024, 7:31 AMShaun Byrnes
03/22/2024, 7:31 AMShaun Byrnes
03/22/2024, 7:31 AM'p.deleted': [ false, false, false, false, false, false ],
'p.name': [
'1 cup, pieces',
'100g',
'1 slice, NFS',
'1 Guideline amount on regular sandwich',
'1 Guideline amount on large sandwich',
'1 Quantity not specified'
],
'p.verified': [ true, true, true, true, true, true ],Shaun Byrnes
03/22/2024, 7:35 AM:merge was default behaviour)Shaun Byrnes
03/22/2024, 7:36 AM:nest behavior was better as it would put an object inside the joined/referenced arrayShaun Byrnes
03/22/2024, 7:37 AMportions[].referenceCollectionName{}Shaun Byrnes
03/22/2024, 7:37 AMHarpreet Sangar
03/22/2024, 7:42 AM{
"name": "Foods",
"fields": [
{
"name": "name",
"type": "string"
},
{
"name": "portions",
"type": "object[]"
},
{
"name": "portions.portion_id",
"type": "string[]",
"reference": "Portions.portion_id",
"optional": true
}
],
"enable_nested_fields": true
}
{
"name": "Portions",
"fields": [
{
"name": "portion_id",
"type": "string"
},
{
"name": "quantity",
"type": "int32"
},
{
"name": "unit",
"type": "string"
}
]
}
Indexed these documents:
{
"portion_id": "portion_a",
"quantity": 500,
"unit": "g"
}
{
"portion_id": "portion_b",
"quantity": 1,
"unit": "lt"
}
{
"portion_id": "portion_c",
"quantity": 500,
"unit": "ml"
}
and this food:
{
"name": "Milk",
"portions": [
{
"portion_id": "portion_b",
"count": 3
},
{
"count": 3
},
{
"portion_id": "portion_c",
"count": 1
}
]
}
For
{"collection", "Foods"},
{"q", "*"},
{"include_fields", "$Portions(*:merge)"}
I receive
{
"name": "Milk",
"portions": [
{
"count": 3,
"id": "1",
"portion_id": "portion_b",
"quantity": 1,
"unit": "lt"
},
{
"count": 3
},
{
"count": 1,
"id": "2",
"portion_id": "portion_c",
"quantity": 500,
"unit": "ml"
}
]
}
with :merge.Harpreet Sangar
03/22/2024, 7:43 AMShaun Byrnes
03/22/2024, 7:44 AMShaun Byrnes
03/22/2024, 7:44 AM{
"created_at": 1709820691,
"default_sorting_field": "",
"enable_nested_fields": true,
"fields": [
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "foodCategories",
"optional": true,
"sort": false,
"stem": false,
"type": "string[]"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "name",
"optional": false,
"sort": true,
"stem": false,
"type": "string"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "deleted",
"optional": false,
"sort": false,
"stem": false,
"type": "bool"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "verified",
"optional": false,
"sort": false,
"stem": false,
"type": "bool"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "water",
"optional": false,
"sort": false,
"stem": false,
"type": "bool"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "supplement",
"optional": false,
"sort": false,
"stem": false,
"type": "bool"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "fdcId",
"optional": true,
"sort": false,
"stem": false,
"type": "int32"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "userId",
"optional": true,
"sort": false,
"stem": false,
"type": "string"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "uid",
"optional": false,
"sort": false,
"stem": false,
"type": "string"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "foodBrandId",
"optional": true,
"reference": "foodBrands.uid",
"sort": false,
"stem": false,
"type": "string"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "portions",
"optional": false,
"sort": false,
"stem": false,
"type": "object[]"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "portions.portionSizeId",
"optional": false,
"reference": "portionSizes.uid",
"sort": false,
"stem": false,
"type": "string[]"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "portions.amount",
"optional": true,
"sort": false,
"stem": false,
"type": "float[]"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "portions.multiple",
"optional": false,
"sort": false,
"stem": false,
"type": "float[]"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "portions.type",
"optional": false,
"sort": false,
"stem": false,
"type": "int32[]"
},
{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "portions.measureUnitId",
"optional": true,
"sort": false,
"stem": false,
"type": "string[]"
}
],
"name": "food",
"num_documents": 20175,
"symbols_to_index": [],
"token_separators": []
}Kishore Nallan
03/22/2024, 7:53 AMKishore Nallan
03/22/2024, 7:53 AMShaun Byrnes
03/22/2024, 8:00 AMKishore Nallan
03/23/2024, 9:38 AMShaun Byrnes
03/23/2024, 9:49 AMKishore Nallan
03/23/2024, 4:42 PM0.26.0.rc67 and check.Shaun Byrnes
03/24/2024, 1:28 AMKishore Nallan
03/24/2024, 2:23 AMShaun Byrnes
03/24/2024, 3:37 AMShaun Byrnes
03/24/2024, 7:47 AMShaun Byrnes
03/24/2024, 7:47 AM:nest and :mergeShaun Byrnes
03/24/2024, 7:47 AMShaun Byrnes
03/26/2024, 7:51 AMShaun Byrnes
03/26/2024, 7:52 AMShaun Byrnes
03/26/2024, 7:52 AMID โ Description Key Prefix Actions Collections Expires At
8 Search only key for all user collections u9RB [ "documents:search" ] [ "user_.+" ] 01/01/4021, 10:59:59Shaun Byrnes
03/26/2024, 7:55 AMQWJLMFVkVFlNeUx3dDh4aVBwbDJGbHlGdEFObEFNTmQ4bjYrL1dCNzRHRT11OVJCeyJleHBpcmVzX2F0IjoxNzE0MDMwMDI0LCJmaWx0ZXJfYnkiOiIkdXNlcl91c2VyRm9vZCh1c2VySWQ6PXRTMVJnZElpUllZNVZGZ1dMNVAxSTdFSTlVTjIpIiwibGltaXRfaGl0cyI6NTB9Shaun Byrnes
03/26/2024, 7:55 AMAbK0UdTYMyLwt8xiPpl2FlyFtANlAMNd8n6+/WB74GE=u9RB{"expires_at":1714030024,"filter_by":"$user_userFood(userId:=tS1RgdIiRYY5VFgWL5P1I7EI9UN2)","limit_hits":50}Shaun Byrnes
03/26/2024, 7:55 AMShaun Byrnes
03/26/2024, 7:55 AMShaun Byrnes
03/26/2024, 7:56 AMorg.typesense.api.exceptions.RequestUnauthorized: Request failed with status code: 401.Server said: Forbidden - a valid `x-typesense-api-key` header must be sent.Shaun Byrnes
03/26/2024, 7:56 AMShaun Byrnes
03/26/2024, 7:56 AMShaun Byrnes
03/26/2024, 7:58 AM&& $user_userFood(userId:=tS1RgdIiRYY5VFgWL5P1I7EI9UN2) to the filter by clause sent locally from client, get results back - so I dont see its a malformed query eitherShaun Byrnes
03/26/2024, 7:58 AMKishore Nallan
03/26/2024, 9:34 AMShaun Byrnes
03/26/2024, 9:34 AMShaun Byrnes
03/26/2024, 9:34 AMKishore Nallan
03/26/2024, 9:35 AMShaun Byrnes
03/26/2024, 9:36 AMShaun Byrnes
03/26/2024, 9:36 AMShaun Byrnes
03/26/2024, 9:36 AMKishore Nallan
03/26/2024, 9:36 AMu9RBShaun Byrnes
03/26/2024, 9:36 AMShaun Byrnes
03/26/2024, 9:37 AMKishore Nallan
03/26/2024, 9:37 AMShaun Byrnes
03/26/2024, 9:37 AMShaun Byrnes
03/26/2024, 9:38 AMShaun Byrnes
03/27/2024, 2:28 AMKishore Nallan
03/27/2024, 5:32 AMKishore Nallan
03/27/2024, 5:33 AM$user_userFood --> what is name of the primary collection queried?Shaun Byrnes
03/27/2024, 5:35 AMShaun Byrnes
03/27/2024, 5:36 AMpub_foodShaun Byrnes
03/27/2024, 5:36 AMfood)Shaun Byrnes
03/27/2024, 5:37 AMKishore Nallan
03/27/2024, 5:39 AMShaun Byrnes
03/27/2024, 5:39 AMKishore Nallan
03/27/2024, 5:47 AMShaun Byrnes
03/27/2024, 5:53 AMKishore Nallan
03/27/2024, 6:04 AMKishore Nallan
03/27/2024, 6:05 AMuser_.+ but here we are querying against pub_foodShaun Byrnes
03/27/2024, 6:06 AMShaun Byrnes
03/27/2024, 6:06 AMKishore Nallan
03/27/2024, 6:06 AMShaun Byrnes
03/27/2024, 6:06 AMShaun Byrnes
03/27/2024, 6:07 AMShaun Byrnes
03/27/2024, 6:07 AMKishore Nallan
03/27/2024, 6:08 AMShaun Byrnes
03/27/2024, 6:08 AMShaun Byrnes
03/27/2024, 6:08 AMShaun Byrnes
03/27/2024, 6:09 AMShaun Byrnes
03/27/2024, 6:09 AMKishore Nallan
03/27/2024, 6:09 AMKishore Nallan
03/27/2024, 6:09 AMShaun Byrnes
03/27/2024, 6:10 AMShaun Byrnes
03/27/2024, 6:10 AMShaun Byrnes
03/27/2024, 6:10 AMKishore Nallan
03/27/2024, 6:11 AMShaun Byrnes
03/27/2024, 6:11 AMShaun Byrnes
03/27/2024, 6:11 AMKishore Nallan
03/27/2024, 6:11 AMShaun Byrnes
03/27/2024, 6:11 AMKishore Nallan
03/27/2024, 6:11 AMShaun Byrnes
03/27/2024, 6:11 AMKishore Nallan
03/27/2024, 6:12 AMShaun Byrnes
03/27/2024, 6:12 AMShaun Byrnes
03/27/2024, 6:12 AM[ "pub_.+" ]Shaun Byrnes
03/27/2024, 6:12 AMShaun Byrnes
03/27/2024, 6:13 AMpub_foodShaun Byrnes
03/27/2024, 6:13 AM$user_userFood(userId:=x)Kishore Nallan
03/27/2024, 6:13 AMShaun Byrnes
03/27/2024, 6:14 AMpub_food would allow ?Shaun Byrnes
03/27/2024, 6:14 AMKishore Nallan
03/27/2024, 6:15 AMShaun Byrnes
03/27/2024, 6:16 AMShaun Byrnes
03/27/2024, 6:16 AMpub_foodShaun Byrnes
03/27/2024, 6:17 AMuser_Shaun Byrnes
03/27/2024, 6:17 AM[ "user_.+" ]Shaun Byrnes
03/27/2024, 6:17 AMuser_userFoodKishore Nallan
03/27/2024, 6:18 AMuser_.+ and then be able to execute this query?Shaun Byrnes
03/27/2024, 6:19 AMShaun Byrnes
03/27/2024, 6:20 AM[ "pub_.+" ] thereby stopping any user from querying collections aliased with user_ etc.Shaun Byrnes
03/27/2024, 6:20 AM$user_userFood(userId:=x) includedShaun Byrnes
03/27/2024, 6:20 AMpub_food (which is allowed based on parent)Shaun Byrnes
03/27/2024, 6:21 AMuser_userFood inside the filterby will work, because its already explicitly defined in food schemaKishore Nallan
03/27/2024, 6:21 AMShaun Byrnes
03/27/2024, 6:21 AMuserFood / user_userFood alias directly, it wont workShaun Byrnes
03/27/2024, 6:22 AMpub_Kishore Nallan
03/27/2024, 6:22 AMShaun Byrnes
03/27/2024, 6:22 AMShaun Byrnes
03/27/2024, 6:22 AMShaun Byrnes
03/27/2024, 6:23 AM8 Search only key for all user collections u9RB [ "documents:search" ] [ "user_.+" ]
can still remain, but not in useShaun Byrnes
03/27/2024, 6:23 AMKishore Nallan
03/27/2024, 6:23 AMKishore Nallan
03/27/2024, 6:23 AMShaun Byrnes
03/27/2024, 6:24 AMShaun Byrnes
03/27/2024, 6:24 AMparamaters = { ...defaultParamaters, ...{ filter_by: `$user_userFood(userId:=${userId})`, limit_hits: 50 } };
result.userFood = client.keys().generateScopedSearchKey(userSearchAPIKey, paramaters);
which we need to change userSearchAPIKey > publicSearchAPIKey i thinkKishore Nallan
03/27/2024, 6:25 AMShaun Byrnes
03/27/2024, 6:25 AMpub_ allowed for collections, thus, searching pub_food or pub_meals will succeed and then the referenced collections should be all good as they were definedShaun Byrnes
03/27/2024, 6:26 AMuserFood and not via alias of user_userFood , i would think that should workShaun Byrnes
03/27/2024, 6:28 AMShaun Byrnes
03/27/2024, 6:28 AMShaun Byrnes
03/27/2024, 6:29 AMKishore Nallan
03/27/2024, 6:29 AMShaun Byrnes
03/27/2024, 6:29 AMShaun Byrnes
03/27/2024, 6:29 AMShaun Byrnes
03/27/2024, 6:48 AMtypes of string[] because the field is array of string ids. When joining with "include_fields": "$pub_exerciseTypes(name, deleted :nest) as exerciseTypes" we get like this:
exerciseTypes: [
{ deleted: false, name: 'Crossfit' },
{ deleted: false, name: 'Recreational' },
{ deleted: false, name: 'Casual' },
{ deleted: false, name: 'Cardio' }
],
types: [
'n1NByFcILax1z1qyZ8H3',
'WMCsYQMLLj2FvjW8do6k',
'06svI9qtNw4oIBwGXsGv',
'1LO22SsEC0UEvDV2Daif'
],
Which is correct as we had 4 ids, each one gets an object which matched and the respective data based on match. However, is there any way to present/map the results more like this instead:
exerciseTypes: [ // or could be 'types' too, doesn't matter
{ uid: 'n1NByFcILax1z1qyZ8H3', deleted: false, name: 'Crossfit' },
{ uid: 'WMCsYQMLLj2FvjW8do6k', deleted: false, name: 'Recreational' },
{ uid: '06svI9qtNw4oIBwGXsGv', deleted: false, name: 'Casual' },
{ uid: '1LO22SsEC0UEvDV2Daif', deleted: false, name: 'Cardio' }
],
Tried changing schema for types to object[] to get it more like the above, but of course it doesn't work as types is not an array of objects, just an array of strings. As you know, something like the above/desired is possible with type object[] and :merge assuming the referenced field started out in an array of objects. But we have no need here to over complicate an array of simple data type (string/int) into an array of objects like types: [{ uid: 'a' }, { uid: 'b' }] as this is overkill and also interferes with filtering on ID in types field anyhow.
I guess the idea is rather than 2 arrays returned - source which is array of simple values like int/string and then another array of objects, containing each field, in same position, merging in the original array of simple values into the result would present the data much more cohesively.
Thoughts?Kishore Nallan
03/27/2024, 7:00 AMtypes field refers to the id field of exericeTypes collection?Shaun Byrnes
03/27/2024, 7:00 AMKishore Nallan
03/27/2024, 7:01 AMid as well in include_fields?Shaun Byrnes
03/27/2024, 7:01 AMKishore Nallan
03/27/2024, 7:01 AM"include_fields": "$pub_exerciseTypes(id, name, deleted :nest)Shaun Byrnes
03/27/2024, 7:01 AMShaun Byrnes
03/27/2024, 7:02 AMtypesShaun Byrnes
03/27/2024, 7:02 AMtypes: [
'n1NByFcILax1z1qyZ8H3',
'WMCsYQMLLj2FvjW8do6k',
'06svI9qtNw4oIBwGXsGv',
'1LO22SsEC0UEvDV2Daif'
],
per docShaun Byrnes
03/27/2024, 7:02 AMShaun Byrnes
03/27/2024, 7:02 AMexerciseTypesShaun Byrnes
03/27/2024, 7:03 AMShaun Byrnes
03/27/2024, 7:03 AMtypes[] and exerciseTypes[]Shaun Byrnes
03/27/2024, 7:03 AMKishore Nallan
03/27/2024, 7:03 AMShaun Byrnes
03/27/2024, 7:03 AMShaun Byrnes
03/27/2024, 7:03 AMShaun Byrnes
03/27/2024, 7:04 AMShaun Byrnes
03/27/2024, 7:04 AMShaun Byrnes
03/27/2024, 7:05 AMShaun Byrnes
03/27/2024, 7:05 AMKishore Nallan
03/27/2024, 7:05 AMShaun Byrnes
03/27/2024, 7:06 AMShaun Byrnes
03/27/2024, 7:06 AMShaun Byrnes
03/27/2024, 7:06 AMShaun Byrnes
03/27/2024, 7:06 AM:merge=keyKishore Nallan
03/27/2024, 7:07 AMn1NByFcILax1z1qyZ8H3 as part of the { deleted: false, name: 'Crossfit' } object?Shaun Byrnes
03/27/2024, 7:07 AMkey in merged arrayShaun Byrnes
03/27/2024, 7:07 AMShaun Byrnes
03/27/2024, 7:08 AM{ deleted: false, name: 'Crossfit' } this is the results back from query after joinShaun Byrnes
03/27/2024, 7:08 AM{
"allowAutoIncrement": false,
"allowSets": false,
"allowWarmupSets": false,
"amrapVisibility": 2,
"bodyParts": [
"kWEtuAPIgWoqWbf3jauS",
"OXCLMw5spVo9Xnacjhzr",
"PFWvobeFyYHBDHiiklgJ",
"m1EWX9asdXtGDPJB9iWN",
"Uqz1pdhGaJXtabsBlaU0"
],
"deleted": false,
"displayWeightCalculator": false,
"distance": {
"precision": 0
},
"distanceVisibility": 0,
"equipment": [],
"height": {
"precision": null
},
"heightVisibility": 2,
"isBodyweight": true,
"isLap": false,
"lapsVisibility": 1,
"name": "Swimming",
"predefined": true,
"repsVisibility": 2,
"setting": 2,
"tempoVisibility": 0,
"trackrpeVisibility": false,
"types": [
"n1NByFcILax1z1qyZ8H3",
"WMCsYQMLLj2FvjW8do6k",
"8TGOKPmPlyGsMMmFs8KD",
"1LO22SsEC0UEvDV2Daif"
],
"uid": "b5kF7VC2pTsVAISDtPmx",
"verified": true,
"weightVisibility": 2
}Kishore Nallan
03/27/2024, 7:08 AMKishore Nallan
03/27/2024, 7:09 AMShaun Byrnes
03/27/2024, 7:09 AM"include_fields": "$pub_bodyParts(name, deleted :nest) as bodyPartsData, $pub_exerciseTypes(name, deleted :nest) as exerciseTypes"Shaun Byrnes
03/27/2024, 7:09 AM{
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "bodyParts",
"optional": false,
"reference": "bodyParts.uid",
"sort": false,
"stem": false,
"type": "string[]"
},Shaun Byrnes
03/27/2024, 7:10 AMShaun Byrnes
03/27/2024, 7:10 AMShaun Byrnes
03/27/2024, 7:10 AMShaun Byrnes
03/27/2024, 7:11 AMKishore Nallan
03/27/2024, 7:11 AM"include_fields": "$pub_bodyParts(name, deleted :nest) as bodyPartsData, $pub_exerciseTypes(uid, name, deleted :nest) as exerciseTypes"Shaun Byrnes
03/27/2024, 7:12 AMShaun Byrnes
03/27/2024, 7:15 AMShaun Byrnes
03/27/2024, 7:15 AMShaun Byrnes
03/27/2024, 7:15 AMShaun Byrnes
03/27/2024, 7:15 AMKishore Nallan
03/27/2024, 7:16 AMShaun Byrnes
03/27/2024, 7:16 AMShaun Byrnes
03/27/2024, 7:16 AMKishore Nallan
03/27/2024, 7:16 AMShaun Byrnes
03/27/2024, 7:17 AMShaun Byrnes
03/27/2024, 7:17 AMShaun Byrnes
03/27/2024, 7:18 AMShaun Byrnes
03/27/2024, 7:18 AMShaun Byrnes
03/27/2024, 7:18 AMKishore Nallan
03/27/2024, 7:20 AMShaun Byrnes
03/27/2024, 7:20 AMShaun Byrnes
03/27/2024, 7:20 AMKishore Nallan
03/27/2024, 7:21 AMShaun Byrnes
03/27/2024, 7:23 AMShaun Byrnes
03/27/2024, 8:20 AMShaun Byrnes
03/27/2024, 8:21 AMpub_food when changing the scoped keys to be created from a parent key which allows collections [ "pub_.+" ]Shaun Byrnes
03/27/2024, 8:21 AMuser_userFoodShaun Byrnes
03/27/2024, 8:22 AMShaun Byrnes
03/27/2024, 8:23 AMShaun Byrnes
03/27/2024, 8:23 AMShaun Byrnes
03/27/2024, 8:23 AMKishore Nallan
03/27/2024, 8:24 AMShaun Byrnes
03/27/2024, 8:24 AMShaun Byrnes
03/27/2024, 8:24 AMShaun Byrnes
03/27/2024, 8:24 AMShaun Byrnes
03/27/2024, 8:24 AMBerend de Boer
03/28/2024, 12:41 AMShaun Byrnes
03/28/2024, 2:13 AMShaun Byrnes
03/28/2024, 2:13 AMShaun Byrnes
03/28/2024, 2:13 AMShaun Byrnes
03/28/2024, 2:14 AMBerend de Boer
03/28/2024, 3:09 AMShaun Byrnes
03/28/2024, 4:40 AMShaun Byrnes
03/28/2024, 4:41 AMKishore Nallan
03/28/2024, 5:07 AM0.26.0.rc68 which contains a syntax change for join. The nest/merge of fields syntax is now through the strategy keyword:
"include_fields": "$authors(*, strategy: merge)"Shaun Byrnes
03/28/2024, 5:15 AMShaun Byrnes
03/28/2024, 5:15 AMKishore Nallan
03/28/2024, 5:17 AM0.26.0.rc67Kishore Nallan
03/28/2024, 5:17 AMShaun Byrnes
03/28/2024, 6:00 AMShaun Byrnes
03/28/2024, 6:00 AMShaun Byrnes
03/28/2024, 6:02 AMstrategy after last comma yea? That way $coll(id, strategy, strategy: nest) would work still correct ?Shaun Byrnes
03/28/2024, 6:02 AMstrategyHarpreet Sangar
03/28/2024, 6:05 AMstrategy: is what the parser looks for. So having strategy porp would not interfere.Shaun Byrnes
03/28/2024, 6:05 AMShaun Byrnes
03/28/2024, 6:05 AMstrategy:nest as in no spaceShaun Byrnes
03/28/2024, 6:06 AMstrategy\:\s*(nest|merge)Shaun Byrnes
03/28/2024, 6:06 AM