Issues with Aggregation and Analytic Rules
TLDR Pawel experienced issues with aggregation in a complex data collection. Kishore Nallan identified it as a bug and provided an updated software build to solve the problem.
3
Aug 23, 2023 (1 month ago)
Pawel
11:29 AMI have an issue with aggregation (analytic rules)
I would like to have separate aggregation for each base collection.
I created aggregation collections as mentioned in the documentation
'name' => $name,
'fields' => [
['name' => 'q', 'type' => 'string'],
['name' => 'count', 'type' => 'int32'],
],
I added rules for each collection:
{"rules":[{"name":"magento_2_default_reviews_meta_aggregation_rule","params":{"destination":{"collection":"magento_2_default_reviews_meta_aggregation"},"limit":1000,"source":{"collections":["magento_2_default_reviews_meta_1692788921"]}},"type":"popular_queries"},{"name":"magento_2_czech_market_reviews_aggregation_rule","params":{"destination":{"collection":"magento_2_czech_market_reviews_aggregation"},"limit":1000,"source":{"collections":["magento_2_czech_market_reviews_1692788919"]}},"type":"popular_queries"},{"name":"magento_2_default_urls_aggregation_rule","params":{"destination":{"collection":"magento_2_default_urls_aggregation"},"limit":1000,"source":{"collections":["magento_2_default_urls_1692788923"]}},"type":"popular_queries"},{"name":"magento_2_czech_market_reviews_meta_aggregation_rule","params":{"destination":{"collection":"magento_2_czech_market_reviews_meta_aggregation"},"limit":1000,"source":{"collections":["magento_2_czech_market_reviews_meta_1692788922"]}},"type":"popular_queries"},{"name":"magento_2_default_reviews_aggregation_rule","params":{"destination":{"collection":"magento_2_default_reviews_aggregation"},"limit":1000,"source":{"collections":["magento_2_default_reviews_1692788917"]}},"type":"popular_queries"},{"name":"magento_2_czech_market_eav_attributes_aggregation_rule","params":{"destination":{"collection":"magento_2_czech_market_eav_attributes_aggregation"},"limit":1000,"source":{"collections":["magento_2_czech_market_eav_attributes_1692788915"]}},"type":"popular_queries"},{"name":"magento_2_default_categories_aggregation_rule","params":{"destination":{"collection":"magento_2_default_categories_aggregation"},"limit":1000,"source":{"collections":["magento_2_default_categories_1692788517"]}},"type":"popular_queries"},{"name":"magento_2_default_products_children_aggregation_rule","params":{"destination":{"collection":"magento_2_default_products_children_aggregation"},"limit":1000,"source":{"collections":["magento_2_default_products_children_1692788636"]}},"type":"popular_queries"},{"name":"magento_2_czech_market_products_aggregation_rule","params":{"destination":{"collection":"magento_2_czech_market_products_aggregation"},"limit":1000,"source":{"collections":["magento_2_czech_market_products_1692788586"]}},"type":"popular_queries"},{"name":"magento_2_default_products_aggregation_rule","params":{"destination":{"collection":"magento_2_default_products_aggregation"},"limit":1000,"source":{"collections":["magento_2_default_products_1692788523"]}},"type":"popular_queries"},{"name":"magento_2_czech_market_urls_aggregation_rule","params":{"destination":{"collection":"magento_2_czech_market_urls_aggregation"},"limit":1000,"source":{"collections":["magento_2_czech_market_urls_1692788954"]}},"type":"popular_queries"},{"name":"magento_2_default_cms_pages_aggregation_rule","params":{"destination":{"collection":"magento_2_default_cms_pages_aggregation"},"limit":1000,"source":{"collections":["magento_2_default_cms_pages_1692788906"]}},"type":"popular_queries"},{"name":"magento_2_czech_market_products_children_aggregation_rule","params":{"destination":{"collection":"magento_2_czech_market_products_children_aggregation"},"limit":1000,"source":{"collections":["magento_2_czech_market_products_children_1692788809"]}},"type":"popular_queries"},{"name":"magento_2_default_cms_blocks_aggregation_rule","params":{"destination":{"collection":"magento_2_default_cms_blocks_aggregation"},"limit":1000,"source":{"collections":["magento_2_default_cms_blocks_1692788902"]}},"type":"popular_queries"},{"name":"magento_2_czech_market_categories_aggregation_rule","params":{"destination":{"collection":"magento_2_czech_market_categories_aggregation"},"limit":1000,"source":{"collections":["magento_2_czech_market_categories_1692788521"]}},"type":"popular_queries"},{"name":"magento_2_czech_market_cms_blocks_aggregation_rule","params":{"destination":{"collection":"magento_2_czech_market_cms_blocks_aggregation"},"limit":1000,"source":{"collections":["magento_2_czech_market_cms_blocks_1692788904"]}},"type":"popular_queries"},{"name":"magento_2_czech_market_cms_pages_aggregation_rule","params":{"destination":{"collection":"magento_2_czech_market_cms_pages_aggregation"},"limit":1000,"source":{"collections":["magento_2_czech_market_cms_pages_1692788908"]}},"type":"popular_queries"},{"name":"magento_2_default_gdpr_cookies_aggregation_rule","params":{"destination":{"collection":"magento_2_default_gdpr_cookies_aggregation"},"limit":1000,"source":{"collections":["magento_2_default_gdpr_cookies_1692788909"]}},"type":"popular_queries"},{"name":"magento_2_czech_market_gdpr_cookies_aggregation_rule","params":{"destination":{"collection":"magento_2_czech_market_gdpr_cookies_aggregation"},"limit":1000,"source":{"collections":["magento_2_czech_market_gdpr_cookies_1692788911"]}},"type":"popular_queries"},{"name":"magento_2_default_eav_attributes_aggregation_rule","params":{"destination":{"collection":"magento_2_default_eav_attributes_aggregation"},"limit":1000,"source":{"collections":["magento_2_default_eav_attributes_1692788912"]}},"type":"popular_queries"}]}
After this, every single time, only the last added rule is working (firs in the rules)
I tested it on two, three and 20 collections, In this case only magento_2_default_reviews_meta_aggregation is collection queries
Also In server logs I see only this one:
I20230823 11:24:09.380872 1402 analytics_manager.cpp:297] Query aggregation for collection: magento_2_default_reviews_meta_aggregation
I20230823 11:24:09.588793 1402 analytics_manager.cpp:311] Top K aggregation for collection: magento_2_default_reviews_meta_aggregation
Kishore Nallan
11:31 AM1
Pawel
11:37 AMKishore Nallan
11:38 AMPawel
11:38 AMPawel
11:39 AMKishore Nallan
11:39 AMKishore Nallan
11:40 AMtypesense/typesense:0.25.1.rc2
Pawel
11:41 AMKishore Nallan
11:42 AM0.25.1.rc2
Pawel
11:42 AMPawel
11:43 AMPawel
11:43 AMPawel
11:44 AMKishore Nallan
11:46 AM
Kishore Nallan
11:46 AMPawel
11:50 AM404 Sorry - we could not find this page. Ironic isn't it?
Pawel
12:07 PMKishore Nallan
12:08 PMPawel
12:13 PMPawel
12:14 PMPawel
12:15 PMPawel
12:17 PMKishore Nallan
12:17 PMKishore Nallan
12:17 PMPawel
12:18 PMPawel
12:18 PMKishore Nallan
12:18 PMPawel
12:18 PM1
Pawel
12:20 PM1
Kishore Nallan
12:26 PMmagento_2_default_reviews_meta_aggregation
?Pawel
12:27 PMKishore Nallan
01:41 PMKishore Nallan
02:16 PM0.25.1.rc3
. Please try it out and confirm. Thank you.Aug 24, 2023 (1 month ago)
Pawel
06:14 AMIt is working now as expected.
Thank you Kishore Nallan so much!
Kishore Nallan
06:23 AMTypesense
Indexed 2779 threads (79% resolved)
Similar Threads
Resolving Typesense Analytics "top_queries" Collection Issue
Zain faced issues with typesense analytics and an empty "top_queries" collection. Jason and Kishore Nallan assisted in troubleshooting, identified a bug, and suggested a fix in `0.25.0.rc66`. The fix worked for Zain.
Troubleshooting Typesense API Analytics Query Suggestions
Md was confused about implementing Typesense's Analytics Query Suggestions and experienced issues with collections returning no hits. Assistance from Kishore Nallan eventually led to the identification that analytics had to be enabled. They also discussed tracking duplicate and empty queries, resulting in Md creating a Github issue.
Typesense Server Crash and Counting Issue
Dan reported a server crash and counting issues with Typesense server version 0.26.0.rc1. Jason suggested an upgrade to `0.26.0.rc13` while Kishore Nallan committed to investigate the issue based on a sample dataset from Dan.
Troubleshooting 400 Error When Upgrading Typesense Firestore Extension
Orion experienced a `400` error after updating the Typesense Firestore extension, causing issues with cloud functions. They traced the issue back to a data type conflict in their Typesense collection schema after updating. With help from Jason and Kishore Nallan, they resolved the issue by recreating the collection.
Troubleshooting Typesense Connection Issue and Data Retrieval
Felix encountered issues connecting to a server-created typesense and retrieving data collections. Kishore Nallan and Jason provided guidance, highlighting errors in the original code and suggesting changes to the URL and curl command.