This message was deleted.
# community-help
s
This message was deleted.
f
Hey @Marri Shiva, I should clarify a couple of things here. The
--enable-search-analytics
flag and automatic aggregation are actually two separate features. The flag simply enables search analytics functionality overall, rather than handling the aggregation aspect specifically. Regarding automatic aggregation, could you help me understand what exactly you're looking to disable? Just so you know, the aggregation only kicks in after you've set up either a no hits rule or a popular queries rule. If you'd like to change the default behavior, you can use the
enable_auto_aggregation
parameter when creating your Analytics rule and set it to
false
. This way, events won't automatically get sent to the server. Docs
m
Here is my requirement we have two types of searches: global search and shop-level search. We want to track and analyze search behavior with the following metadata: •
userId
•
shopId
•
zipcode
•
query (q)
•
count
• date (for date-wise analytics) Since the default
popular_queries
rule only stores
q
and
count
, we plan to log and aggregate analytics ourselves, so we can support both custom metadata and date-wise reports. To achieve this, we’d like to confirm the following: 1. If we set
"enable_auto_aggregation": false
while creating a
popular_queries
rule, will that completely disable automatic writes to the
destination_collection
? 2. Is there a recommended approach to implement custom search analytics with date-wise grouping, or can we rely on manually writing to a collection for this purpose?
f
Since you're going to use either an external tool for storing the analytic data, or writing to another Typesense collection for storing the analytics data, I'm not sure what's the purpose of creating a
popular_queries
rule is
m
As per the docs I’ve created an analytics rule with the following configuration:
Copy code
{
  "name": "qa_product_queries_aggregation",
  "type": "popular_queries",
  "params": {
    "source": {
      "collections": [
        "qa-products"
      ]
    },
    "destination": {
      "collection": "qa-product-queries"
    },
    "expand_query": false,
    "enable_auto_aggregation": false,
    "limit": 1000,
    "events": [
      {
        "type": "search",
        "name": "qa-products_search_event"
      }
    ]
  }
}
However, when I try to retrieve the rule using:
GET /analytics/rules
it returns an empty object or list, even though the creation call was successful. Also, when I try to send an event like this: { "type": "search", "name": "qa-products_search_event", "data": { "q": "mango", "user_id": "123", "shop_id": "shop_456", "zipcode": "560001", "timestamp": 1717580000000 } } I get the error: {"message": "No analytics rule defined for event name qa-products_search_event"} Why the rule doesn’t show up when I retrieve it? Why the event is not being matched, even though the name matches exactly?
f
What Typesense Version are you using?
m
v29.0.rc26
f
Could you give me your cluster's ID either here or on DM?
m
vjzoc4yu2fq3k7rmp
f
Is it okay with you if I remove the old Analytics rule and create a new one that will be correct?
You have one like this:
Copy code
{
  "rules": [
    {
      "name": "qa_product_queries_aggregation",
      "params": {
        "destination": {
          "collection": "qa-product-queries"
        },
        "expand_query": false,
        "limit": 1000,
        "source": {
          "collections": [
            "qa-products"
          ]
        }
      },
      "type": "popular_queries"
    }
  ]
}
m
okay
f
Copy code
curl "https://vjzoc4yu2fq3k7rmp-1.a1.typesense.net/analytics/events" -X POST \
     -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
     -d '{
            "type": "search",
            "name": "qa-products_search_event",
            "data": {
                  "q": "dummy search",
                  "user_id": "1234"
            }
        }
Could you try this now? It should work.
m
may i know the rule configuration?
f
Copy code
{
  "name": "product_queries_aggregation",
  "type": "popular_queries",
  "params": {
    "source": {
      "collections": [
        "qa-products"
      ],
      "enable_auto_aggregation": false,
      "events": [
        {
          "type": "search",
          "name": "qa-products_search_event"
        }
      ]
    },
    "destination": {
      "collection": "qa-product_queries"
    },
    "limit": 1000
  }
}
You can delete this and create a new one if you prefer. key thing is to set the
enable_auto_aggregation
to false and define a search event as well
m
what is the issue in my configuration ?
f
It seems you had created an analytics rule of the same name prior. Deleting it and re-creating fixed it
m
"destination": { "collection": "qa-product_queries" } this collection is not exited in my cluster
it should be qa-product-queries
f
Doesn't
qa-product-queries
exist and is the same as the one you posted?
m
if i replace destination collection with qa-product-queries, i am getting same {"message": "No analytics rule defined for event name qa-products_search_event"}
in my post i mentioned qa-product-queries
but in your configuration you have used qa-product_queries for desination collection
f
Sorry but I don't follow. This sequence of requests works:
Copy code
❮ curl -k "https://vjzoc4yu2fq3k7rmp-1.a1.typesense.net/analytics/rules" \
      -X POST \
      -H "Content-Type: application/json" \
      -H "X-TYPESENSE-API-KEY: redacted" \
      -d '{
  "name": "product_queries_aggregation",
  "type": "popular_queries",
  "params": {
    "source": {
      "collections": [
        "qa-products-1745829004935"
      ],
      "enable_auto_aggregation": false,
      "events": [
        {
          "type": "search",
          "name": "qa-products_search_event"
        }
      ]
    },
    "destination": {
      "collection": "qa-product-queries" <- Hyphen instead of underscore
    },
    "limit": 1000
  }
}
'

{"name":"product_queries_aggregation","params":{"destination":{"collection":"qa-product-queries"},"limit":1000,"source":{"collections":["qa-products-1745829004935"],"enable_auto_aggregation":false,"events":[{"name":"qa-products_search_event","type":"search"}]}},"type":"popular_queries"}%

❯ curl "https://vjzoc4yu2fq3k7rmp-1.a1.typesense.net/analytics/events" -X POST \
     -H "X-TYPESENSE-API-KEY: redacted" \
     -d '{
            "type": "search",
            "name": "qa-products_search_event",
            "data": {
                  "q": "dummy search",
                  "user_id": "1234"
            }
        }'

{"ok": true}%
m
I have a question. The destination collection has the following fields:
q
,
count
, and
user_id
. In the POST call, you are passing both
q
and
user_id
. The
q
value is being saved in the destination collection, and the
count
is being populated automatically. However, the
user_id
is not being saved in the destination collection. Why is that?
f
user_id
isn't evaluated for popularity rules, and won't be saved, regardless if you have it defined in the schema or not
m
I understand from your response that
user_id
is not used or stored when using
popular_queries
rules. However, the API returns an error if I omit
user_id
from the analytics event payload: `{"message": "search event json data fields should contain
user_id
as string value."}` If
user_id
is not stored or evaluated, could you please clarify what its purpose is in the context of analytics events? Why is it required in the payload?
f
Search events shouldn't require a
user_id
, only converion and clicks should require it. Is this throwing for a
search
event type?
m
yes
f
So after debugging I saw that even though we don't save it on the collection, the user id is still needed for validation and future features may depend on it
m
Okay