Hey All I am bit struggling with typesense analytics I tried typesense #community-help

Hey All , I am bit struggling with typesense analy...

Zain Haider

08/10/2023, 10:03 AM

Hey All , I am bit struggling with typesense analytics. I tried to get the most searched product but somehow my “top_queries” collection is empty. I don’t get any error but still my “top_queries” remains empty.Below is the docker file I used to run my typesense server. Used this for reference : https://gist.github.com/jasonbosco/0c49916fc2e7a54f30d961aac111cf4a Used this compose file to run my server :

version: '3.4'

services:

typesense:

image: typesense/typesense:0.25.0.rc62

restart: on-failure

ports:

- "8108:8108"

volumes:

- ./typesense-data:/data

command: '--data-dir /data --api-key=xyz --enable-cors --enable-search-analytics=true --analytics-flush-interval=60'

TIA

Kishore Nallan

08/10/2023, 11:08 AM

Can you post the schema of the top_queries collection?

Zain Haider

08/10/2023, 1:06 PM

yes sure

Zain Haider

08/10/2023, 1:07 PM

Copy code

curl -k "<http://localhost:8108/collections>" -X POST -H "Content-Type: application/json" \
      -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{
        "name": "top_queries",
        "fields": [
          {"name": "q", "type": "string" },
          {"name": "count", "type": "int32" }
        ]
      }'

used this req to create the schema

Zain Haider

08/10/2023, 1:08 PM

I have actually simply referred to this article. and just changed my source collection https://gist.github.com/jasonbosco/0c49916fc2e7a54f30d961aac111cf4a

Zain Haider

08/10/2023, 1:59 PM

this.typeSenseClient.collections( collectionName ).retrieve();

this is how i am retrieving the collection . I am using typesense client for node/nest js

Zain Haider

08/10/2023, 2:22 PM

@Kishore Nallan could you please help , what i am doing wrong?

Kishore Nallan

08/10/2023, 2:29 PM

Sorry I'm tied up with a couple of other ongoing issues. Someone will investigate and get back to you soon.

Jason Bosco

08/10/2023, 2:30 PM

Could you also do a

GET /analytics/rules

and post the output?

Zain Haider

08/10/2023, 2:33 PM

@Jason Bosco {“rules”[{“name”“top_queries”,“params”{“destination”{“collection”“top queries”},“limit”1000,“source”{“collections”[“products”]}},“type”:“popular_queries”}]}

Zain Haider

08/10/2023, 2:34 PM

@Kishore Nallan sure . sorry for disturbing man . Kudos for building a great product and community 🙌

Kishore Nallan

08/10/2023, 2:37 PM

No worries!

🙌 1

Jason Bosco

08/10/2023, 2:37 PM

Could you post the schema of your source collection as well and the search query you’re sending it? (Could you post it in a code block in Slack - three backticks, otherwise Slack replaces quotes and makes it hard to parse through)

Zain Haider

08/10/2023, 2:42 PM

yes @Jason Bosco

Zain Haider

08/10/2023, 2:45 PM

Product Schema:

const productSchema: CollectionCreateSchema = {

name: this.collectionName,

fields: [

{ name: "id", type: "string" },

{ name: "name", type: "string" },

{ name: "description", type: "string" },

{ name: "category", type: "string[]" },

{ name: "brand", type: "string" },

{ name: "company", type: "string" },

{ name: "status", type: "bool" },

{ name: "isHidden", type: "bool" }

};

await this.searchService.createCollection( productSchema );

Search Query:-

const searchParams: SearchParams = {

enable_overrides: true,

filter_by: "status:=true && isHidden:=false",

q: query,

query_by: "name,description,category,brand,company"

};

const product = await this.searchService.searchFromCollection( this.collectionName, searchParams );

Jason Bosco

08/10/2023, 3:15 PM

It’s hard to the debug with the variables in the code… Could you come up with a set of curl commands like this that replicates the issue end-to-end, so I can try and replicate it on my end?

Zain Haider

08/10/2023, 3:17 PM

sure

Zain Haider

08/10/2023, 4:45 PM

Steps to repro

Jason Bosco

08/10/2023, 4:49 PM

Step 6 Now search on Product collection (Repeat Step 3)

Repeated the step 3 time to check the count of query “garam” You’re waiting at least 60s after this step right? Because that’s what this parameter does:

analytics-flush-interval=60

Jason Bosco

08/10/2023, 4:50 PM

Also on step 7, that only gets the collection schema of the top_queries collection. You want to do an export of the data in the collection to get all the search terms

Zain Haider

08/10/2023, 6:36 PM

yes i have waited 60 seconds but still its giving me 0 docs

Zain Haider

08/10/2023, 6:36 PM

curl -H "X-TYPESENSE-API-KEY: xyz" \

"<http://localhost:8108/collections/top_queries/documents/search>\

?q=*&query_by=q"

this req gives this result

{"facet_counts":[],"found":0,"hits":[],"out_of":0,"page":1,"request_params":{"collection_name":"top_queries","per_page":10,"q":"*"},"search_cutoff":false,"search_time_ms":3}

Zain Haider

08/10/2023, 6:36 PM

this request is also returning 0 docs

Jason Bosco

08/10/2023, 7:15 PM

I was able to replicate the issue. Will take a look and keep you posted

🙌 1

Zain Haider

08/10/2023, 7:34 PM

Thankyou so much @Jason Bosco

Kishore Nallan

08/11/2023, 5:19 AM

@Zain Haider I've a fix for this in

0.25.0.rc66

-- can you please try that out?

Zain Haider

08/11/2023, 6:22 AM

@Kishore Nallan sure , I will update here after trying that

Zain Haider

08/11/2023, 8:41 AM

@Kishore Nallan its working for the repro doc . Thanks for the instant support 🙌

🙌 1

Zain Haider

08/11/2023, 8:41 AM

let me try it with js client too and update you here

Neil

09/26/2023, 3:16 PM

Hey guys! I'm having a similar issue here where analytics collection isn't populating - I'm using Typesense Cloud and have the scheme pretty basic as depicted in the docs:

Copy code

"fields": [
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "q",
      "optional": false,
      "sort": false,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "count",
      "optional": false,
      "sort": true,
      "type": "int32"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "timestamp",
      "optional": true,
      "sort": true,
      "type": "int64"
    }
  ]

One of my main collections (I have 2) has a scheme like this - not sure if it matters:

Copy code

"fields": [
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": ".*",
      "optional": true,
      "sort": false,
      "type": "auto"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "category",
      "optional": true,
      "sort": false,
      "type": "object"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "description",
      "optional": true,
      "sort": false,
      "type": "string"
    }

Appreciate the help!

Jason Bosco

09/26/2023, 3:18 PM

We found another issue on Typesense Cloud with analytics. Could you upgrade to

0.25.2.rc1

and try again?

Neil

09/26/2023, 3:19 PM

Hey Jason - thanks for replying. I'm using TS Cloud, I didn't realize we could upgrade instances on that. Can you guide me to how please

Jason Bosco

09/26/2023, 3:20 PM

You want to open your Cluster dashboard -> Cluster configuration (on the side pane) -> Modify (button on top) and then you'll be able to change the Typesense Version (and other parameters if needed) and schedule a change for now or later

Neil

09/26/2023, 3:27 PM

Got it thanks - will give that a try shortly

👍 1

Neil

09/26/2023, 7:41 PM

That seems to have done the trick - thanks!

👍 1

Open in Slack

Previous Next