When using Union search, the following payload is ...
# community-help
m
When using Union search, the following payload is returned
Copy code
{
  "found": 78536,
  "hits": [
    ... (10 items)
  ],
  "out_of": 78536,
  "page": 74,
  "search_cutoff": false,
  "search_time_ms": 9,
  "union_request_params": [
    {
      "collection": "post",
      "found": 77751,
      "per_page": 10,
      "q": "*"
    },
    {
      "collection": "podcast",
      "first_q": "*",
      "found": 245,
      "per_page": 10,
      "q": "*"
    },
    {
      "collection": "video",
      "first_q": "*",
      "found": 540,
      "per_page": 10,
      "q": "*"
    }
  ]
Adding a page parameter works fine up until page 75, however for page 76 the hits array is empty
Copy code
{
  "found": 78536,
  "hits": [],
  "out_of": 78536,
  "page": 76,
  "search_cutoff": false,
  "search_time_ms": 9,
  "union_request_params": [
    {
      "collection": "post",
      "found": 77751,
      "per_page": 10,
      "q": "*"
    },
    {
      "collection": "podcast",
      "found": 245,
      "per_page": 10,
      "q": "*"
    },
    {
      "collection": "video",
      "found": 540,
      "per_page": 10,
      "q": "*"
    }
  ]
}
However, when querying the post collection directly (
/collections/post/documents/search
) it works as expected.
Copy code
{
  "facet_counts": [],
  "found": 77751,
  "hits": [
    ... (10 items)
  ],
  "out_of": 77751,
  "page": 200,
  "request_params": {
    "collection_name": "post",
    "first_q": "*",
    "per_page": 10,
    "q": "*"
  },
  "search_cutoff": false,
  "search_time_ms": 25
}
j
CC: @Harpreet Sangar
m
I've shared a similar scenario privately with @Harpreet Sangar. It seems my issue is related to the
group_by
optimization introduced in v29.RC4. Everything was working fine on RC3 and earlier versions. I had tested the RCs extensively, but unfortunately, I only noticed this issue after upgrading to the v29 GA release in production. 😞 I had to downgrade back to v28.
m
Thanks for the reply. Unfortunately, I'm still running v28 on my cluster so I can probably rule that out.
h
@Michael Keegan Is it possible for you to share the reproduction steps in this format? https://gist.github.com/jasonbosco/7c3432713216c378472f13e72246f46b
m
Sure. Leave it with me.
j
Good morning @Harpreet Sangar, I'm a colleague of @Michael Keegan. I've created a reproduction in the format you requested. Are you happy for me to PM you the .sh file? Thanks
h
Yes.
j
Thank you. The file contains the following curl requests: # Union search, all collections ## Page 1, 10 per page, 78602 found, 10 returned ## Page 59, 10 per page, 78602 found, 10 returned ## Page 60, 10 per page, 78602 found, 9 returned ## Page 61, 10 per page, 78602 found, 0 returned # Union search, just posts collection ## Page 25, 10 per page, 77816 found, 10 returned ## Page 26, 10 per page, 77816 found, 0 returned # Posts collection directly ## Page 7000, 10 per page, 77816 found, 10 returned This should be enough to demonstate the issue but, if you need anything more, let me know.
h
https://github.com/typesense/typesense/pull/2428/files fixes the pagination issue with union.
👍 1