#community-help

Issue with `included_fields` Command in Typesense

TLDR SamHendley encountered an issue with the included_fields command in Typesense versions 0.23.0 and 0.24.0.rc17. Jason helped identify it as a bug in the 0.24.X version, which was later addressed in release 0.24.0.rcn19.

Powered by Struct AI
Oct 14, 2022 (14 months ago)
SamHendley
Photo of md5-a9a351e11d64f05b41fec183816a0cda
SamHendley
06:42 PM
Am I doing something dumb or should I be able to ask for an ‘unindexed’ document field in included_fields during a search? Right now I get an error saying my field is not in the schema (though I can see it is in document if I query by id). I went down a rabbit hole and got it mostly working using the new nested functionality in bleeding edge but I don’t ever need it to search it and would like it be left alone as much as possible.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
06:43 PM
You should be able to get unindexed fields using include_fields (you had a typo btw)
06:44
Jason
06:44 PM
May I know what version of Typesense you’re using?
SamHendley
Photo of md5-a9a351e11d64f05b41fec183816a0cda
SamHendley
06:44 PM
0.23.0 and now 0.24.0.rc17
06:45
SamHendley
06:45 PM
Both of them give me an error of format: Could not find a field named "source" in the schema.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
06:45 PM
Could you modify this code snippet to reproduce the error with your particular schema and a few sample records: https://gist.github.com/jasonbosco/7c3432713216c378472f13e72246f46b
SamHendley
Photo of md5-a9a351e11d64f05b41fec183816a0cda
SamHendley
06:46 PM
will do
06:50
SamHendley
06:50 PM
curl "" \
       -X POST \
       -H "Content-Type: application/json" \
       -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
       -d '{
         "name": "companies",
         "fields": [
           {"name": "company_name", "type": "string" },
           {"name": "num_employees", "type": "int32" },
           {"name": "country", "type": "string", "facet": true }
         ],
         "default_sorting_field": "num_employees"
       }'
       
curl "" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -H "Content-Type: text/plain" \
        -X POST \
        -d '{"id": "124","company_name": "Stark Industries","num_employees": 5215,"country": "USA", "source" : {"data" :"value"}}
            {"id": "125","company_name": "Acme Corp","num_employees": 2133,"country": "CA", "source" : {"data": "value2"}}'
            
curl "" \
        -X POST \
        -H "Content-Type: application/json" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -d '{
          "searches": [
            {
              "collection": "companies",
              "q": "stark"
            }
          ]
        }'

curl "" \
        -X POST \
        -H "Content-Type: application/json" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -d '{
          "searches": [
            {
              "collection": "companies",
              "q": "stark"
            }
          ]
        }'
06:53
SamHendley
06:53 PM
Removing included_fields is actually a workaround for me but it does mean I pull a ton of unneeded data out of the service. In my case that’s not critical because I do have a service fronting typesense so I’m not exposing internal data to the end client but it is wasteful
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
06:56 PM
Hmm, that code snippet works for me:

➜  ~ curl "" \
        -X POST \
        -H "Content-Type: application/json" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -d '{
          "searches": [
            {
              "collection": "companies",
              "q": "stark"
            }
          ]
        }' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   546    0   403  100   143  42524  15089 --:--:-- --:--:-- --:--:-- 78000
{
  "results": [
    {
      "facet_counts": [],
      "found": 1,
      "hits": [
        {
          "document": {
            "company_name": "Stark Industries",
            "source": {
              "data": "value"
            }
          },
          "highlights": [
            {
              "field": "company_name",
              "matched_tokens": [
                "Stark"
              ],
              "snippet": "<mark>Stark</mark> Industries"
            }
          ],
          "text_match": 72341265420648450
        }
      ],
      "out_of": 2,
      "page": 1,
      "request_params": {
        "collection_name": "companies",
        "per_page": 10,
        "q": "stark"
      },
      "search_cutoff": false,
      "search_time_ms": 0
    }
  ]
}
06:56
Jason
06:56 PM
I’m running this on 0.23.1 though
SamHendley
Photo of md5-a9a351e11d64f05b41fec183816a0cda
SamHendley
07:00 PM
Yep that appears to work. Maybe new bug in 0.24.X ?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
07:02 PM
Yup, I see the error in 0.24.0.rcn17. Will look into it
SamHendley
Photo of md5-a9a351e11d64f05b41fec183816a0cda
SamHendley
07:03 PM
awesome, thanks for your help.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
07:03 PM
Thank you for catching that!
Oct 17, 2022 (14 months ago)
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
08:19 PM
FYI: We’ve addressed this issue in 0.24.0.rcn19

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Typesense Bug Fix with `canceled_at` Field and Upgrade Concerns

Mateo reported an issue regarding the treatment of an optional field by Typesense which was confirmed a bug by Jason. After trying an upgrade, an error arose. Jason explained the bug was due to a recent change and proceeded to downgrade their version. Future upgrade protocols were discussed.

3

74
10mo

Trouble in Implementing Deeply Nested Search

Anirudh is struggling to implement a two-level nested search. Jason asked for some specific examples to study the issue. Anirudh provided some material, realizing that adding top fields helped but might over-index. Jason then suggested reporting this issue on GitHub.

2

21
3d

Inconsistent Search Results with Typesense

David reported inconsistencies with infix searching using Typesense, despite no change in configuration. Upon review, Jason could not consistently reproduce the issue and suggested potential fixes including a debug build on the user's cluster. The issue remains unresolved.

6

59
1mo

Issue with Embedding Error in Version 0.25.0.rc63

Bill reported a bug in version 0.25.0.rc63 regarding a problem with updating or emplacing a document and receiving an embedding error. This was resolved in version 0.25.0.rc65, but further discussion ensued regarding the function of 'index' in the update feature.

5

63
4mo

Cold Start Problem with Dynamic Collections

Adrian reported cold start issues with dynamic collections. Jason suggested using wildcard `*` for query_by parameters, upgrading to `0.25.0.rc34`, and clarified conventions. Adrian's issues were resolved but they reported a limitation that will potentially be addressed.

6

39
6mo