#community-help

Issue with `included_fields` Command in Typesense

TLDR SamHendley encountered an issue with the included_fields command in Typesense versions 0.23.0 and 0.24.0.rc17. Jason helped identify it as a bug in the 0.24.X version, which was later addressed in release 0.24.0.rcn19.

14mo

Join the chat

Oct 14, 2022 (14 months ago)

Photo of md5-a9a351e11d64f05b41fec183816a0cda

SamHendley

06:42 PM

Am I doing something dumb or should I be able to ask for an ‘unindexed’ document field in included_fields during a search? Right now I get an error saying my field is not in the schema (though I can see it is in document if I query by id). I went down a rabbit hole and got it mostly working using the new nested functionality in bleeding edge but I don’t ever need it to search it and would like it be left alone as much as possible.

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

06:43 PM

You should be able to get unindexed fields using include_fields (you had a typo btw)

06:44

Jason

06:44 PM

May I know what version of Typesense you’re using?

SamHendley

06:44 PM

0.23.0 and now 0.24.0.rc17

06:45

SamHendley

06:45 PM

Both of them give me an error of format: Could not find a field named "source" in the schema.

Jason

06:45 PM

Could you modify this code snippet to reproduce the error with your particular schema and a few sample records: https://gist.github.com/jasonbosco/7c3432713216c378472f13e72246f46b

SamHendley

06:46 PM

will do

06:50

SamHendley

06:50 PM

curl "" \
       -X POST \
       -H "Content-Type: application/json" \
       -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
       -d '{
         "name": "companies",
         "fields": [
           {"name": "company_name", "type": "string" },
           {"name": "num_employees", "type": "int32" },
           {"name": "country", "type": "string", "facet": true }
         ],
         "default_sorting_field": "num_employees"
       }'
       
curl "" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -H "Content-Type: text/plain" \
        -X POST \
        -d '{"id": "124","company_name": "Stark Industries","num_employees": 5215,"country": "USA", "source" : {"data" :"value"}}
            {"id": "125","company_name": "Acme Corp","num_employees": 2133,"country": "CA", "source" : {"data": "value2"}}'
            
curl "" \
        -X POST \
        -H "Content-Type: application/json" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -d '{
          "searches": [
            {
              "collection": "companies",
              "q": "stark"
            }
          ]
        }'

curl "" \
        -X POST \
        -H "Content-Type: application/json" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -d '{
          "searches": [
            {
              "collection": "companies",
              "q": "stark"
            }
          ]
        }'

06:53

SamHendley

06:53 PM

Removing included_fields is actually a workaround for me but it does mean I pull a ton of unneeded data out of the service. In my case that’s not critical because I do have a service fronting typesense so I’m not exposing internal data to the end client but it is wasteful

Jason

06:56 PM

Hmm, that code snippet works for me:

➜  ~ curl "" \
        -X POST \
        -H "Content-Type: application/json" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -d '{
          "searches": [
            {
              "collection": "companies",
              "q": "stark"
            }
          ]
        }' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   546    0   403  100   143  42524  15089 --:--:-- --:--:-- --:--:-- 78000
{
  "results": [
    {
      "facet_counts": [],
      "found": 1,
      "hits": [
        {
          "document": {
            "company_name": "Stark Industries",
            "source": {
              "data": "value"
            }
          },
          "highlights": [
            {
              "field": "company_name",
              "matched_tokens": [
                "Stark"
              ],
              "snippet": "<mark>Stark</mark> Industries"
            }
          ],
          "text_match": 72341265420648450
        }
      ],
      "out_of": 2,
      "page": 1,
      "request_params": {
        "collection_name": "companies",
        "per_page": 10,
        "q": "stark"
      },
      "search_cutoff": false,
      "search_time_ms": 0
    }
  ]
}

06:56

Jason

06:56 PM

I’m running this on 0.23.1 though

SamHendley

07:00 PM

Yep that appears to work. Maybe new bug in 0.24.X ?

Jason

07:02 PM

Yup, I see the error in 0.24.0.rcn17. Will look into it

SamHendley

07:03 PM

awesome, thanks for your help.

Jason

07:03 PM

Thank you for catching that!

Oct 17, 2022 (14 months ago)

Jason

08:19 PM

FYI: We’ve addressed this issue in 0.24.0.rcn19

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Typesense Bug Fix with `canceled_at` Field and Upgrade Concerns

Mateo reported an issue regarding the treatment of an optional field by Typesense which was confirmed a bug by Jason. After trying an upgrade, an error arose. Jason explained the bug was due to a recent change and proceeded to downgrade their version. Future upgrade protocols were discussed.

10mo

Trouble in Implementing Deeply Nested Search

Anirudh is struggling to implement a two-level nested search. Jason asked for some specific examples to study the issue. Anirudh provided some material, realizing that adding top fields helped but might over-index. Jason then suggested reporting this issue on GitHub.

Inconsistent Search Results with Typesense

David reported inconsistencies with infix searching using Typesense, despite no change in configuration. Upon review, Jason could not consistently reproduce the issue and suggested potential fixes including a debug build on the user's cluster. The issue remains unresolved.

1mo

Issue with Embedding Error in Version 0.25.0.rc63

Bill reported a bug in version 0.25.0.rc63 regarding a problem with updating or emplacing a document and receiving an embedding error. This was resolved in version 0.25.0.rc65, but further discussion ensued regarding the function of 'index' in the update feature.

4mo

Cold Start Problem with Dynamic Collections

Adrian reported cold start issues with dynamic collections. Jason suggested using wildcard `*` for query_by parameters, upgrading to `0.25.0.rc34`, and clarified conventions. Adrian's issues were resolved but they reported a limitation that will potentially be addressed.

6mo