#community-help

Troubleshooting Typesense Setup and Understanding Facets and Keywords

TLDR Demitri encountered errors when exploring Typesense for the first time. Jason guided them through troubleshooting and discussed facets, keyword settings, and widget configurations. Helin shared a Python demo app and its source code to help Demitri with their project.

Powered by Struct AI
raised_hands1
56
18mo
Solved
Join the chat
Mar 04, 2022 (18 months ago)
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
01:25 AM
Hi All- I’m exploring Typesense for the first time and walking through the demo here: https://typesense.org/docs/guide/search-ui-components.html#building-search-uis. When I try to perform a search on the page I get the following error:
[Error] Error: 404 - Could not find a facet field named `*` in the schema. — TypesenseInstantsearchAdapter.js:81
    _callee$ (app.a6a4d504.js:7599)
    tryCatch (app.a6a4d504.js:192)
    invoke (app.a6a4d504.js:423)
    asyncGeneratorStep (app.a6a4d504.js:890)
    _next (app.a6a4d504.js:912)
    promiseReactionJob
[Error] Unhandled Promise Rejection: Error: 404 - Could not find a facet field named `*` in the schema.
    (anonymous function) ([email protected]:2:32449)
    (anonymous function) ([email protected]:2:105993)
    (anonymous function) ([email protected]:2:32639)
    (anonymous function) (instantsearch.js@4.39.1:2:49241)
    (anonymous function)
    promiseReactionJob

Is this something I am missing in app.js?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
01:39 AM
Hmmm, I wonder if this is an issue with a recent version of create-instantsearch-app. Do you see a widget called dynamicWidgets in app.js? If so, could you remove it and try again?
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:01 AM
Different. Now I get Could not find a facet named 'abstract' in the schema . One of my fields in my schema is called “abstract”.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:02 AM
Could you share your full schema?
02:02
Jason
02:02 AM
Also, is the "abstract" field set to "facet: true" in the schema?
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:04 AM
Sure, below. It is set to “false”. I’m not sure what that means.
{
  "name": "ascl_entries",
  "fields": [
    {
      "name": "ascl_id",
      "type": "string",
      "facet": false,
      "optional": false,
      "index": true
    },
    {
      "name": "title",
      "type": "string",
      "facet": false,
      "optional": false,
      "index": true
    },
    {
      "name": "credit",
      "type": "string",
      "facet": false,
      "optional": false,
      "index": true
    },
    {
      "name": "abstract",
      "type": "string",
      "facet": false,
      "optional": false,
      "index": true
    },
    {
      "name": "topic_id",
      "type": "string",
      "facet": false,
      "optional": false,
      "index": true
    },
    {
      "name": "bibcode",
      "type": "string",
      "facet": false,
      "optional": false,
      "index": true
    },
    {
      "name": "views",
      "type": "string",
      "facet": false,
      "optional": false,
      "index": true
    },
    {
      "name": "preferred_citation",
      "type": "string",
      "facet": false,
      "optional": false,
      "index": true
    },
    {
      "name": "site_list",
      "type": "string[]",
      "facet": false,
      "optional": false,
      "index": true
    },
    {
      "name": "used_in",
      "type": "string[]",
      "facet": false,
      "optional": false,
      "index": true
    },
    {
      "name": "described_in",
      "type": "string[]",
      "facet": false,
      "optional": false,
      "index": true
    }
  ],
  "default_sorting_field": ""
}

Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:04 AM
You want to set "facet": true for the abstract field if you want to use it with instant search
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:04 AM
This is how it was created in Python:
schema = {
    "name" : "ascl_entries",
    "fields" : [
        { "name":"ascl_id", "type":"string" },
        { "name":"title", "type":"string"},
        { "name":"credit", "type":"string"},
        { "name":"abstract", "type":"string"},
        { "name":"topic_id", "type":"string"},
        { "name":"bibcode", "type":"string"},
        { "name":"views", "type":"string"},
        { "name":"preferred_citation", "type":"string"},
        { "name":"site_list", "type":"string[]"},
        { "name":"used_in", "type":"string[]"},
        { "name":"described_in", "type":"string[]"}
        #{ "name":"keywords", "type":"string[]"},
    ]#,
#  "default_sorting_field": "title" # optional
}
02:05
Demitri
02:05 AM
Great, I will try that.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:05 AM
On a side note, these look like academic papers? If so, any reason you're trying to filter/facet by a long description field like abstract?
02:06
Jason
02:06 AM
Filtering is only useful if most records share common values for the field you're filtering on
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:09 AM
Adding "facet":True to the schema worked. I don’t have the text of the full papers, just the abstracts. I’m not sure what you mean though; the intent was to be able to filter on arbitrary words, e.g. “hydrogen”.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:10 AM
Abstracts are essentially paragraph summaries of the paper right?
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:10 AM
Yes.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:11 AM
If you want to find keywords within the abstract, you want to use that field in query_by
02:12
Jason
02:12 AM
Wait, actually I (might have) assumed (incorrectly) that you're trying to use the refinementList widget with the instantsearch. If my assumption is wrong, ignore me!
02:13
Jason
02:13 AM
But I'm curious how your instantsearch widget configuration code looks, that would require the abstract field to be set as a facet...
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:14 AM
If you are referring to the keywords field in my schema, the data set I am using as a sample doesn’t have any of those populated. I’m not familiar with the refinementList? This is literally my first time using Typesense; just trying to get a minimum working example to see how I might integrate it into a web app. This is the whole of the app.js, pretty short. I minimally modified it from the template.
const { algoliasearch, instantsearch } = window;

import TypesenseInstantSearchAdapter from "typesense-instantsearch-adapter";

const typesenseInstantsearchAdapter = new TypesenseInstantSearchAdapter({
  server: {
    apiKey: "zPOYYT6TdNOIK9h87bzv5GPtZXIMaxJC", // Be sure to use the search-only-api-key
    nodes: [
      {
        host: "localhost",
        port: "8108",
        protocol: "http"
      }
    ]
  },
  // The following parameters are directly passed to Typesense's search API endpoint.
  //  So you can pass any parameters supported by the search endpoint below.
  //  queryBy is required.
  additionalSearchParameters: {
    query_by: "title,abstract"
  }
});
const searchClient = typesenseInstantsearchAdapter.searchClient;

const search = instantsearch({
  searchClient,
  indexName: "ascl_entries"
});

search.addWidgets([
  instantsearch.widgets.searchBox({
    container: '#searchbox',
  }),
  instantsearch.widgets.hits({
    container: '#hits',
       templates: {
          item: `
            <div>
              <img src="" align="left" alt="" />
              <div class="hit-name">
                {{#helpers.highlight}}{ "attribute": "title" }{{/helpers.highlight}}
              </div>
              <div class="hit-abstract">
                {{#helpers.highlight}}{ "attribute": "abstract" }{{/helpers.highlight}}
              </div>
              <div class="hit-ascl_id">ASCL ID: </div>
            </div>
          `,
        },
  }),
  instantsearch.widgets.configure({
    facets: ['abstract'],
    maxValuesPerFacet: 20,
  }),
//   instantsearch.widgets.dynamicWidgets({
//     container: '#dynamic-widgets',
//     fallbackWidget({ container, attribute }) {
//       return instantsearch.widgets.refinementList({
//         container,
//         attribute,
//       });
//     },
//     widgets: [],
//   }),
  instantsearch.widgets.pagination({
    container: '#pagination',
  }),
]);

search.start();

Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:15 AM
Ah, I see what's triggering that.

You can also comment out this part:

instantsearch.widgets.configure({
    facets: ['abstract'],
    maxValuesPerFacet: 20,
  }),
02:15
Jason
02:15 AM
And then you should be able to remove facet: true from abstract
02:16
Jason
02:16 AM
For context: here's what faceting is usually used for: https://typesense.org/docs/0.22.2/api/documents.html#facet-results
02:17
Jason
02:17 AM
Setting facet: true on a field with almost all unique values across records ends up using a lot of memory...
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:18 AM
Ah, ok. Yes, commented that out and it worked. Should I be thinking “facet” == “keyword”?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:18 AM
Yup, that's a good field to set as a facet
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:19 AM
Oh, I just mean conceptually.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:23 AM
Ah, Keyword is a little broad to use in this context...

Here's one example: if you have a list of products, you'd typically set color, brand as faceted field, so you can display a filter to users that lets them drill-down products based on color or brand.
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:28 AM
Thanks, that makes sense. If you don’t mind me asking, I’m hoping to create an interface that allows users to search for small blog entries, think Stack Overflow sized or a little bigger. I want to be able to specify both keyword searches (so, faceted) and free form text. If one selects “planets” as a keyword, I’d get a unique list of all keywords for the entries that match “planets”. Is something like this possible? Would that UI need to be built from scratch?
02:30
Demitri
02:30 AM
I think the more general question is that I have the working example, but I’m not sure how to take it and customize it for my purposes.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:30 AM
> I’d get a unique list of all keywords for the entries that match “planets”.
Did you mean unique list of records, or keywords?
02:32
Jason
02:32 AM
Or do you have a UI mockup you're trying to build?
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:36 AM
Unique list of keywords. Selecting one would display a list such that if a second were selected, at least one entry would be found. I have a concept in my head; still need to sketch it out.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:36 AM
Sounds like a hierarchy of keywords then?
02:37
Jason
02:37 AM
In any case, yeah this should be doable with Instantsearch, but a hierarchy of checkboxes might need a custom widget - they only have a hierarchy of single-select items out of the box
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:37 AM
Yes, functionally.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:38 AM
For eg: see the browse by categories widget on the left here: https://ecommerce-store.typesense.org/
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:38 AM
I don’t visualize it as hierarchical. For example, let’s say the corpus is code snippets. Selecting “python” would include “file I/O”, “regex”, etc. but maybe not “Objective-C”.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:39 AM
Ah, you would have to build that mapping and put it inside each record
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:40 AM
Given above, I would have “python” and “file I/O” as predefined keywords in the record. Is that what you mean?
02:40
Demitri
02:40 AM
In the interface you linked, I suppose I’m looking for an AND search rather than OR.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:40 AM
So you'd have create an array field called say "keywords" and add "python" to the list, for any of records that have "“file I/O”, “regex”, etc
02:43
Jason
02:43 AM
> In the interface you linked, I suppose I’m looking for an AND search rather than OR.
This is also possible. The brand filter uses this refinementList widget which does an OR by default. You can change it to AND like this: https://www.algolia.com/doc/api-reference/widgets/refinement-list/js/#widget-param-operator
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:45 AM
Right, that would work. Fully curated content. Re: link, ah, perfect. I am very experienced on the backend side of things but relatively new to JS frontends+packaging. What would you recommend I learn next to build this? Also, the example above is a full app; is this something I can embed inside an existing app (e.g. Flask, not React)?
02:49
Demitri
02:49 AM
Though this is a complex question - I don’t want to take too much time from your evening! I saw that you have office hours. I can book something there if that’s better.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:49 AM
Yup, you can use regular <script> tags to load instantsearch like this: https://www.algolia.com/doc/guides/building-search-ui/installation/js/#directly-in-your-page

You can also load the instantsearch adapter to get those widgets to work with Typesense via a script tag like this: https://github.com/typesense/typesense-instantsearch-adapter#installation

And after that, you can just use regular JS inside script tags in your html views to load the widgets
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:51 AM
Ah, I think that’s exactly what I was looking for. Are these widgets pure JS or part of a framework? I suppose I can start by looking at the source of one.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:52 AM
The widgets come as part of instantsearch.js and are standalone, so you don't have to install any other frameworks
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
02:53 AM
Thanks, that’s really helpful. I will dive into the documentation and see how far I get. Thank you so much for your time and help, I really appreciate it!
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:54 AM
Happy to help!
04:41
Jason
04:41 AM
CC: Helin ^
Helin
Photo of md5-13cb5b835890cb2f6752bb2525a6ea01
Helin
04:47 AM
Happy to help another Python developer to build frontend! Demitri, this is a demo app made purely by Python (~100 lines of code): https://demo.pyweb.io/typesense/app/ let me know if you’re interested in its implementation details.
Mar 15, 2022 (18 months ago)
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
03:28 PM
Hi Helin - sorry, I only just saw your reply! Thanks so much for the pointer. I’d definitely be interested in hearing more about the implementation.
Mar 30, 2022 (18 months ago)
Helin
Photo of md5-13cb5b835890cb2f6752bb2525a6ea01
Helin
07:35 PM
Demitri sorry for not checking messages for a long time…. Will send it over.
Demitri
Photo of md5-01b4600434aa419becd17a9f7773e2ff
Demitri
08:09 PM
Helin Thanks! I appreciate it.
Helin
Photo of md5-13cb5b835890cb2f6752bb2525a6ea01
Helin
10:23 PM
Demitri here is the source code: https://github.com/pywebio/demos/blob/main/typesense_search_app.py
You can either try it on localhost (after pip install pywebio), or run it on build.pyweb.io.
Note that the live demo does not work anymore as my Typesense cloud usage has exceeded the free tier limit. I should probably ask for a complete free account for this demo purpose (cc Jason 🙂 )
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
10:50 PM
Helin Feel free to use the endpoints and API key used in this demo https://recipe-search.typesense.org/. You should be able to get it from the network tab
raised_hands1
Mar 31, 2022 (18 months ago)
Helin
Photo of md5-13cb5b835890cb2f6752bb2525a6ea01
Helin
12:06 AM
Thanks to Jason’s support, the live demo is back up online with full search and create feature now: https://demo.pyweb.io/live/typesense_search/app/ . 🎉