If I use filter_by := and pick a field that is optional, is there a way to not have it filter out the documents that don't have that field? In the example below, ideally I would want to filter_by shoes and have doc1 and doc2 returned (doc1 because it matches the filter, doc2 because it does not have the category field at all)

filter_by category:=shoes

doc1:
{
  category: "shoes",
  other: ...
}

doc2:
{
  other: ...
}

doc3:
{
  category: "boots",
  other: ...
}

The only way to do something like this is to have some placeholder value for the missing value like

NIL

and then use optional filtering to prioritize field values that are not NIL.

{
  "sort_by": "_eval(other!:=NIL):desc"
}

This will prioritize fields without NIL value ahead but won't entirely drop NIL fields.

Some docs for filtering on missing values: https://typesense.org/docs/guide/tips-for-filtering.html#filtering-for-empty-fields And some docs on sorting based on conditions, that Kishore is referring to: https://typesense.org/docs/29.0/api/search.html#sorting-based-on-conditions

is this a feature that could be requested? (having filtering not remove documents where the field does not exist)

We do have a feature request open for handling missing values (to query for them etc.). We will take a look at what we can do here as part of that in future.