I am using Airbyte to push data to Typesense. `fac...
# community-help
j
I am using Airbyte to push data to Typesense.
facet
is set to
false
for all of the fields:
"facet": false
. It looks that’s the default. I can update the fields on typesense Cloud, but after a full sync, all the changes are reverted. What is the best way to persist
"facet": true
for certain fields? Previously, I was able to set
"enable_nested_fields": true,
by creating a custom connector for Airbyte and I hope this change doesn’t require that.
j
It sounds like Airbyte might be creating a new schema on each full sync. So you would have to indeed use a custom connector and then set your schema in that connector
👍 1
This use-case would also be a good thing to bring up with the AirByte team, to see if they can allow for a custom schema to be set in the connector config, instead of having to create a whole new connector just for this purpose
👍 1
j
@Jason Bosco any new considerations on this? Airbyte doesn’t do this. What other options I have?
j
You could try building your own airbyte connector as a fork of the Typesense one
Or alternatively, here are other strategies to consider: https://typesense.org/docs/guide/syncing-data-into-typesense.html
j
I made one, the only problem is that I have to hard code the fields that I want the
facet
to be
true
. If any of those fields change, or I am dealing with another source, I have to update my connector, deploy again and re-do all my Airbyte connections to use it. It will be hard to manage, unless I am missing something?
Added info: Our data rests in BigQuery before typesense. If that helps with any suggestions.
j
I can't speak to the airbyte configurability piece... I'm not too familiar with it