https://typesense.org logo
Join Slack
Powered by
the consequence of this I assume is that my data w...
# community-help
r
the consequence of this I assume is that my data will take longer to retrieve if I want to retrieve those fields (not search through them)? Anything else I should be aware of?
j
That's pretty much it. We do use RocksDB to store these docs on disk, so it should still be fast to retrieve. But if you notice any performance issues, using SSDs would help
r
so question, why would one use this? dataset too big and can't fit on ram?
j
This would be useful for cases where you don't need to search through all the data in your record, but still want to use the data from the record for say display purposes. So instead of having to make an API call to Typesense first to search through the data and then separately make an API call to your database to fetch other fields for the record, you can put all related data in Typesense to reduce multiple API calls
For eg: let's say you're storing metadata about videos in your records. You want to allow users to search by title and author of the video, and you also want to link to say the Youtube link in your search UI. Though you're not searching directly in the Youtube link field, you can still store it in the record, so you can use to render your search UI efficiently, with just the response from Typesense
r
thanks for the detailed explanation
that's what I was going to use it for. but I'm concerned it might impact the results if they all have this extra field that I don't search through. that said thinking about it, the extra database call, wouldn't be any better.
This isn't a concern now I'm just getting started
but is there a way to keep it in memory but not be searchable?
ignore that I define what fields get searched anyway, so it doesn't matter
thanks 🙂
j
I'd recommend benchmarking to see how much performance impact there is if you add non-indexed fields to the document. I'd suspect it's minimal based on what I've seen. For eg, in the songs showcase I do exactly what I mentioned above - the URLs for each song are unindexed and stored on disk and it still seems pretty fast.
Correction to what I said earlier: once the list of document IDs are determined from in-memory indices, we actually fetch all fields from disk (unless specified otherwise in the 
include_fields
param) to assemble the final result document, regardless of whether it's indexed or not. So performance will be identical.
r
thanks
@Jason Bosco just a follow up on previous statement. So even if all my fields are in memory indices, everything (or what's specificied in include_fields) still gets fetched from the disk?
j
That’s correct
Everything = the final documents that will be returned as part of the response, so 10 documents by default since that’s the default per_page value
👍 1
Open in Slack
PreviousNext
Powered by