#community-help

Discussion on Typesense Data Size and Locating API Key

TLDR Sruli asked whether Typesense could handle a large character load. Jason assured this was possible, and corrected an initial overestimate about data size to avoid unnecessary cost for the user. Sruli also queried about a local API key, which Jason guided to find.

Powered by Struct AI

2

16
10mo
Solved
Join the chat
Dec 23, 2022 (10 months ago)
Sruli
Photo of md5-8c57c435e01337c18bf32b8f7749cabf
Sruli
04:06 PM
Hi everyone 👋
Was curious about the limits of the full-text searching. Will typesense be able to search through say a list of books that contain a certain word or phrase efficiently?
It shouldn’t be more than 500000 characters long.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:10 PM
Yeah this should work fine with Typesense. 500K characters, is about 500MB 500KB of data… I’ve seen users index 100s of GB of data in Typesense.
04:11
Jason
04:11 PM
You want to break out the book contents into one record per paragraph as described here: https://typesense.org/docs/guide/tips-for-searching-common-types-of-data.html#long-pieces-of-text
Sruli
Photo of md5-8c57c435e01337c18bf32b8f7749cabf
Sruli
04:11 PM
Are you sure about that 500MB number? In a txt file it’s only around 900kb
04:11
Sruli
04:11 PM
That would increase my costs dramatically
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:12 PM
Oh wait, sorry bad math.
04:12
Jason
04:12 PM
500K characters is 500KB
04:14
Jason
04:14 PM
Typesense requires about 2x-3x of RAM. So for every 500KB of data, you’d need about 1.5MB of RAM
Sruli
Photo of md5-8c57c435e01337c18bf32b8f7749cabf
Sruli
04:17 PM
Thanks! 👍

1

05:54
Sruli
05:54 PM
Having trouble finding my API key when running locally, do I first need to create an account?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
05:54 PM
May I know how you’re running Typesense?
05:55
Jason
05:55 PM
Did you use docker / native binary or RPM / DEB package?
Sruli
Photo of md5-8c57c435e01337c18bf32b8f7749cabf
Sruli
05:55 PM
Mac
05:55
Sruli
05:55 PM
Used brew to install
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
05:56 PM
You’ll find the API key in the location mentioned in the text just below the code snippet: https://typesense.org/docs/guide/install-typesense.html#from-homebrew-on-macos
Sruli
Photo of md5-8c57c435e01337c18bf32b8f7749cabf
Sruli
05:56 PM
Cool, thanks!

1