Hello! I'm using typesense in a intensive write environment. I'm using client side batching, and server side batching in default (40). I'm inserting/updating nearly 20 million documents continually within two hours, but I'm suffering from a bottleneck. I'm using 4VCPU + 16gb machine with v29. Are there any best practices or tips in these use cases?

@Nicolas, This is quite a substantial amount of data, and in this case, the CPUs will play a much more significant role in processing than the RAM. You'll want to check the import API and parallelize it accordingly. If you are using embeddings, enabling the GPU is highly recommended. Here are a few key points to keep in mind: - Utilize the bulk import API to efficiently load data into your newly created cluster. I recommend starting with a batch size of 1,000 documents per import API call, and setting the concurrency to N-1 parallel import API calls, where N represents the number of CPU cores in your cluster. - You may encounter 503 errors when importing into Typesense, as this is part of the built-in back-pressure mechanism. Be sure to handle these errors in your indexing pipeline, as described in the documentation: https://typesense.org/docs/guide/syncing-data-into-typesense.html#handling-http-503s If you still face a significant bottleneck after making these adjustments, it may indicate that 4 vCPUs are insufficient for your use case without experiencing bottlenecks.

Hi @Alan Martini, thanks for you help. I'm not using embeddings. I'm using bulk import. Do you recommend increase this server-side batching from TS? On docs I saw that's saying "you rarely want to modify this parameter". Okay, so with 4vCPUs, it would be 2 parallel imports for a better performance? I never encoutered 503 error during this insertions. It's just taking too long to process them, going into timeout.

I'm using bulk import. Do you recommend increase this server-side batching from TS? On docs I saw that's saying "you rarely want to modify this parameter".

IIRC they are the same. It will be the amount that TS will send over Typesense. On the second point, most users won't have an intensive writing pattern, especially of 10mi/hours, so there is no need to tweak it.

Okay, so with 4vCPUs, it would be 2 parallel imports for a better performance?

Your cluster has 4vCPUs? Then it's 3

So, I'm batching 5k and inserting using batch_size of 1k on Typesense.

Does it make sense? Or would be better if both were 1k?

1k is a good starting point, but you'll really want to measure what works best in your scenario. Try with 1k, then 2k. If it yields good results, try increasing further. If not, try decreasing to 500 and so on.

@Alan Martini does the size of the collection influences on the velocity on indexing?

Absolutely. The size of each document as well. The more documents and the bigger they are (and the more complex, with objects and so on), the longer it should take. Some datasets can take as long as 10 hours to index.

@Alan Martini great points. Thank you very much. I saw that the batch_size parameter on bulk import is the way that TS will queue the insertions. I'm wondering if the client side batch should be the same size of server-side batching?

I'm not sure, will have to check it Hey @Fanis Tharropoulos do you have this info on the top of your head?

thanks anyway, @Alan Martini!

@Nicolas which batch_size are your saying? the parameter for the server? That one will be when the server initializes or restart. Else, the batch amount you set on your client side in the amount that's being sent/processed by Typesense

ARM vs AMD/Intel is difficult to say: depends a lot on workload. But we do also use ARM on Typesense Cloud so it's well supported.

thanks @Kishore Nallan. @Alan Martini I mean this parameter: https://typesense.org/docs/guide/syncing-data-into-typesense.html#client-side-batch-size-vs-server-side-batching

Can you share your schema? I can review it to see what could be causing bottle necks.

yeah, sure @Kishore Nallan.

[{'async_reference': True,

'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'reference': 'collection2.id',

'sort': False,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'string'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'int64'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'int64'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'float'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'float'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'float'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'int32'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'int32'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'int32'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'bool'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'bool'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'int32'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'bool'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': False,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'string'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': True,

'sort': False,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'string'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': False,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'string'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': False,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'string'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': False,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'string'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': False,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'string'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': False,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'string'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': False,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'string'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'int64'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'float'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': False,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'string'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': True,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'int64'},

{'facet': False,

'index': True,

'infix': False,

'locale': '',

'name': 'key',

'optional': False,

'sort': True,

'stem': False,

'stem_dictionary': '',

'store': True,

'type': 'float'}],

{'facet': False,
    'index': True,
    'infix': False,
    'locale': '',
    'name': 'key',
    'optional': False,
    'sort': False,
    'stem': False,
    'stem_dictionary': '',
    'store': True,
    'type': 'string'},

Enabling sorting on string type can be a very cpu intensive operation.

hm, but it's an interesting observation, I can see a lot of sort : True, but today, i'm not using sorting at all. So, turning off these can improve insertion performance, right?

Yes, definitely

Yeah, I've 1 field that use a reference, using async reference. And I really thought that this async reference would cause the contrary effect on performance

Are you planning to join this collection with another?

not now. All the things I need are "already joined"

Then you can remove the reference and async reference as well.

but this is the only one I need to join

Try first without async_reference to see if that helps also.

thank you very much, @Kishore Nallan!

We don't rely too much on batch_size parameter in import calls anymore. We try to batch intelligently inside.

@Kishore Nallan I applied your advices. • I trim up my collection to contain only the needed fields. • Remove the sort = True • change to graviton + CPU optimized

You can try increasing some of these default rocksdb parameters also: https://typesense.org/docs/29.0/api/server-configuration.html#on-disk-db-fine-tuning

Thanks, @Kishore Nallan, I'll give a look! Are there something else I can do?

I think that's all I can think of.

Thank you very much for your support!