https://typesense.org logo
Join Slack
Powered by
Hi guys and girls :wave:, I'm currently trying ou...
# community-help
s
Hi guys and girls 👋, I'm currently trying out the Semantic Vestor Search with the embedding an ML model. We have a 2 RAM cloud server with autoscaling enabled. but the node keeps going into OUT_OF_MEMORY. I'm not sure what we need to do to make this workable?
these are my fields (within PHP)
Copy code
[
                'name' => 'city',
                'type' => 'int32',
                'sort' => true,
            ],
            [
                'name' => 'contractType',
                'type' => 'string',
                'facet' => true,
            ],
            [
                'name' => 'contractTypeFull',
                'type' => 'auto'
            ],
            [
                'name' => 'coordinates',
                'type' => 'geopoint'
            ],
            [
                'name' => 'end_date',
                'type' => 'int32',
            ],
            [
                'name' => 'id',
                'type' => 'string',
            ],
            [
                'name' => 'office',
                'type' => 'string',
            ],
            [
                'name' => 'openings',
                'type' => 'number',
            ],
            [
                'name' => 'post_code',
                'type' => 'string',
            ],
            [
                'name' => 'post_date_timestamp',
                'type' => 'int32',
            ],
            [
                'name' => 'start_date',
                'type' => 'int32',
            ],
            [
                'name' => 'sector',
                'type' => 'string',
                'facet' => true,
            ],
            [
                'name' => 'sectorFull',
                'type' => 'auto'
            ],
            [
                'name' => 'shift',
                'type' => 'string',
                'facet' => true,
            ],
            [
                'name' => 'shiftFull',
                'type' => 'auto'
            ],
            [
                'name' => 'title',
                'type' => 'string',
                'sort' => true,
            ],
            [
                'name' => 'wageMaximum',
                'type' => 'number'
            ],
            [
                'name' => 'wageMinimum',
                'type' => 'number'
            ],
            [
                'name' => 'workRegime',
                'type' => 'string',
                'facet' => true,
            ],
            [
                'name' => 'workRegimeFull',
                'type' => 'auto'
            ],
            [
                'name' => 'embedding',
                'type' => 'float[]',
                'embed' => [
                    'from' => ['title'],
                    'model_config' => [
                        'model_name' => 'ts/multilingual-e5-large'
                    ]
                ],
            ],
k
multilingual-e5-large
requires atleast 2.5G RAM of memory I think. It might not fit with 2 GB RAM server. Try a smaller model.
s
thank you for the quick reply @Kishore Nallan, which model should you recommend? I need semantic search in Dutch language only
k
For specific languages you will have to try the models and decide. But usage of 
-large
models will exceed 2 GB memory.
s
ok, I'm still learning about it all
it has been 15 minute since I've tried the apply of the model. but my node is still spiking, is this normal @Kishore Nallan?
can I cancel processes in the background or so?
it keeps switching between the Healthy and OUT_OF_MEMORY constantly
but I can't do anything at the moment
k
What do you mean by "apply of the model"? The cluster might have gotten into an unstable state because of out of memory. You have to try dropping the collection or can just terminate and recreate the cluster.
s
the thing is, it was with the creation of the collection and the collection isn't even there in the list
k
Yeah it's probably struggling to load the model into memory via swap.
s
by terminating the cluster, I will loose all data @Kishore Nallan?
m
Can try to remove the collections first.
k
Yes you have to reindex the data. If this is a production cluster you can DM me the cluster ID and I can try and fix it.
m
No worries, it's not a production cluster yet. I will try deleting the collections first, if it doesn't resolve, I'll spin up another cluster 🙂
👍 1
k
We do have some checks to prevent this from happening (verifying that there is enough memory for large model creation to go through so I am surprised why this didn't work here). So I will still appreciate if you can DM me the cluster ID so I can confirm.
Cluster has recovered.
s
thx @Kishore Nallan 🙏 , am I safe to try 
'ts/multilingual-e5-small'
model with our setup?
k
Yes that should work unless there is a bug with the multilingual family that causes the actual problem. Please try, we will know soon! 🙂
s
I'll keep you posted
👍 1
m
Also upgrading to 4GB at the moment, just to be sure 😅
Open in Slack
PreviousNext
Powered by