#community-help

Ensuring Uniqueness During Records Creation in Typesense

TLDR Krish inquired about ensuring uniqueness during record creation in Typesense. Kishore Nallan explained that uniqueness can be maintained using the 'id' field in documents, and when creating documents, the system automatically assigns a unique id if none is provided.

Powered by Struct AI
raised_hands1
19
9mo
Solved
Join the chat
Dec 14, 2022 (9 months ago)
Krish
Photo of md5-7dff5651efcba5c9cdfac93918dc7eeb
Krish
08:20 AM
Another question: while creating records how do I ensure uniqueness? I tried looking up your doc, but didn't find anything. Does Typesense check for dup documents while creation or should the client implement it?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:04 AM
The id field in the document is used as the identifier by convention. It can be used to ensure uniqueness.
Krish
Photo of md5-7dff5651efcba5c9cdfac93918dc7eeb
Krish
11:31 AM
So, are you saying that it would error out if there is an id collision during a create() call?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:31 AM
Yes
Krish
Photo of md5-7dff5651efcba5c9cdfac93918dc7eeb
Krish
11:32 AM
Which also means, the client will have to supply a custom id for the create calls based on some algo, right?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:33 AM
If no id is provided then id is autoassigned in running sequence. If an id is provided we will use that.
Krish
Photo of md5-7dff5651efcba5c9cdfac93918dc7eeb
Krish
11:34 AM
got it, essentially if no id is provided we'd never run into an id collision during a create() call.
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:34 AM
Correct. During create call, if you pass the return_id=true flag then the created id is returned (in 0.24 RC)
Krish
Photo of md5-7dff5651efcba5c9cdfac93918dc7eeb
Krish
11:35 AM
ahh, interesting... ok I on 0.23, not sure if its there in it, But it doesn't make sense for the client to keep track of ids, right?
11:37
Krish
11:37 AM
I was thinking of generating a number of type int32 using crc32 hashing mechanism on the doc. Would you recommend anything else?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:38 AM
id must be a string. You can just hash the document to produce a SHA and use that as ID.
Krish
Photo of md5-7dff5651efcba5c9cdfac93918dc7eeb
Krish
11:38 AM
wouldn't sha be compute heavy?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:39 AM
Depends, but there are other non-cryptographic hashes available which might be faster.
Krish
Photo of md5-7dff5651efcba5c9cdfac93918dc7eeb
Krish
11:40 AM
could you recommend one? One you thing would be faster?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:40 AM
murmurhash or xxhash
Krish
Photo of md5-7dff5651efcba5c9cdfac93918dc7eeb
Krish
11:40 AM
All this while I thought ID has to be a number. Glad its a string
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:41 AM
The Typesense assigned id happens to be an integer simply because we have a running sequence to choose from that won't duplicate but it can be anything.
Krish
Photo of md5-7dff5651efcba5c9cdfac93918dc7eeb
Krish
11:41 AM
awesome ,,,
11:42
Krish
11:42 AM
Thank you Kishore. I think I have what I need to proceed.
raised_hands1