Hi <@U01PL2YSG8L>, i just checked your answer unde...
# community-helpa
Anton Khatunzev
09/30/2021, 3:58 PMHi @Kishore Nallan, i just checked your answer under https://github.com/typesense/typesense/issues/329
is there documentation or any examples for this feature available?
k
Kishore Nallan
09/30/2021, 3:59 PMSorry, let me post a comment to that issue.
a
Anton Khatunzev
09/30/2021, 4:01 PMAnd one more question: i do need to install v0.22.0 server to use this feature, correct? If it is so, where or when can i get it?
k
Kishore Nallan
09/30/2021, 4:02 PMYes, Linux binary is fine?
Kishore Nallan
09/30/2021, 4:02 PMI've updated my comment on that issue with usage instructions.
a
Anton Khatunzev
09/30/2021, 4:05 PMyes, i need the linux binary
k
Kishore Nallan
09/30/2021, 4:08 PMDownload here:
Copy code
<https://dl.typesense.org/releases/0.22.0.rcs10/typesense-server-0.22.0.rcs10-linux-amd64.tar.gz>a
Anton Khatunzev
09/30/2021, 4:10 PMthank you!
š 1
Anton Khatunzev
10/04/2021, 12:43 PMhello Kishore, i have couple updates:
1. Geosearch seems to be working fine and fast. I had no time to debug it properly though...
2. With v.0.22.0 server i got that bad issue when indexing documents(using upsert). We have a lot of documents(about 75k) in our db that needs to be imported into TS. Documents themselves are pretty big. In my current scenario they contain a lot of text and some big non-indexed json fields. We import these documents in batches(5k records in a batch). At the end of the import of the each batch i analyze the response returned by TS client. And it appeared that when i sent 5k records payload, i got back response containning ~4.8k "ImportDocumentResponse" objects. No errors. So my code throws an exception at this stage. I decided to check if this happens on a smaller batches. And it did. I got the same problem on 2k records batch: 2k documents sent, 1996 ImportDocumentResponse items back. So i also decided to check if documents in the tail of the payload are really missing in the TS collection. And it seems that they are there(they do exist on TS side)! Very weird...Also, i need to mention that some time ago i had a problem with similar symptoms. I was able to solve it by setting ConnectionTimeout setting to 0 for my TS client. But now i'm a bit lost... i will try to find more precise testcase... But for now just letting you know that v0.22.0 behaves differently during the import...
k
Kishore Nallan
10/04/2021, 12:46 PM@Anton Khatunzev We've made the import atomic in 0.22.0 -- earlier it is possible that if your connection times out mid-way, imports could end mid way too. However, with 0.22.0 the entire response body is first buffered and then only indexing begins, so it is atomic.
Kishore Nallan
10/04/2021, 12:47 PMCan you please try using curl and see if you find a similar mismatch in response?
a
Anton Khatunzev
10/04/2021, 12:48 PM@Kishore Nallan Hi, yes, i think i will have to... As i said, the payloads are quite big, so i'll try to find more precise testcase first
Anton Khatunzev
10/04/2021, 2:13 PMhello @Kishore Nallan
Anton Khatunzev
10/04/2021, 2:14 PMi got the testcase
Anton Khatunzev
10/04/2021, 2:14 PMtried it with 3k batch using curl. And i got 2997 items in the response
Anton Khatunzev
10/04/2021, 2:14 PMit happens from time to time
Anton Khatunzev
10/04/2021, 2:15 PMwould it be possible to do the screenshare?
k
Kishore Nallan
10/04/2021, 2:18 PMOh cool, is it possible to share that dataset with me?
a
Anton Khatunzev
10/04/2021, 2:19 PMyes
Anton Khatunzev
10/04/2021, 2:19 PMbut it is 100M jsonl. How would you like to get it?
Anton Khatunzev
10/04/2021, 2:20 PMcan we do a screenshare maybe? so you can see the problem?
k
Kishore Nallan
10/04/2021, 2:20 PMDo you have a public s3 bucket I can download from? You can encrypt with my gpg ID.
Kishore Nallan
10/04/2021, 2:20 PMI can do a screenshare in 30 mins.
a
Anton Khatunzev
10/04/2021, 2:22 PMok, we dont use our s3 for such things, so i'll prepare gdrive link if you dont mind
k
Kishore Nallan
10/04/2021, 2:23 PMYes that works!
a
Anton Khatunzev
10/04/2021, 2:31 PMk
Kishore Nallan
10/04/2021, 2:36 PMDownloading
Kishore Nallan
10/04/2021, 2:40 PMDone, I will try reproducing locally.
a
Anton Khatunzev
10/04/2021, 2:40 PMyou may need to trigger it several times. Sometimes it returns 3k items as it should, but sometimes less than 3k which is not correct. Numbers are always different btw
k
Kishore Nallan
10/04/2021, 2:40 PM@Anton Khatunzev Do you use HTTPS?
a
Anton Khatunzev
10/04/2021, 2:40 PMat the moment no
k
Kishore Nallan
10/04/2021, 2:41 PMOkay I will try with http then.
Kishore Nallan
10/04/2021, 2:49 PMUnable to reproduce. Tried 10 times š¤ Are you running this against a server or on local dev?
a
Anton Khatunzev
10/04/2021, 2:53 PMagainst the server
Anton Khatunzev
10/04/2021, 2:53 PMi can show you i think
k
Kishore Nallan
10/04/2021, 2:53 PMOkay let's do that.
a
Anton Khatunzev
10/04/2021, 2:54 PMk
Kishore Nallan
10/05/2021, 12:28 PM@Anton Khatunzev I tried uploading multiple times to a server in Frankfurt, which is pretty far from me, but I could not reproduce the line count mismatch. Can I give you a debug build with the
ida
Anton Khatunzev
10/05/2021, 12:29 PMhi, yes, sure. But i dont know when i will be able to test this. Bit busy with another stuff at the moment...
k
Kishore Nallan
10/05/2021, 12:31 PMOkay, no worries, please ping when you are free sometime this week. In the mean time, I will try launching another server even further away and see if that helps to reproduce.
Kishore Nallan
10/05/2021, 12:49 PMAlso, are you sure that the typesense process did not crash? The truncated http response can also happen if the typesense server had crashed and was subsequently restarted by systemd. Checking the process age will confirm if process had crashed.
a
Anton Khatunzev
10/05/2021, 2:00 PMok, next time i'll take a look at the process age. Yesterday it looked like it did not crash. But i'm not 100% sure
k
Kishore Nallan
10/05/2021, 2:01 PMš
Kishore Nallan
10/07/2021, 10:18 AM@Anton Khatunzev I've been able to reproduce this issue. Working on a fix. Will keep you posted.
a
Anton Khatunzev
10/07/2021, 10:19 AMGreat! Thank you
k
Kishore Nallan
10/09/2021, 4:02 PM@Anton Khatunzev Please try with this binary: http://dl.typesense.org/releases/0.22.0.rcs14/typesense-server-0.22.0.rcs14-linux-amd64.tar.gz
Kishore Nallan
10/12/2021, 10:42 AM@Anton Khatunzev Did you get a chance to try with
rcs14a
Anton Khatunzev
10/14/2021, 11:19 AMhi @Kishore Nallan, sorry i forgot to give you feedback. Yes, i installed the rcs14. And it seems to work fine with 500-items batches. No errors. Previously i had problems even with smaller ones. At the moment i cant check it with bigger batches. But probably will be able to do that soon
Anton Khatunzev
10/14/2021, 11:19 AMBut i think the problem gone away
k
Kishore Nallan
10/14/2021, 11:45 AMThanks for confirming!
a
Anton Khatunzev
10/21/2021, 2:36 PMHello @Kishore Nallan i just got another weird problem with TS. I have a collection having field named "displaydata_title"(of type string) and when i try to include this field into sort_by statement i get this error:
"message":Ā "CouldĀ notĀ findĀ aĀ fieldĀ namedĀ
displaydata_titlek
Kishore Nallan
10/21/2021, 2:37 PMWe don't support sort by string.
a
Anton Khatunzev
10/21/2021, 2:37 PMoh...
k
Kishore Nallan
10/21/2021, 2:37 PMIt's on our roadmap to support.
a
Anton Khatunzev
10/21/2021, 2:37 PMok, thank you
k
Kishore Nallan
10/21/2021, 2:37 PMš
Kishore Nallan
10/25/2021, 8:50 AMCan you please give me a sample dataset to reproduce?