#community-help

Typesense Connection Issue Resolution

TLDR James faced a 'connection refused' error trying to run typesense locally. Kishore Nallan identified an issue due to a glibc upgrade, provided a recent build for James to try and advised a restart after installation. The problem was subsequently resolved.

Powered by Struct AI
35
22mo
Solved
Join the chat
Jan 07, 2022 (22 months ago)
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:30 AM
Hey guys, i'm trying to run typesense locally, the systemd process is active, but i'm getting a connection refused response when i try to curl http://localhost:8101/health, any idea why?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:31 AM
Default typesense port is 8108, can you please check that?
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:31 AM
netstat -tulpn doesn't show that port active
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:31 AM
Any errors in the Typesense logs? Also, what OS and version are you on?
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:32 AM
on ubuntu 21.04, and nothing in /var/log/typesense
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:33 AM
Ah, I recently came across an issue on Ubuntu 21.04 because of the glibc upgrade in that version, once second.
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:33 AM
Sorry, forget i upgraded to 21.10 a few days ago
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:33 AM
I will have to get you a recent 0.23 RC build where I have fixed this error.
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:34 AM
that would be great, thanks
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:34 AM
Are you using DEB or tar.gz binary?
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:35 AM
i thought there might be some issue with a previous install of typesense
08:35
James
08:35 AM
the .deb
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:35 AM
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:35 AM
ok, one sec
08:36
James
08:36 AM
hmm unfortunately didn't work
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:37 AM
What happens?
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:37 AM
i mean nothing that i can tell, systemd shows it as active, no errors, but /var/log/typesense is also empty which i'm not sure it should be
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:38 AM
Did you kill the previous server before deb install?
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:38 AM
i dpkg -P'd it, which i think does that automatically
08:38
James
08:38 AM
ill restart it just in case
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:38 AM
Here is the other issue where another user faced the same problem, which seemed to have been fixed on the 0.23.rc11 build: https://github.com/typesense/typesense/issues/478#issuecomment-1002652599
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:39 AM
ah ok restarting it worked!
08:39
James
08:39 AM
thanks Kishore
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:39 AM
๐Ÿ˜…
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:39 AM
was pulling my hair out figuring out why it wasn't working lol
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:39 AM
Thanks for confirming!
08:40
Kishore Nallan
08:40 AM
Yeah it took a long time for me to figure this out as well. I only managed to narrow it down by running it under valgrind and googling on the cryptic error, working backwards upto a glibc library change.
08:41
Kishore Nallan
08:41 AM
Since the change landed only in late September it is pretty fresh and looks like both latest Ubuntu and Fedora have adopted that immediately.
08:41
Kishore Nallan
08:41 AM
Sorry for the hassle!
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:42 AM
No worries, and thanks for the quick fix. You have a link to the diff? i'm curious to see what exactly caused the problem
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:45 AM
It was an issue in one of the dependencies we use (brpc). Glibc v2.34 has stopped exposing the _dl_sym function publicly, which this library was depending on to avoid a deadlock during some initialization code. Here's the change: https://github.com/typesense/incubator-brpc/commit/a48506a635072ae2abf370798a47038fbcd230ff

It was some metrics/analyzer related code for which I could not find a proper work around, so I had to just delete it for now :face_with_hand_over_mouth:
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:48 AM
ah ok, nothing too serious then, nice work
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:48 AM
๐Ÿ™‚ :ty:
James
Photo of md5-7d2796ababc14477f4c09abc2e8edb05
James
08:49 AM
alright, i've got to head off for a bit, thanks again for the quick support Kishore, take it easy
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:49 AM
๐Ÿ‘‹ see you