#community-help

Apple Silicon Binary & Compiling Typesense Issue

TLDR Bruno seeks to compile a native Apple Silicon binary of Typesense but encounters issues with RocksDB. Jason and Kishore Nallan assist, but the issue remains unresolved.

6mo

Join the chat

Jun 14, 2023 (6 months ago)

Photo of md5-d9ca032e3941589aafa8433269974f96

Bruno

09:43 PM

Hi everyone, quick q: is there a native Apple Silicon binary of Typesense available? Tried a couple of options including https://github.com/typesense/typesense/issues/868 and compiling locally (currently stuck at RocksDB), any pointers greatly appreciated...

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

09:44 PM

We don’t have publish native binary for M1, but you should be able to run the one we publish using Rosetta

Bruno

09:45 PM

Much appreciated, on a fairly low spec M2 Air, trying to avoid installing it...

Jason

09:45 PM

If you want to compile from source, I would recommend switching to v0.25-join branch and using bazel build, which is much easier

09:46

Jason

09:46 PM

https://typesense-community.slack.com/archives/C01P749MET0/p1686562368211039

Bruno

09:46 PM

Including native M2? Thatd be great

Jason

09:46 PM

We haven’t tried compiling on M2 yet… so not sure if you might run into issues

Bruno

09:47 PM

Only one way to find out...

09:47

Bruno

09:47 PM

Thanks Jason!

Jason

09:47 PM

But I was able to compile on M1, with one issue with getting OS metrics for the metrics.json endpoint, but everything else worked

09:47

Jason

09:47 PM

Let me know how it goes!

Bruno

09:48 PM

Off my Slack emoji game, so keeping it a simple: 👍

10:31

Bruno

10:31 PM

Unfortunately failing with RocksDB again, building it standalone with make checkruns fine.

10:33

Bruno

10:33 PM

-- Build files have been written to: /private/var/tmp/_bazel_bruno/4892beb866cdff20896715721c7b2cc5/sandbox/darwin-sandbox/519/execroot/__main__/bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb.build_tmpdir

+ /private/var/tmp/_bazel_bruno/4892beb866cdff20896715721c7b2cc5/sandbox/darwin-sandbox/519/execroot/__main__/external/cmake-3.25.0-macos-universal/bin/cmake --build . --config Release --target rocksdb -j8

[ 0%] Building CXX object CMakeFiles/rocksdb.dir/cache/cache_key.cc.o
[ 1%] Building CXX object CMakeFiles/rocksdb.dir/cache/cache_reservation_manager.cc.o
[ 1%] Building CXX object CMakeFiles/rocksdb.dir/cache/charged_cache.cc.o
make[3]: ccache: No such file or directory
make[3]: *** [CMakeFiles/rocksdb.dir/build.make:132: CMakeFiles/rocksdb.dir/cache/charged_cache.cc.o] Error 127
make[3]: *** Waiting for unfinished jobs....
make[3]: ccache: No such file or directory

make[3]: *** [CMakeFiles/rocksdb.dir/build.make:118: CMakeFiles/rocksdb.dir/cache/cache_reservation_manager.cc.o] Error 127

[ 1%] Building CXX object CMakeFiles/rocksdb.dir/cache/cache_entry_roles.cc.o
make[3]: ccache: No such file or directory
make[3]: *** [CMakeFiles/rocksdb.dir/build.make:104: CMakeFiles/rocksdb.dir/cache/cache_key.cc.o] Error 127
make[3]: ccache: No such file or directory
make[3]: *** [CMakeFiles/rocksdb.dir/build.make:90: CMakeFiles/rocksdb.dir/cache/cache_entry_roles.cc.o] Error 127
[ 2%] Building CXX object CMakeFiles/rocksdb.dir/cache/cache.cc.o
make[3]: ccache: No such file or directory
make[3]: *** [CMakeFiles/rocksdb.dir/build.make:76: CMakeFiles/rocksdb.dir/cache/cache.cc.o] Error 127
[ 2%] Building CXX object CMakeFiles/rocksdb.dir/cache/compressed_secondary_cache.cc.o
make[3]: ccache: No such file or directory

make[3]: *** [CMakeFiles/rocksdb.dir/build.make:160: CMakeFiles/rocksdb.dir/cache/compressed_secondary_cache.cc.o] Error 127

[ 2%] Building CXX object CMakeFiles/rocksdb.dir/cache/clock_cache.cc.o
make[3]: ccache: No such file or directory
make[3]: *** [CMakeFiles/rocksdb.dir/build.make:146: CMakeFiles/rocksdb.dir/cache/clock_cache.cc.o] Error 127
[ 3%] Building CXX object CMakeFiles/rocksdb.dir/cache/fast_lru_cache.cc.o
make[3]: ccache: No such file or directory
make[3]: *** [CMakeFiles/rocksdb.dir/build.make:174: CMakeFiles/rocksdb.dir/cache/fast_lru_cache.cc.o] Error 127
make[2]: *** [CMakeFiles/Makefile2:141: CMakeFiles/rocksdb.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:148: CMakeFiles/rocksdb.dir/rule] Error 2
make: *** [Makefile:169: rocksdb] Error 2
_____ END BUILD LOGS _____

rules_foreign_cc: Build wrapper script location: bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb_foreign_cc/wrapper_build_script.sh

rules_foreign_cc: Build script location: bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb_foreign_cc/build_script.sh

rules_foreign_cc: Build log location: bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb_foreign_cc/CMake.log

Target //:typesense-server failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 59.159s, Critical Path: 58.60s
INFO: 6 processes: 6 internal.
FAILED: Build did NOT complete successfully

10:33

Bruno

10:33 PM

I'll look into the arm build script later, if you see something stand out please let me know.

10:34

Bruno

10:34 PM

Very stupid question, from a guy who barely compiled a bunch of microcontroller firmware: is it really supposed to append the .o to .cc or replace it?

Jason

10:39 PM

Could you run xcode-select --install to install all the tools required for building?

Bruno

11:18 PM

And then set to a specific path manually?

Jason

11:19 PM

You shouldn’t have to set PATH manually… may be restart terminal once you’ve run xcode-select --install

Bruno

11:48 PM

Same issue, still mysteriously can't find the right rocksdb files in ccache.

Jason

11:49 PM

Kishore Nallan Any thoughts on this?

Bruno

11:56 PM

❤️

Jun 15, 2023 (6 months ago)

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

10:48 AM

That's a really odd error. Unfortunately without actually building on M2 it will be difficult to debug. We will try to do that before the next release and provide a native binary for M1/M2 Mac.

For now, you will have to use the ARM docker image.

Bruno

02:56 PM

Thanks Kishore Nallan! /private/.../CMakeFiles/rocksdb.dir/cache/] is indeed empty even after running bazel with --sandbox_debug

03:02

Bruno

03:02 PM

.cc.o still looks weird to me, but assume the build script is just missing a line or two as the dir is completely empty.

03:10

Bruno

03:10 PM

And really appreciate the responsiveness and Jason being in here - I'm bit of an edge case with my first post-Thinkpad Macbook and generally being an old grudge not wanting to deal with overhead like Rosetta or Docker, getting slightly odd vibes from Meili and appreciating the Vector search, that's why a cleanly compiled Apple Silicon binary would make a huge difference for me.

03:10

Bruno

03:10 PM

#custdev context

Jason

04:33 PM

Thank you for that context! Let me get back to you on this in a few days - we need to setup a fresh build environment to replicate the issue

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Trouble Running Typesense Docker Image on AMD Phenom II X4 955 Processor

Ian struggles to run Typesense on an AMD Phenom II X4 955 Processor, experiencing the docker image to shut down immediately upon startup. Jason and Kishore Nallan suggest multiple troubleshooting steps, identifying the potential issue as the processor's lack of support for MSSE4 flags.

14mo

ARM Support for Typesense on AWS Graviton Servers

Tatu inquires on ARM support for Typesense. Kishore Nallan informs it's on the roadmap with end of year completion. After some troubleshooting, they successfully test an ARM build, preparing it for the 0.23 release.

26mo

Discussing Search API Limitations and Solutions

Sidharth had problems with search API response limitations and sorting issues. Kishore Nallan suggested multi_search query and provided links for an updated version. After installation, some timeout and performance issues were encountered, partially resolved by adjusting client timeout values.

14mo

Troubleshooting Typesense Binary Segmentation Fault on a Mac

Daniel sought help for a segmentation fault while using Typesense on a MacBook Air. Kishore Nallan guided them through the debugging process and suggested that it might be a compatibility issue with the Mac OS version. Daniel plans to try building Typesense on their own.

32mo

Slow, High CPU Write Operations After Collection Drop in Typesense

Himank discussed an issue in Typesense where deleting and recreating a collection led to slow write operations and high CPU usage. Kishore Nallan suggested using an alias to avoid this issue. Numerous tests and debugging was conducted as pboros contributed with local testing. Kishore Nallan aimed to start implementing a range delete and full db compaction after deletion to potentially solve the issue.

232

17mo