https://typesense.org logo
Join Slack
Powered by
Hello! I wanted to contribute to the typesense rep...
# contributions
d
Hello! I wanted to contribute to the typesense repo, but I'm having issues with setting up my dev environment. I've been using the latest Ubuntu instance within VirtualBox and following the development.Dockerfile and getting stuck while running "make -j8" on the incubator-brpc. Any help is appreciated!
For more info I'm using the latest version of Ubuntu. I installed gcc-10.1.0 compiler from source and set the PATH variable, but couldn't setup the LD_LIBRARY_PATH. When running make -j8 I face include issues:
Copy code
In file included from /usr/include/features.h:502,
                 from /usr/include/x86_64-linux-gnu/c++/13/bits/os_defines.h:39,
                 from /usr/include/x86_64-linux-gnu/c++/13/bits/c++config.h:679,
                 from /usr/include/c++/13/new:40,
                 from /opt/incubator-brpc-a48506a635072ae2abf370798a47038fbcd230ff/src/butil/lazy_instance.h:38,
                 from /opt/incubator-brpc-a48506a635072ae2abf370798a47038fbcd230ff/src/butil/third_party/dmg_fp/dtoa_wrapper.cc:7:
/usr/include/x86_64-linux-gnu/sys/cdefs.h:331:60: error: macro "__has_attribute" requires an identifier
  331 | #if __GNUC_PREREQ (2,5) || __glibc_has_attribute (__const__)
      |                                                            ^
[  2%] Building CXX object CMakeFiles/PROTO_LIB.dir/idl_options.pb.cc.o
In file included from /usr/include/features.h:502,
                 from /usr/include/x86_64-linux-gnu/bits/libc-header-start.h:33,
                 from /usr/include/stdint.h:26,
                 from /usr/local/gcc-10.1.0/lib/gcc/x86_64-linux-gnu/10.1.0/include/stdint.h:9,
                 from /opt/incubator-brpc-a48506a635072ae2abf370798a47038fbcd230ff/src/butil/third_party/superfasthash/superfasthash.c:28:
/usr/include/x86_64-linux-gnu/sys/cdefs.h:331:60: error: macro "__has_attribute" requires an identifier
  331 | #if __GNUC_PREREQ (2,5) || __glibc_has_attribute (__const__)

...

In file included from /usr/include/c++/13/bits/ios_base.h:46,
                 from /usr/include/c++/13/ios:44,
                 from /usr/include/c++/13/istream:40,
                 from /usr/include/c++/13/sstream:40,
                 from /opt/incubator-brpc-a48506a635072ae2abf370798a47038fbcd230ff/src/butil/logging.h:30,
                 from /opt/incubator-brpc-a48506a635072ae2abf370798a47038fbcd230ff/src/butil/lazy_instance.h:44:
/usr/include/c++/13/system_error:183:38: error: expected an identifier for the attribute name
  183 |   [[__nodiscard__, __gnu__::__const__]]
      |                                      ^
/usr/include/c++/13/system_error:188:38: error: expected an identifier for the attribute name
  188 |   [[__nodiscard__, __gnu__::__const__]]

...
k
We don't use docker for builds anymore. We use Bazel so the following will allow you to build the project: bazel build @com_google_protobuf//:protobuf_headers bazel build @com_google_protobuf//:protobuf_lite bazel build @com_google_protobuf//:protobuf bazel build @com_google_protobuf//:protoc bazel build //:typesense-server
l
hi, I'm on a Mac (m3 pro) and when running 
bazel build //:typesense-server
(after all the other commands), I get:
Copy code
ERROR: /private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/external/com_github_brpc_brpc/BUILD.bazel:231:13: in objc_library rule @com_github_brpc_brpc//:macos_lib: Expected action_config for 'objc++-compile' to be configured
ERROR: /private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/external/com_github_brpc_brpc/BUILD.bazel:231:13: in objc_library rule @com_github_brpc_brpc//:macos_lib:
Traceback (most recent call last):
	File "/virtual_builtins_bzl/common/objc/objc_library.bzl", line 55, column 82, in _objc_library_impl
	File "/virtual_builtins_bzl/common/objc/compilation_support.bzl", line 312, column 50, in _register_compile_and_archive_actions
	File "/virtual_builtins_bzl/common/objc/compilation_support.bzl", line 416, column 74, in _cc_compile_and_link
	File "/virtual_builtins_bzl/common/objc/compilation_support.bzl", line 182, column 29, in _compile
Error in compile: Expected action_config for 'objc++-compile' to be configured
ERROR: /private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/external/com_github_brpc_brpc/BUILD.bazel:231:13: Analysis of target '@com_github_brpc_brpc//:macos_lib' failed
ERROR: Analysis of target '//:typesense-server' failed; build aborted:
ok, I had to install XCode, now that part passes, but I have a new error:
Copy code
_____ BEGIN BUILD LOGS _____
xcrun: error: SDK "macosx14.5" cannot be located
xcrun: error: SDK "macosx14.5" cannot be located
xcrun: error: unable to lookup item 'Path' in SDK 'macosx14.5'
+ XCODE_VERSION_OVERRIDE=15.4.0.15F31d
+ APPLE_SDK_VERSION_OVERRIDE=14.5
+ APPLE_SDK_PLATFORM=MacOSX
+ ZERO_AR_DATE=1
+ AR=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/346/execroot/__main__/external/local_config_cc/libtool
+ ARFLAGS=-o
+ CC=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/346/execroot/__main__/external/local_config_cc/wrapped_clang
+ CFLAGS='-D_FORTIFY_SOURCE=1 -fstack-protector -fcolor-diagnostics -Wall -Wthread-safety -Wself-assign -fno-omit-frame-pointer -g0 -O2 -DNDEBUG -DNS_BLOCK_ASSERTIONS=1 DEBUG_PREFIX_MAP_PWD=. -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -no-canonical-prefixes -pthread -no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__=redacted -D__TIMESTAMP__=redacted -D__TIME__=redacted -target arm64-apple-macosx14.5'
+ LD=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/346/execroot/__main__/external/local_config_cc/cc_wrapper.sh
+ LDFLAGS='-lc++ -fobjc-link-runtime -headerpad_max_install_names -no-canonical-prefixes -target arm64-apple-macosx14.5 -lc++ -target arm64-apple-macosx14.5 -undefined error'
+ ./configure --without-guile --with-guile=no --disable-dependency-tracking --prefix=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/346/execroot/__main__/bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/rules_foreign_cc/toolchains/make
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... build-aux/install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports the include directive... yes (GNU style)
checking for gcc... /private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/346/execroot/__main__/external/local_config_cc/wrapped_clang
checking whether the C compiler works... no
configure: error: in `/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/346/execroot/__main__/bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/rules_foreign_cc/toolchains/make.build_tmpdir':
configure: error: C compiler cannot create executables
See `config.log' for more details
_____ END BUILD LOGS _____
uhm, this time it was the xcode cli, next error:
Copy code
_____ BEGIN BUILD LOGS _____

Bazel external C/C++ Rules. Building library rocksdb

Environment:______________
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
BAZEL_CXXOPTS=-std=c++17
CMAKE_OSX_ARCHITECTURES=arm64
TMPDIR=/var/folders/84/r9g1zqz53pj7cg1z90cqq75h0000gn/T/
SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk
BUILD_WRAPPER_SCRIPT=bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb_foreign_cc/wrapper_build_script.sh
EXT_BUILD_ROOT=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__
XCODE_VERSION_OVERRIDE=15.4.0.15F31d
INSTALLDIR=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb
__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
ZERO_AR_DATE=1
PATH=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__:/bin:/usr/bin:/usr/local/bin
BUILD_TMPDIR=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb.build_tmpdir
PWD=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__
EXT_BUILD_DEPS=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb.ext_build_deps
SHLVL=3
BUILD_LOG=bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb_foreign_cc/CMake.log
BUILD_SCRIPT=bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb_foreign_cc/build_script.sh
APPLE_SDK_PLATFORM=MacOSX
APPLE_SDK_VERSION_OVERRIDE=14.5
_=/usr/bin/env
__________________________
+ /private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/external/cmake-3.25.0-macos-universal/bin/cmake -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb.build_tmpdir/crosstool_bazel.cmake -DCMAKE_INSTALL_PREFIX=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb '-DCMAKE_PREFIX_PATH=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb.ext_build_deps;external/com_github_google_snappy;bazel-out/darwin_arm64-fastbuild/bin/external/com_github_google_snappy' -DCMAKE_RANLIB= -DWITH_GFLAGS=OFF -DWITH_ALL_TESTS=OFF -DPORTABLE=1 -DWITH_SNAPPY=1 -DROCKSDB_BUILD_SHARED=OFF -DWITH_TESTS=OFF -DWITH_TOOLS=OFF -DUSE_RTTI=1 -DCMAKE_CXX_FLAGS=-Wno-error=uninitialized -DCMAKE_MAKE_PROGRAM=/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/rules_foreign_cc/toolchains/make/bin/make -G 'Unix Makefiles' /private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/external/rocksdb
-- The CXX compiler identification is AppleClang 15.0.0.15000309
-- The C compiler identification is AppleClang 15.0.0.15000309
-- The ASM compiler identification is Clang with GNU-like command-line
-- Found assembler: /private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/external/local_config_cc/wrapped_clang
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/external/local_config_cc/wrapped_clang_pp - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/external/local_config_cc/wrapped_clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Could NOT find Snappy (missing: Snappy_DIR)
-- Found Snappy: /private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb.ext_build_deps/lib/libsnappy.a
-- Performing Test HAVE_OMIT_LEAF_FRAME_POINTER
-- Performing Test HAVE_OMIT_LEAF_FRAME_POINTER - Success
-- Performing Test HAS_ARMV8_CRC
-- Performing Test HAS_ARMV8_CRC - Success
--  HAS_ARMV8_CRC yes
-- Performing Test BUILTIN_ATOMIC
-- Performing Test BUILTIN_ATOMIC - Success
-- Could NOT find uring (missing: uring_LIBRARIES uring_INCLUDE_DIR)
-- Enabling RTTI in all builds
-- Performing Test HAVE_FALLOCATE
-- Performing Test HAVE_FALLOCATE - Failed
-- Performing Test HAVE_SYNC_FILE_RANGE_WRITE
-- Performing Test HAVE_SYNC_FILE_RANGE_WRITE - Failed
-- Performing Test HAVE_PTHREAD_MUTEX_ADAPTIVE_NP
-- Performing Test HAVE_PTHREAD_MUTEX_ADAPTIVE_NP - Failed
-- Looking for malloc_usable_size
-- Looking for malloc_usable_size - not found
-- Looking for sched_getcpu
-- Looking for sched_getcpu - not found
-- Looking for getauxval
-- Looking for getauxval - not found
-- Looking for F_FULLFSYNC
-- Looking for F_FULLFSYNC - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- ROCKSDB_PLUGINS:
-- ROCKSDB PLUGINS TO BUILD
-- Found Git: /usr/bin/git (found version "2.39.3 (Apple Git-146)")
-- JNI library is disabled
-- Configuring done
-- Generating done
-- Build files have been written to: /private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/bazel-out/darwin_arm64-fastbuild/bin/external/rocksdb/rocksdb.build_tmpdir
+ /private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/external/cmake-3.25.0-macos-universal/bin/cmake --build . --config Release --target rocksdb -j8
[  0%] Building CXX object CMakeFiles/rocksdb.dir/cache/cache_entry_roles.cc.o
[  1%] Building CXX object CMakeFiles/rocksdb.dir/cache/cache.cc.o
[  2%] Building CXX object CMakeFiles/rocksdb.dir/cache/cache_reservation_manager.cc.o
[  2%] Building CXX object CMakeFiles/rocksdb.dir/cache/charged_cache.cc.o
[  3%] Building CXX object CMakeFiles/rocksdb.dir/cache/cache_key.cc.o
[  3%] Building CXX object CMakeFiles/rocksdb.dir/cache/fast_lru_cache.cc.o
[  3%] Building CXX object CMakeFiles/rocksdb.dir/cache/clock_cache.cc.o
[  3%] Building CXX object CMakeFiles/rocksdb.dir/cache/compressed_secondary_cache.cc.o
[  3%] Building CXX object CMakeFiles/rocksdb.dir/cache/lru_cache.cc.o
[  3%] Building CXX object CMakeFiles/rocksdb.dir/cache/sharded_cache.cc.o
/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/external/rocksdb/cache/clock_cache.cc:855:15: error: variable 'i' set but not used [-Werror,-Wunused-but-set-variable]
  for (size_t i = 0; &array_[current] != h; i++) {
              ^
[  4%] Building CXX object CMakeFiles/rocksdb.dir/db/arena_wrapped_db_iter.cc.o
[  4%] Building CXX object CMakeFiles/rocksdb.dir/db/blob/blob_contents.cc.o
[  4%] Building CXX object CMakeFiles/rocksdb.dir/db/blob/blob_fetcher.cc.o
/private/var/tmp/_bazel_lussoluca/29cef7eb4572e7796291d661bc53e7b9/sandbox/darwin-sandbox/513/execroot/__main__/external/rocksdb/cache/lru_cache.cc:98:12: error: variable 'count' set but not used [-Werror,-Wunused-but-set-variable]
  uint32_t count = 0;
           ^
1 error generated.
Error in child process '/usr/bin/xcrun'. 1
make[3]: *** [CMakeFiles/rocksdb.dir/build.make:146: CMakeFiles/rocksdb.dir/cache/clock_cache.cc.o] Error 1
make[3]: *** Waiting for unfinished jobs....
1 error generated.
Error in child process '/usr/bin/xcrun'. 1
make[3]: *** [CMakeFiles/rocksdb.dir/build.make:188: CMakeFiles/rocksdb.dir/cache/lru_cache.cc.o] Error 1
make[2]: *** [CMakeFiles/Makefile2:141: CMakeFiles/rocksdb.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:148: CMakeFiles/rocksdb.dir/rule] Error 2
make: *** [Makefile:169: rocksdb] Error 2
_____ END BUILD LOGS _____
k
Hmm this build just directly works for me when we run in a VM. Looks like there is some strict warning mode enabled that fails the compilation.
l
do you mean in a MacOs VM?
k
Yes
l
ok, so maybe my environment have some issue
k
Which version of Bazel are you using?
l
bazel 6.2.0
k
We use the same version.
The problem is this:
Copy code
-Wunused-but-set-variable
Not sure why it's getting added on your environment. Maybe a newer clang is used and it enables it by default.
l
my clang version is
Copy code
Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
k
I don't have access to the VM at the moment to check clang.
l
no problem, I'll keep trying. What I'm trying to do is to extend the conversation features to be able to send more parameters to openai
I can open an issue, but I like to provide a MR with it
k
I think you have to add an explicit 
-Wno-unused-but-set-variable
flag here along with existing flag: https://github.com/typesense/typesense/blob/main/bazel/rocksdb.BUILD#L40
l
do you know how to pass multiple flags?
k
Actually, a small tweak:
Copy code
"@platforms//os:macos": ["-DCMAKE_CXX_FLAGS=-Wno-error=uninitialized -Wno-error=unused-but-set-variable"],
The above should help the build pass
l
thanks
I still have the same error...
k
Sorry I have to debug on a mac vm on a new clang which is a lot of work to setup. I recommend using a Linux VM or Docker to build for now.
l
ok, I'll try with a Linux VM
thanks
d
Hi Kishore! Is there a good way to run one specific test file with bazel test?
Scratch that, I was able to use --test_filter to do specific tests
Though, I'm not entirely sure if I setup it correctly
k
Yes that looks correct
Open in Slack
PreviousNext
Powered by