Manav Kothari
06/11/2025, 8:19 AMhas_email and has_phone boolean fields with index:true and sort:true. Searching now takes 15 seconds with sorting.
How can I optimize this?
I need to recommend records which has email and phone both available.
Here is the search query:
{
highlight_fields: 'none',
include_fields: 'id,organization_id,linkedin_url,email,email_status,mobile_number,title,seniority,job_start_date,first_name,last_name,name,city,state,country,country_code,country_region,organizations.id,organizations.website_url,organizations.linkedin_url,organizations.founded_year,organizations.primary_domain,organizations.name,organizations.phone',
sort_by: 'has_email : desc, has_phone : desc',
page: 1,
per_page: 25,
q: '*',
use_cache: false
}Fanis Tharropoulos
06/11/2025, 10:35 AM_eval([(has_email:true):3, (has_phone):2])):descFanis Tharropoulos
06/11/2025, 10:35 AMsort: true on either fieldManav Kothari
06/11/2025, 11:28 AM_eval([(has_email:true):3, (has_phone:true):2])):desc it but it doesn't give similar results as sort_by: 'has_email : desc, has_phone : desc'Fanis Tharropoulos
06/11/2025, 11:31 AM_eval([(has_email:true && has_phone:true):3, (has_phone):2, (has_email:true):2 ])):desc is probably the best. It prioritizes documents that have both an email and a phone number and then still ranks documents that have at least one of the two higher than the restFanis Tharropoulos
06/11/2025, 11:32 AMManav Kothari
06/11/2025, 11:45 AMFanis Tharropoulos
06/11/2025, 12:18 PMManav Kothari
06/11/2025, 12:19 PMFanis Tharropoulos
06/11/2025, 12:21 PMq=*?Manav Kothari
06/11/2025, 12:22 PM{
highlight_fields: 'none',
include_fields: 'id,organization_id,linkedin_url,email,email_status,mobile_number,title,seniority,job_start_date,first_name,last_name,name,city,state,country,country_code,country_region,organizations.id,organizations.website_url,organizations.linkedin_url,organizations.founded_year,organizations.primary_domain,organizations.name,organizations.phone',
sort_by: '_eval([(has_email:true && has_phone:true):3, (has_phone):2, (has_email:true):2 ])):desc',
page: 1,
per_page: 25,
q: '*',
use_cache: false
}Fanis Tharropoulos
06/11/2025, 12:23 PMuse_cache set to false? Also, what's your CPU and RAM usage like?Fanis Tharropoulos
06/11/2025, 12:24 PMsort parameter to false for the two boolean fields?Manav Kothari
06/11/2025, 12:27 PMsort:true and index: true , there is no major production traffic. i use r7g.8xlarge instanceFanis Tharropoulos
06/11/2025, 12:33 PMManav Kothari
06/11/2025, 12:34 PMFanis Tharropoulos
06/11/2025, 12:34 PMv29.0rc29Manav Kothari
06/11/2025, 12:38 PMsort: false for both the indexes?Fanis Tharropoulos
06/11/2025, 12:39 PMManav Kothari
06/11/2025, 12:40 PMFanis Tharropoulos
06/11/2025, 12:43 PM/stats.json and the /metrics.json endpoints? https://typesense.org/docs/0.20.0/api/cluster-operations.html#cluster-metricsManav Kothari
06/11/2025, 12:45 PM{
"system_cpu10_active_percentage": "0.00",
"system_cpu11_active_percentage": "0.00",
"system_cpu12_active_percentage": "0.00",
"system_cpu13_active_percentage": "0.00",
"system_cpu14_active_percentage": "0.00",
"system_cpu15_active_percentage": "0.00",
"system_cpu16_active_percentage": "0.00",
"system_cpu17_active_percentage": "0.00",
"system_cpu18_active_percentage": "0.00",
"system_cpu19_active_percentage": "0.00",
"system_cpu1_active_percentage": "100.00",
"system_cpu20_active_percentage": "0.00",
"system_cpu21_active_percentage": "0.00",
"system_cpu22_active_percentage": "0.00",
"system_cpu23_active_percentage": "0.00",
"system_cpu24_active_percentage": "0.00",
"system_cpu25_active_percentage": "0.00",
"system_cpu26_active_percentage": "0.00",
"system_cpu27_active_percentage": "0.00",
"system_cpu28_active_percentage": "0.00",
"system_cpu29_active_percentage": "0.00",
"system_cpu2_active_percentage": "0.00",
"system_cpu30_active_percentage": "0.00",
"system_cpu31_active_percentage": "0.00",
"system_cpu32_active_percentage": "0.00",
"system_cpu3_active_percentage": "0.00",
"system_cpu4_active_percentage": "0.00",
"system_cpu5_active_percentage": "0.00",
"system_cpu6_active_percentage": "0.00",
"system_cpu7_active_percentage": "0.00",
"system_cpu8_active_percentage": "0.00",
"system_cpu9_active_percentage": "0.00",
"system_cpu_active_percentage": "3.12",
"system_disk_total_bytes": "527295578112",
"system_disk_used_bytes": "200185749504",
"system_memory_total_bytes": "265475792896",
"system_memory_total_swap_bytes": "0",
"system_memory_used_bytes": "129162891264",
"system_memory_used_swap_bytes": "0",
"system_network_received_bytes": "456099623132",
"system_network_sent_bytes": "52271553736",
"typesense_memory_active_bytes": "102067798016",
"typesense_memory_allocated_bytes": "93931910896",
"typesense_memory_fragmentation_ratio": "0.08",
"typesense_memory_mapped_bytes": "124438773760",
"typesense_memory_metadata_bytes": "19556841344",
"typesense_memory_resident_bytes": "102067798016",
"typesense_memory_retained_bytes": "2121363554304"
}Manav Kothari
06/11/2025, 12:46 PM{
"delete_latency_ms": 0,
"delete_requests_per_second": 0,
"import_latency_ms": 0,
"import_requests_per_second": 0,
"latency_ms": {
"GET /health": 0,
"GET /metrics.json": 103,
"GET /stats.json": 0
},
"overloaded_requests_per_second": 0,
"pending_write_batches": 1,
"requests_per_second": {
"GET /health": 0.1,
"GET /metrics.json": 0.5,
"GET /stats.json": 0.5,
"POST /multi_search": 0.2
},
"search_latency_ms": 0,
"search_requests_per_second": 0,
"total_requests_per_second": 1.3,
"write_latency_ms": 0,
"write_requests_per_second": 0
}Fanis Tharropoulos
06/11/2025, 1:16 PM* for searching, we have to run that eval on every document on that collection, which could be tens of millions, or even hundreds of millions. There isn't a better way to approach this currently, sadly.Manav Kothari
06/11/2025, 2:31 PMFanis Tharropoulos
06/11/2025, 3:19 PM