#community-help

Precision Issues with Int64 Numeric Field in Go Client

TLDR SamHendley expressed concerns about precision and rounding problems with an int64 numeric field using a timestamp. The problem was traced back to a conversion issue in the go client, prompting SamHendley to suggest updating the openapi spec to use double for reduced errors.

11mo

Join the chat

Nov 03, 2022 (11 months ago)

Photo of md5-a9a351e11d64f05b41fec183816a0cda

SamHendley

01:28 AM

What is the precision and rounding on min/max stats for a int64 numeric field? I am using it with timestamp (unix seconds) and the values appear to be getting rounded. The lowest field in my collection is 1662633479 but the reported min in facet response is 1662633500 which is a little bit higher. Is there any chance this is being converted to float during the stats phase?

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

01:37 AM

We do use a double for facet stats computation so it's possible that has an impact on the precision. Do you have a small reproduceable example that we can use for testing?

SamHendley

01:11 PM

I couldn’t reproduce in curl. Turns out the conversion is happening in the go client when it parses the result. In the openapi spec the min/max/avg fields are declared to be float which is then declared a float32 in the generated code. It looks like that could be at least a float64 for golang if the spec format was ‘double’ instead of ‘float’.

Kishore Nallan

01:11 PM

Ah yes, that must be it.

SamHendley

01:16 PM

I’d recommend updating the openapi spec to use double since the underlying data is often 64 bits. Fortunatley this is the only place the float term is used in the openapi spec so the impact to generated clients/consumers would be minor but it would probably need a ‘major’ version bump for semver sake

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2779 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Resolving Floating Point Arithmetic in Facets

Claudiu reported an issue with floating point value representation in facets and found a workaround by converting everything to int32. Kishore Nallan validated this fix and also committed to finding a solution. Later, Erick faced a related error.

21mo

Setting dynamic range faceting in Typesense

Jacob requested dynamic range faceting in Typesense, and explanation for fetching max/min values. Kishore Nallan clarified pricing bucket sizes were user dependent and explained the stats return as part of facet results.

1mo

Improving System Performance and Typesense Query Efficiency

SamHendley was experiencing performance issues with Typesense's large-scale system testing and proposed several improvements. Both Jason and Kishore Nallan addressed the suggestions and corrected some misconceptions. They provided further clarification and recommended upgrades for better performance.

9mo