#community-help

Resolving Float Value Issues in PHP and Typesense.

TLDR Zildjian was having issues with float values being treated as int64 in Typesense. Kishore Nallan identified and rectified the issue, guiding the user through a software upgrade process.

Powered by Struct AI

1

1

29
2mo
Solved
Join the chat
Jul 20, 2023 (2 months ago)
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
05:27 AM
Hello, I am trying to set a float value to a field, however this is being saved as a int64

the field is an object and each field inside it should be a float (see screenshot). however when I do an import and encountered the scenario where values are a whole number but a float type in PHP, it automatically sets the field type as int64 on ts.

I would appreciate any answers/help. Thank you :bow:

with decimals
value: 123.2564
value when cast to float: (float) 123.2564 = 123.2564
type on php: float
value when passed to ts: 123.2564
type when passed to ts: float
ts treats this as: float

without decimals/whole number
value: 123
value when cast to float: (float) 123 = 123 (same result even if using round or number_format)
type on php: float
value when passed to ts: 123
type when passed to ts: float
ts treats this as: int64
Image 1 for Hello, I am trying to set a float value to a field, however this is being saved as a int64

the field is an object and each field inside it should be a float (see screenshot). however when I do an import and encountered the scenario where values are a whole number but a float type in PHP, it automatically sets the field type as int64 on ts.

I would appreciate any answers/help. Thank you :bow:

*with decimals*
value: 123.2564
value when cast to float: (float) 123.2564 = 123.2564
type on php: float
value when passed to ts: 123.2564
type when passed to ts: float
ts treats this as: float

*without decimals/whole number*
value: 123
value when cast to float: (float) 123 = 123 (same result even if using round or number_format)
type on php: float
value when passed to ts: 123
type when passed to ts: float
ts treats this as: int64Image 2 for Hello, I am trying to set a float value to a field, however this is being saved as a int64

the field is an object and each field inside it should be a float (see screenshot). however when I do an import and encountered the scenario where values are a whole number but a float type in PHP, it automatically sets the field type as int64 on ts.

I would appreciate any answers/help. Thank you :bow:

*with decimals*
value: 123.2564
value when cast to float: (float) 123.2564 = 123.2564
type on php: float
value when passed to ts: 123.2564
type when passed to ts: float
ts treats this as: float

*without decimals/whole number*
value: 123
value when cast to float: (float) 123 = 123 (same result even if using round or number_format)
type on php: float
value when passed to ts: 123
type when passed to ts: float
ts treats this as: int64
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
05:29 AM
👋 I remember fixing a similar issue on the 0.25 RC builds. Would you be able to quickly try locally against a docker build of 0.25 RC?
05:30
Kishore Nallan
05:30 AM
For e.g. on typesense/typesense:0.25.0.rc48 build.
05:33
Kishore Nallan
05:33 AM
Also the schema looks a bit weird (first screenshot). fields should be part of the collection schema but in the PHP code it seems to be part of regularPrice object field.
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
01:38 PM
Thank you for your response I appreciate it. I am trying to have a nested object for regularPrice, and inside the object should be any property (eg. AUD, NZD, USD etc.) that has a float value

so when returned on the document it should have something like this
{
  // other properties
  "regularPrice": {
    "AUD": 20.20,
    "NZD": 20.00,
    "USD": 12.00,
  }
}
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:43 PM
You can just define a regularPrice field of type object. The types of inner child fields will be auto detected.
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
01:45 PM
yes I tried that however, if document 1 sets regularPrice.AUD to int64 (because of typesense treating a whole number as int64 even if the type of php is float)

then all other document's regularPrice.AUD should be int64. now the problem I have is that some documents are floats and this will error out when importing those documents
01:46
Zildjian
01:46 PM
that is why I explicitly define it to have floats and hoping that would force it
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:53 PM
Ok can you try with field name as regularPrice.*
01:53
Kishore Nallan
01:53 PM
And type as float
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
02:03 PM
ok I'll try
02:03
Zildjian
02:03 PM
thanks
Jul 21, 2023 (2 months ago)
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
01:20 AM
hello again, so I tried this and it now affects other objects
Image 1 for hello again, so I tried this and it now affects other objectsImage 2 for hello again, so I tried this and it now affects other objects
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:51 AM
This is probably fixed in recent 0.25 RC builds (https://github.com/typesense/typesense/issues/973).
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
02:50 AM
this was fixed on 0.25.0.rc24, and our version is rc27
Image 1 for this was fixed on 0.25.0.rc24, and our version is rc27
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:01 AM
Can you please post a smaller code snippet that directly reproduces the issue so I can try it out and find out where it's exactly going wrong? The gist should contain:

1. Creating a collection with a schema with 1 or 2 fileds
2. A sample document ingested that causes the error
With the above it will be easy for me to reproduce what's happening. You can use curl as well, see an example here: https://gist.github.com/kishorenc/865511899735ab23d82b29d8592751c5
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
06:17 AM
hello, here's the command I tried. the results are on the screenshot. thanks
export TYPESENSE_API_KEY=REDACTED
export TYPESENSE_HOST=REDACTED
export TYPESENSE_COLLECTION_NAME=test_products

curl -k "${TYPESENSE_HOST}/collections" -X POST -H "Content-Type: application/json" \
      -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{
        "name": "test_products",      
        "fields": [              
          {"name": "regularPrice.*", "type": "float"}, 
          {"name": "title", "type": "string"}
        ],
        "enable_nested_fields": true       
      }'

curl "${TYPESENSE_HOST}/collections/${TYPESENSE_COLLECTION_NAME}/documents" -X POST \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{"title": "Product 1", "regularPrice": { "AUD": 1, "NZD": 2.2 }}'

curl "${TYPESENSE_HOST}/collections/${TYPESENSE_COLLECTION_NAME}/documents" -X POST \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{"title": "Product 1", "regularPrice": { "AUD": 1.0, "NZD": 2.2 }}'
Image 1 for hello, here's the command I tried. the results are on the screenshot. thanks
```export TYPESENSE_API_KEY=REDACTED
export TYPESENSE_HOST=REDACTED
export TYPESENSE_COLLECTION_NAME=test_products

curl -k "${TYPESENSE_HOST}/collections" -X POST -H "Content-Type: application/json" \
      -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{
        "name": "test_products",      
        "fields": [              
          {"name": "regularPrice.*", "type": "float"}, 
          {"name": "title", "type": "string"}
        ],
        "enable_nested_fields": true       
      }'

curl "${TYPESENSE_HOST}/collections/${TYPESENSE_COLLECTION_NAME}/documents" -X POST \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{"title": "Product 1", "regularPrice": { "AUD": 1, "NZD": 2.2 }}'

curl "${TYPESENSE_HOST}/collections/${TYPESENSE_COLLECTION_NAME}/documents" -X POST \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{"title": "Product 1", "regularPrice": { "AUD": 1.0, "NZD": 2.2 }}'```
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
06:18 AM
:ty: will get back to you
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
06:19 AM
thanks for looking into this. I really appreciate it
Jul 23, 2023 (2 months ago)
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
11:09 PM
Hello, I was wondering if you have replicated the issue?
Jul 24, 2023 (2 months ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:28 AM
Yes I was able to reproduce. Will be looking to fix it in the next few days.

1

Jul 25, 2023 (2 months ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
02:49 AM
I've a fix for this. Do you use Typesense Cloud?
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
02:50 AM
yes
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
02:50 AM
Please ping me the cluster ID and I will schedule an upgrade or you can also verify the fix locally first?
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
02:57 AM
can we do the upgrade ourselves via the cluster configuration?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
02:57 AM
Yes look for 0.25.0.rc54 version
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
02:58 AM
Thanks a lot, I really appreciate your help. I'll let you know how it go

1

04:00
Zildjian
04:00 AM
it worked! you're a legend! thanks.
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
04:05 AM
Glad to hear!