#community-help

Resolving Float Value Issues in PHP and Typesense.

TLDR Zildjian was having issues with float values being treated as int64 in Typesense. Kishore Nallan identified and rectified the issue, guiding the user through a software upgrade process.

Powered by Struct AI

1

1

Jul 20, 2023 (4 months ago)
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
05:27 AM
Hello, I am trying to set a float value to a field, however this is being saved as a int64

the field is an object and each field inside it should be a float (see screenshot). however when I do an import and encountered the scenario where values are a whole number but a float type in PHP, it automatically sets the field type as int64 on ts.

I would appreciate any answers/help. Thank you :bow:

with decimals
value: 123.2564
value when cast to float: (float) 123.2564 = 123.2564
type on php: float
value when passed to ts: 123.2564
type when passed to ts: float
ts treats this as: float

without decimals/whole number
value: 123
value when cast to float: (float) 123 = 123 (same result even if using round or number_format)
type on php: float
value when passed to ts: 123
type when passed to ts: float
ts treats this as: int64
Image 1 for Hello, I am trying to set a float value to a field, however this is being saved as a int64

the field is an object and each field inside it should be a float (see screenshot). however when I do an import and encountered the scenario where values are a whole number but a float type in PHP, it automatically sets the field type as int64 on ts.

I would appreciate any answers/help. Thank you :bow:

*with decimals*
value: 123.2564
value when cast to float: (float) 123.2564 = 123.2564
type on php: float
value when passed to ts: 123.2564
type when passed to ts: float
ts treats this as: float

*without decimals/whole number*
value: 123
value when cast to float: (float) 123 = 123 (same result even if using round or number_format)
type on php: float
value when passed to ts: 123
type when passed to ts: float
ts treats this as: int64Image 2 for Hello, I am trying to set a float value to a field, however this is being saved as a int64

the field is an object and each field inside it should be a float (see screenshot). however when I do an import and encountered the scenario where values are a whole number but a float type in PHP, it automatically sets the field type as int64 on ts.

I would appreciate any answers/help. Thank you :bow:

*with decimals*
value: 123.2564
value when cast to float: (float) 123.2564 = 123.2564
type on php: float
value when passed to ts: 123.2564
type when passed to ts: float
ts treats this as: float

*without decimals/whole number*
value: 123
value when cast to float: (float) 123 = 123 (same result even if using round or number_format)
type on php: float
value when passed to ts: 123
type when passed to ts: float
ts treats this as: int64
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
05:29 AM
👋 I remember fixing a similar issue on the 0.25 RC builds. Would you be able to quickly try locally against a docker build of 0.25 RC?
05:30
Kishore Nallan
05:30 AM
For e.g. on typesense/typesense:0.25.0.rc48 build.
05:33
Kishore Nallan
05:33 AM
Also the schema looks a bit weird (first screenshot). fields should be part of the collection schema but in the PHP code it seems to be part of regularPrice object field.
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
01:38 PM
Thank you for your response I appreciate it. I am trying to have a nested object for regularPrice, and inside the object should be any property (eg. AUD, NZD, USD etc.) that has a float value

so when returned on the document it should have something like this
{
  // other properties
  "regularPrice": {
    "AUD": 20.20,
    "NZD": 20.00,
    "USD": 12.00,
  }
}
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:43 PM
You can just define a regularPrice field of type object. The types of inner child fields will be auto detected.
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
01:45 PM
yes I tried that however, if document 1 sets regularPrice.AUD to int64 (because of typesense treating a whole number as int64 even if the type of php is float)

then all other document's regularPrice.AUD should be int64. now the problem I have is that some documents are floats and this will error out when importing those documents
01:46
Zildjian
01:46 PM
that is why I explicitly define it to have floats and hoping that would force it
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:53 PM
Ok can you try with field name as regularPrice.*
01:53
Kishore Nallan
01:53 PM
And type as float
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
02:03 PM
ok I'll try
02:03
Zildjian
02:03 PM
thanks
Jul 21, 2023 (4 months ago)
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
01:20 AM
hello again, so I tried this and it now affects other objects
Image 1 for hello again, so I tried this and it now affects other objectsImage 2 for hello again, so I tried this and it now affects other objects
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:51 AM
This is probably fixed in recent 0.25 RC builds (https://github.com/typesense/typesense/issues/973).
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
02:50 AM
this was fixed on 0.25.0.rc24, and our version is rc27
Image 1 for this was fixed on 0.25.0.rc24, and our version is rc27
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:01 AM
Can you please post a smaller code snippet that directly reproduces the issue so I can try it out and find out where it's exactly going wrong? The gist should contain:

1. Creating a collection with a schema with 1 or 2 fileds
2. A sample document ingested that causes the error
With the above it will be easy for me to reproduce what's happening. You can use curl as well, see an example here: https://gist.github.com/kishorenc/865511899735ab23d82b29d8592751c5
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
06:17 AM
hello, here's the command I tried. the results are on the screenshot. thanks
export TYPESENSE_API_KEY=REDACTED
export TYPESENSE_HOST=REDACTED
export TYPESENSE_COLLECTION_NAME=test_products

curl -k "${TYPESENSE_HOST}/collections" -X POST -H "Content-Type: application/json" \
      -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{
        "name": "test_products",      
        "fields": [              
          {"name": "regularPrice.*", "type": "float"}, 
          {"name": "title", "type": "string"}
        ],
        "enable_nested_fields": true       
      }'

curl "${TYPESENSE_HOST}/collections/${TYPESENSE_COLLECTION_NAME}/documents" -X POST \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{"title": "Product 1", "regularPrice": { "AUD": 1, "NZD": 2.2 }}'

curl "${TYPESENSE_HOST}/collections/${TYPESENSE_COLLECTION_NAME}/documents" -X POST \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{"title": "Product 1", "regularPrice": { "AUD": 1.0, "NZD": 2.2 }}'
Image 1 for hello, here's the command I tried. the results are on the screenshot. thanks
```export TYPESENSE_API_KEY=REDACTED
export TYPESENSE_HOST=REDACTED
export TYPESENSE_COLLECTION_NAME=test_products

curl -k "${TYPESENSE_HOST}/collections" -X POST -H "Content-Type: application/json" \
      -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{
        "name": "test_products",      
        "fields": [              
          {"name": "regularPrice.*", "type": "float"}, 
          {"name": "title", "type": "string"}
        ],
        "enable_nested_fields": true       
      }'

curl "${TYPESENSE_HOST}/collections/${TYPESENSE_COLLECTION_NAME}/documents" -X POST \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{"title": "Product 1", "regularPrice": { "AUD": 1, "NZD": 2.2 }}'

curl "${TYPESENSE_HOST}/collections/${TYPESENSE_COLLECTION_NAME}/documents" -X POST \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{"title": "Product 1", "regularPrice": { "AUD": 1.0, "NZD": 2.2 }}'```
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
06:18 AM
:ty: will get back to you
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
06:19 AM
thanks for looking into this. I really appreciate it
Jul 23, 2023 (4 months ago)
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
11:09 PM
Hello, I was wondering if you have replicated the issue?
Jul 24, 2023 (4 months ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:28 AM
Yes I was able to reproduce. Will be looking to fix it in the next few days.

1

Jul 25, 2023 (4 months ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
02:49 AM
I've a fix for this. Do you use Typesense Cloud?
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
02:50 AM
yes
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
02:50 AM
Please ping me the cluster ID and I will schedule an upgrade or you can also verify the fix locally first?
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
02:57 AM
can we do the upgrade ourselves via the cluster configuration?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
02:57 AM
Yes look for 0.25.0.rc54 version
Zildjian
Photo of md5-cbe120c5ccdca7b15a73bb4a03898b5e
Zildjian
02:58 AM
Thanks a lot, I really appreciate your help. I'll let you know how it go

1

04:00
Zildjian
04:00 AM
it worked! you're a legend! thanks.
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
04:05 AM
Glad to hear!

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Typesense Bug Fix with `canceled_at` Field and Upgrade Concerns

Mateo reported an issue regarding the treatment of an optional field by Typesense which was confirmed a bug by Jason. After trying an upgrade, an error arose. Jason explained the bug was due to a recent change and proceeded to downgrade their version. Future upgrade protocols were discussed.

3

74
10mo

Troubleshooting Typesense Connection Issue and Data Retrieval

Felix encountered issues connecting to a server-created typesense and retrieving data collections. Kishore Nallan and Jason provided guidance, highlighting errors in the original code and suggesting changes to the URL and curl command.

63
24mo

Troubleshooting 400 Error When Upgrading Typesense Firestore Extension

Orion experienced a `400` error after updating the Typesense Firestore extension, causing issues with cloud functions. They traced the issue back to a data type conflict in their Typesense collection schema after updating. With help from Jason and Kishore Nallan, they resolved the issue by recreating the collection.

5

96
14mo

Syncing Data to Typesense Cloud Issue

Jameshwart had a problem syncing data to Typesense Cloud. Kishore Nallan provided a fix to this issue by doing an upgrade in the UI. Zildjian confirmed that they were working on the same project as Jameshwart and that the fix should work.

24
4mo

Resolving Issues with Infix and Prefix in Query Searches

Daren struggled with searches missing values in production. Jason and Kishore Nallan offered insights and created new features to help solve the problem, which was then tested and deployed by Daren.

2

38
5mo