https://typesense.org logo
Join Slack
Powered by
Hey guys - this should be a simple enough issue bu...
# community-help
d
Hey guys - this should be a simple enough issue but i can’t seem to find a solution. I am adding a facet as “R&B”. While displaying it with refinementList widget it is converted to R&B The data is being sent to the cloud as R&B but it’s being stored as R&B
k
👋 Check if the client you are using is encoding the data somehow.
d
Hi Kishore, I am using Instant Search by algolia and there doesn’t seem to be any documentation regarding displaying with refinementList
k
Can you please tell me what the raw API response contain?
d
Copy code
{
  "cat_link": [
    "<http://typesense.local/category/rb/>"
  ],
  "category": [
    "R&B"
  ],
  "comment_count": 0,
  "id": "2522",
  "is_sticky": 0,
  "permalink": "<http://typesense.local/tupac-biggie/>",
  "post_author": "digamber",
  "post_content": "Legendary Battle",
  "post_date": "2022-02-02 15:46:19",
  "post_excerpt": "",
  "post_id": "2522",
  "post_modified": "2022-02-02 15:55:41",
  "post_thumbnail": "<http://typesense.local/wp-content/plugins/search-with-typesense//assets/placeholder.jpg>",
  "post_title": "Tupac & Biggie",
  "post_type": "post",
  "sort_by_date": 1643816779
}
k
Hmm, okay so the text stored in Typesense itself is encoded. You are using the PHP client to index correct? What I meant is, whether the PHP client is somehow encoding this value before it is sent to Typesense. Because the Typesense server does not try to encode the value prior to indexing.
d
I am not using the PHP SDK, It is a CURL request to the Typesense Cloud server.
k
Oh got it. Let me quickly verify.
I'm using the command line curl client, but it's working fine for me: https://gist.github.com/kishorenc/05fe164f092086488d721e0ea1123ccd
d
Let me make a test with Postman too
Ok it may be something with PHP CURL request automatically converting special chars before sending the request
when i do it via Postman - the values stay as it is
k
Yup, that's what I suspected too.
d
Thanks Kishore - appreciate it
👍 1
Hey @Kishore Nallan, I’ve checked the requests multiple times now. And the data is being sent as a JSON string
Copy code
'{"post_content":"Legendary Battle","post_title":"Tupac & Biggie","post_type":"post","post_author":"digamber","comment_count":0,"is_sticky":0,"post_excerpt":"","post_date":"2022-02-02 15:46:19","sort_by_date":1643816779,"post_id":"2522","post_modified":"2022-02-02 17:29:18","permalink":"http:\/\/typesense.local\/tupac-biggie\/","post_thumbnail":"http:\/\/typesense.local\/wp-content\/plugins\/search-with-typesense\/\/assets\/placeholder.jpg","cat_link":["http:\/\/typesense.local\/category\/rb\/"],"category":["R&B"],"id":"2522"}'
There is no encoding before the data is being sent. When i send this data to my own system/server The data comes in as expected - that is I created an endpoint on my site and posted the data to a file - and I am getting the data without the special chars conversion. Is it something that i’m missing here ? An option i need to set ?
k
But you said that postman also sends the data correctly right?
d
yes postman is indeed sending the data correctly, but when i send and retrieve the data myself
the data is being correctly received
data.txt
data.txt
k
What happens when you insert using the php code, but query using postman?
d
Sorry didn’t follow i created an endpoint called “example.com/test.php” and sent the same request as the one i’m sending to typesense When i make a post to that file with the same params as i do for typesense , i log it to data.txt for viewing. As you can see - the data is without the & for special chars here.
Can you elaborate on what you mean by make the same request to postman ?
k
Sorry. What I meant is: a) Index the document into Typesense using CURL client in PHP. b) But query the Typesense search endpoint using a postman request
I want to know if the encoding issue happens during indexing or during the response.
d
Ah - the response with postman is the same
k
Can you post the PHP CURL example where you are sending the data?
d
That’s a bit trickier i’m using it on a wordpress site - so we’re making use of wp_make_request
can you let me know anything specific you’re searching for ?
*that would help
k
There is certainly some funny business happening in the PHP side of things. If postman works correctly (and so does the cli curl client), then the issue can only be with the php client before the POST body is sent.
d
yes I thought the same thing too - but i’m not able to reproduce the encoding happening from the PHP side of things. Any reason why a CURL request would be received differently by typesense maybe ?
I’m running the same code with same data - once sending to my Typesense server and the other to my apache server - and the data being stored is different
k
Don't think that will be the case. It is certainly encode before it is sent. Maybe need to look into 
wp_make_request
Perhaps Apache also detects the encoding and decodes it?
Are you using 
wp_remote_request
?
d
Yes 
wp_remote_request
is the function i am using
applogies i accidentally said make_request earlier
k
Pretty sure that's somehow encoding the data.
d
I was too - but unless apache is automatically decoding it - and i don’t see how it is with a json_string - there must be something else that is causing it to interpret special chars differently
k
d
That is indeed what i am doing
Copy code
$args         = [
			'method'  => $method,
			'headers' => [
				'X-TYPESENSE-API-KEY' => $this->admin_api_key,
				'Content-Type'        => 'application/json'
			],
		];

		if ( ! empty( $data ) ) {
			$args['body'] = is_string( $data ) ? $data : json_encode( $data );
		}
		$result = wp_remote_request( $this->node . $endpoint, $args );
k
This one seems more relevant: https://wordpress.stackexchange.com/a/313036
Can you try setting ` `'data_format' => 'body',``
d
Checking
Nope still getting the same result
additionally i tried logging the result after the upsert request was completed
One thing i’m noticing now is that Tupac & Biggie - in post title isn’t being encoded Only The category with R&B is being encoded.
the category is a string of arrays while post_title is a string
k
My php knowledge isn't enough to help further here. But I think if you post on stackoverflow someone should be able to help easily.
d
@Kishore Nallan appreciate all your help my friend - I was able to figure out the issue WordPress saves all post_title etc as is without html entity encoding it. But for categories they html entity encode it. So since i was viewing the data via the browser / WordPress I was always seeing & instead of & I have now solved the issue. Thank you for your time and help. Cheers.
k
Awesome 😎👍
Open in Slack
PreviousNext
Powered by