Sending non-English text via HTTP in NodeJS

If you are making an HTTP(S) post request via NodeJS, and have non-English text (Hindi, Japanese, or any other script) in the post body, chances are that your requests are failing. I encountered this in one of my recent projects where I was sending Hindi text in the post body. The root cause was that the value sent in the ‘content-length’ header was incorrect.

The post body was in JSON format. My request was looking something like this:

const https = require('https');

const postdata = JSON.stringify({
  first_name: 'यश',
  last_name: 'संघवी'
});

console.log("Postdata length: " + postdata.length.toString());

const options = {
  hostname: 'example.com',
  port: 443,
  path: '/example_path',
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': postdata.length,
  },
};

const req = https.request(options, res => {
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', d => {
    process.stdout.write(d);
  });
});

req.on('error', error => {
  console.error(error);
});

req.write(postdata);
req.end();

If you run the above code, you’ll see that the postdata length is 39. However, if I sent the same request via Postman, and observed the logs in the Postman console, the content-length gets printed as 53.

So what’s going on? Well, the answer is that when you have non-English text, or even if you have English text, it is better to count the length of the utf-8 encoded text. This way, you’ll get the actual number of bytes. Non-English characters may take up more than one byte, and a simple .length() operation on a string won’t reveal that. Here’s the right way to make this request:

const https = require('https');
const utf8 = require('utf8');

const postdata = JSON.stringify({
  first_name: 'यश',
  last_name: 'संघवी'
});

let content = utf8.encode(postdata);

console.log("Postdata length: " + content.length.toString());

const options = {
  hostname: 'example.com',
  port: 443,
  path: '/example_path',
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': content.length,
  },
};

const req = https.request(options, res => {
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', d => {
    process.stdout.write(d);
  });
});

req.on('error', error => {
  console.error(error);
});

req.write(postdata);
req.end();

Now, the length of the utf8 encoded data is 53, as expected. This request will go through (provided that the host and path are correct).


For more tutorials on Nodejs, check out https://iotespresso.com/category/nodejs

Leave a comment

Your email address will not be published.