Sending non-English text via HTTP in NodeJS

If you are making an HTTP(S) post request via NodeJS, and have non-English text (Hindi, Japanese, or any other script) in the post body, chances are that your requests are failing. I encountered this in one of my recent projects where I was sending Hindi text in the post body. The root cause was that the value sent in the ‘content-length’ header was incorrect.

The post body was in JSON format. My request was looking something like this:

const https = require('https');

const postdata = JSON.stringify({
  first_name: 'यश',
  last_name: 'संघवी'
});

console.log("Postdata length: " + postdata.length.toString());

const options = {
  hostname: 'example.com',
  port: 443,
  path: '/example_path',
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': postdata.length,
  },
};

const req = https.request(options, res => {
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', d => {
    process.stdout.write(d);
  });
});

req.on('error', error => {
  console.error(error);
});

req.write(postdata);
req.end();

If you run the above code, you’ll see that the postdata length is 39. However, if I sent the same request via Postman, and observed the logs in the Postman console, the content-length gets printed as 53.

So what’s going on? Well, the answer is that when you have non-English text, or even if you have English text, it is better to count the length of the utf-8 encoded text. This way, you’ll get the actual number of bytes. Non-English characters may take up more than one byte, and a simple .length() operation on a string won’t reveal that. Here’s the right way to make this request:

const https = require('https');
const utf8 = require('utf8');

const postdata = JSON.stringify({
  first_name: 'यश',
  last_name: 'संघवी'
});

let content = utf8.encode(postdata);

console.log("Postdata length: " + content.length.toString());

const options = {
  hostname: 'example.com',
  port: 443,
  path: '/example_path',
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Content-Length': content.length,
  },
};

const req = https.request(options, res => {
  console.log(`statusCode: ${res.statusCode}`);

  res.on('data', d => {
    process.stdout.write(d);
  });
});

req.on('error', error => {
  console.error(error);
});

req.write(postdata);
req.end();

Now, the length of the utf8 encoded data is 53, as expected. This request will go through (provided that the host and path are correct).

---

For more tutorials on Nodejs, check out https://iotespresso.com/category/nodejs