How to make OpenCV.js work

Creating production-ready code with OpenCV.js can be challenging. Here are a few tips to make your life easier!

So you’ve created a great piece of algorithm with OpenCV that allows say, to track an ID in real time. It works great and you’re really happy about it. But now you want to be able to run this code in a browser, to create a real time and smooth experience. Hence, the natural thing to do is to look towards OpenCV’s little brother: OpenCV.js.

Sadly, working with OpenCV.js isn’t exactly straight forward, and can easily turn into a nightmare. Here are a 2 problems you may face, and how to solve them:

1) Watch out for memory leaks!

Opencv.js can cause big memory leaks, and solving them is not exactly a walk in the park. For having faced that problem, I found these two tips to be useful:

1. Never store your images in arrays or dict, because this will prevent the garbage collector to work properly. Instead, always use direct references to your images, and use the .delete() method when you’re done with them.
2. This is a bit strange, but it turned out to work. Instead of using OpenCV functions like you normally would (with a src image and a dst image), do the following (to apply canny edges for instance):

dst = src;
// apply Canny on image
cv.Canny(dst, dst, CANNY_LOWER, CANNY_UPPER);

2) Opencv.js is very heavy

Opencv.js is a heavyweight (~7Mo), so it’ll take a few seconds for a user to load your page. Which in most cases, is a big no no. Luckily enough, there is a way to make that build much smaller. Basically the idea is to create a build which only includes the methods your code needs. This small tutorial will teach you how to create an Opencv.js light build in a few steps:

Step 1: Set up Emscripten

To build Opencv, you’ll need to have Emscripten working. Here is how to do it:

# Get the emsdk repo
git clone https://github.com/emscripten-core/emsdk.git

# Enter that directory
cd emsdk

# Fetch the latest version of the emsdk (not needed the first time you clone)
git pull

# Download and install the latest SDK tools.
./emsdk install latest

# Make the "latest" SDK "active" for the current user. (writes .emscripten file)
./emsdk activate latest

# Activate PATH and other environment variables in the current terminal
source ./emsdk_env.sh

You are ready to start working on opencv.js.

Step 2: Prepare the build

First clone the repo:

git clone https://github.com/opencv/opencv.git

For the following step, you’ll need two things:

• Go back to your own code, and find every OpenCV.js function you need to make it run.
• Go to /YOUR_PATH/opencv/platforms/js/opencv_js.config.py, and keep only these functions.

For instance, at the end of this process, my own opencv_js.config.py file looked like this:

You are are now ready to build opencv.js!

Step 3: Make the build

Warning: If you’ve built OpenCV.js in the past, make sure to start from the new repo you just cloned. Indeed, The building script will look for existing caches and go for the same build as before (and you’ll end up with a 7 Mo build).

Choose the WebAssembly build, as it will allow you to gain a few extra Mo.

python ./platforms/js/build_js.py build_wasm

Chances are, at this point you get this error:

Cannot get Emscripten path, please specify it either by EMSCRIPTEN environment variable or --emscripten_dir option.

This is because for some reason, the source ./emsdk_env.sh doesn’t set the EMSCRIPTEN variable as it should. So you’ll need to pass it by hand:

python ./platforms/js/build_js.py build_wasm --build_wasm --emscripten_dir /YOUR_PATH/emsdk/upstream/emscripten

If everything goes well, you should get the following message at the end of your build:

=====
===== Build finished
=====
OpenCV.js location: /YOUR_PATH/opencv/build_wasm/bin/opencv.js

You can now check the size of the corresponding opencv.js file, It should be much smaller than the original 7 Mo.

To give you an idea using the opencv_js.config.py file displayed above will result in 1.5 Mo build. Should you enable Gzip, and you can get it down to 0.5 Mo in your web app!

Congrats, you’ve created a light weight OpenCV.js build!
PS: Don’t hesitate to check our website for more content and news!

BONUS:

You will find here a small repo allowing you to test your OpenCV.js code in just a few clicks, and see the result inside a browser! (Note: this code has been mostly inspired by this tutorial).

Warning: this app runs with the regular OpenCV.js build. Simply replace the opencv.js file with your own light build, and it’ll run all the same!