2014-04-15

CREATING A MOSAIC

I haven't been working on image processing tools for a long time as I've been quite busy with many other interesting projects ;)

One of them was migrating the old website to Blogger so it will be faster for me to publish new posts.

After I finished it, I decided that I wanted to go back to the basics of image processing: the COLORS.

I wanted to learn more about them, how to represent them, how to compare, how to find "similar" colors... in fact, what do we mean by similar colors?



A couple weeks ago, I saw on a TV program a mosaic formed of many small images that let you see another image when looked from a far distance and I thought that it would be possible, and funny ;P,  to create them directly on the browser so.... why not doing it!?

My goal was to create a mosaic, representing an image, formed by many small tiles of other images on the browser side (everything running on JavaScript) and created on the fly.

My first demo was very simple, we iterate through the original image pixels, we choose which of the available tiles is the one "closer" to the pixel color (we will talk later about this) and then we draw it inside the DOM using a canvas.

As I was using a canvas, the demo was very simple as once we have decided which images we want to draw, we just need to add them to the canvas one by one. Here, I found the "first stone on my pathway" as canvas is not a very fast method to draw elements in the DOM and we were talking BIG this time.

Imagine a "very small" image of 300x300 pixels, in other words, 90.000 pixels. If you are using a relative small tile size of 256x256, we are managing a canvas representing an image of 76.800x76.800 pixels.
If we extrapolate that image size to a normal camera resolution we would have a 5898.2 megapixels camera photo. That's what I call a nice camera sensor ;)

It was obvious that the canvas path was doomed as I wouldn't be able to go much further from there... so I decided to enter a whole new world: WebGL!!!

In the past, I tried a couple times to build some really basic demos using Three.js (a JavaScript library to render elements using WebGL) building some 3D terrain structures in the browser so this was a great opportunity to get a more deep knowledge about it.

I created a second version of the demo using Three.js and the result was amazing as I was able to create a mosaic of 600x600 pixels size using tiles of 512x512 pixels, before my browser started to feel "sluggish".
This means that the browser was rendering an image of 307.200x307.200 pixels, which would be approximately a photo taken with a camera of 94371.8 Megapixels!!!

As reference, the test computer was an Intel Core i5-2400 (3.10GHz) with 8GB of RAM running Windows 7.

At that point, I thought that the demo was missing some "creativity" as I was using fixed tiles so I got the idea to pull images randomly from an external source and use them in the mosaic. I thought that it would be really nice to pull the images from some open source online that would be changing with time so our image generated would change with time.

This ended up being harder than expected due to the restrictions to access pixel information on images (CORS) because we require the images requested to come with agreements to access their pixels data.

My first code version was using Flickr to get the images as it seemed to be supporting CORS headers in their responses but... I found that this is not completely true.

I found out that a big percentage of the requests from Firefox were returning double CORS information in the headers, making the browser to reject them. Also, very strangely the... cough-Yahoo-cough... service was returning double CORS information on 100% of Google Chrome requests... so there was no possibility to access pixel data from images requested from Chrome to Flickr.

I posted the bug at their support forum but there was no answers so I decided to move on. I wanted to finish "baking" my demo :D

I found out that Imgur was also providing random images with fully CORS support with the only difference that their API limits the access to the latest images uploaded (2000) and the returned image lists are pre-calculated every hour. That's not real time but it's far more than enough for this demo as I only need around 100-200 images and the randomness produced is enough for the demo.

So I opted for a dual solution to be able to test both cases:
  • Firefox: pulls the images from Flickr, ignoring the ones with invalid CORS information.
  • Google Chrome: pulls the images from Imgur and all of them are correctly received ;)
  • Others: I didn't really check them for this demo as it was mainly intended for Chrome.


Article references:

No comments :