Bar codes are quite. They don’t make any sounds. You don’t need electricity to broadcast them once they are printed. They can be placed in inconspicuous places. Many phones now have the ability to scan the codes.

Many years ago (January 2010), I had an idea to create a QR Code Scavenger Hunt in Second Life. This was before they were a big hit in the US. It’s nice to see a lot of these ideas come into fruition.

I also experimented with Augmented Reality with ARTag using 2D matrix codes that represent numbers 0 to 2047 (skipping 46 illegal numbers) for alignment.

Back to the subject at hand

I think we already know that I’m going to focus on two dimensional QR codes. I suppose I’m biased. However, let’s take a look at what is available. Maybe I’ll get some inspiration for something else, or find something better suited.

One dimensional

One dimensional codes are perhaps the easiest and quickest codes to scan and translate. They don’t contain much data at all. The data is in a separate location (such as a database), and the barcodes are only a reference number to lookup that information. They are created in such a way that they can be scanned upside-down or right-side-up. Many barcodes represent only digits, but some may have characters as well. The average size is around 10 digits.

If I were to include these within the game, they would purely be fore lookup of information in a database, or to provide a seed for random generation.

Two Dimensional

Composite

Composite codes make up two or more barcodes. They can be two dimensional, one dimensional, or a mix. They often represent multiple types of information. For example, the “ISBN 13 + 5 Digits” may represent both the ISBN of a book, followed by the price in US Pennies.

That was fun looking at an assortment of codes. QR Codes are so widespread that the camera on my phone was able to scan the code itself and tell me what was inside. Other codes need software or hardware to handle them.

Barcode Detection API

Let’s look at what the web api’s support for the Barcode Detection API. It looks like we are in luck. Rather than just QR codes, it supports a wide variety.

We have four codes to pick from that are two-dimensional. Let’s review how much data each could potentially store.

Format	Alpha	Alphanumeric	Digits	Bytes
Aztec	3,067 upper-case	~2,500 upper/lower-case + digits	3,832	1,914
Data Matrix	2,355	2,355	3,116	1,556
PDF417	1,850 upper-case	1,850 uppercase + digits	2,710	1,108
QR Code	1,817 Kanji	4,296 upper-case, digits, space, $%*+-,./:	7,089	2,953

I didn’t realize how much of an advantage QR codes had over other 2D matrices. I didn’t realize how much binary data they could contain. The packetization from the air wobbler app could come in handy for fast transfer of files offline. The QR code can store six of my 487 byte test images with 31 bytes remaining.

Maximum Capacity

Let’s see what these codes look like at max capacity and see if it is difficult to scan them. I’m using Lorem Ipsum and TEC-ID Online Barcode Generator.

My phone is only able to recognize QR codes with the camera and surprisingly it picked up the contents instantly. I thought I would have trouble trying to scan it because it was so large. I’ve had trouble scanning PDF417 codes on drivers licenses before when checking people in to donate blood, and they only contain a name and address. I think its time to create an app.

Barcode Detector is not supported by this browser.

Well… ain’t that special. Looking at the browsers that support Barcode Detection, Google Chrome is listed offering partial support since May 2020. Perhaps local web hosting is not supported without https.

Barcode Detector supported!

So I had to do two things. The first was to create a self-signed certificate.

openssl req -x509 -out localhost.crt -keyout localhost.key \
-newkey rsa:2048 -nodes -sha256 \
-subj '/CN=localhost' -extensions EXT -config <( \
	printf "[dn]\nCN=localhost\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")

Next, I had to find a quick and dirty server that supported SSL. OpenAI’s ChatGPT suggested a php server.

php -S localhost:8000 -t . --ssl --cert localhost.crt --key localhost.key

Awesome! Or not… php doesn’t support it. It’s difficult to tust anything with ChatGPT as it often looks like it knows what its talking about, but doesn’t. I ended up making a web server from node.

const https = require('https');
const fs = require('fs');
const path = require('path');

const options = {
  key: fs.readFileSync('localhost.key'),
  cert: fs.readFileSync('localhost.crt')
};

const contentTypes = {
  js: 'text/javascript',
  html: 'text/html'
}

const server = https.createServer(options, ({url}, res) => {
  console.log(url);
  if(url.endsWith('/')) url += 'index.html';
  let filePath = '.' + url;
  fs.readFile(filePath, (err, content) => {
    if(err) {
      if(err.code === "ENOENT") {
        res.writeHead(404);
        res.end('404: Chuck Norris tried to find the file, but it fled in terror at the sight of him.');
      } else {
        res.writeHead(500);
        res.end('500: Chuck Norris told the server a joke, and it laughed so hard, it crashed.')
      }
    } else {
      let contentType = contentTypes[path.extname(filePath).slice(1)] ?? 'text/plain';
      res.writeHead(200, { 'Content-Type': contentType });
      res.end(content, 'utf-8');
    }
  })
});

const PORT = 8000;
server.listen(PORT, () => {
  console.log(`Server running at https://localhost:${PORT}/`);
});

Yea – it’s missing lots of stuff, but it works.

The scanner is fairly simple. You can ask what barcode formats it supports. Then you create a new barcode scanner object with the formats you want to detect. Then you pass it an image to detect. It comes back later via promise and lets you iterate through the detected barcodes. It even tells you where in the image the barcode was found.

Let’s take it a step further and use the camera to get live barcode detection.

Well that was fun! I can hold my phone up to my laptops camera and scan QR codes. I found a flyer of a civic pride workshop that I sponsored last year.

I added a canvas element to copy the video and drew a path around the detected barcode. I had to do some fancy footwork to detect the barcodes only when the video element was ready, and if the last detection had completed. The end result is that the animation frame may be a little out of sync with the actual barcode detection – but it will be almost spot on.

Let’s make this app actually useful. How about I grab that QR code and create a small icon of it next to the text found inside?

Well, that was a bit of a nightmare going down a few rabbit holes to get to the carrot. I got far by performing a rotation and clipping path on the image with the provided corners of the barcode, but it just was’t far enough. I looked at a few libraries including image-js and opencv as well as a bunch of forums. I settled on opencv that could warp the perspective of images to grab rectangular objects and restore them to have parallel edges.

https://docs.opencv.org/4.5.4/opencv.js

const unwarpClipAsURL = async (original, cornerPoints) => {
  const [
    topLeft,
    topRight,
    bottomRight,
    bottomLeft
  ] = cornerPoints;

  // get unwarped width/height
  const width = Math.sqrt(
    (bottomRight.x - bottomLeft.x) ** 2 + 
    (bottomRight.y - bottomLeft.y) ** 2
  );
  const height = Math.sqrt(
    (topRight.x - bottomRight.x) ** 2 +
    (topRight.y - bottomRight.y) ** 2
  );

  // copy original image to canvas
  const canvas = document.createElement('canvas');
  const ctx = canvas.getContext('2d');
  canvas.width = original.width;
  canvas.height = original.height;
  ctx.drawImage(original, 0, 0, canvas.width, canvas.height);

  // pass canvas to cv
  const sourceImage = await cv.imread(canvas);

  // Convert corners to flat array
  const count = cornerPoints.length;
  const sourcePoints = cornerPoints.reduce(
    (points, point) => [...points, point.x, point.y], 
    []
  );

  // Where the corners should end up
  const destinationPoints = [
    0, 0,
    width, 0,
    width, height,
    0, height
  ];

  const sourceMatrix = cv.matFromArray(
    count, 1, cv.CV_32FC2, sourcePoints
  );
  const destinationMatrix = cv.matFromArray(
    count, 1, cv.CV_32FC2, destinationPoints
  );
  // Warp it
  const matrix = cv.getPerspectiveTransform(
    sourceMatrix,
    destinationMatrix
  );
  const warped = new cv.Mat();
  cv.warpPerspective(
    sourceImage, warped, matrix, sourceImage.size()
  );

  // Cleanup
  sourceMatrix.delete();
  destinationMatrix.delete();
  matrix.delete();

  // Back to image
  const imageData = new ImageData( 
    new Uint8ClampedArray(warped.data),
    warped.cols,
    warped.rows
  );
  canvas.width = width;
  canvas.height = height;
  ctx.putImageData(imageData, 0, 0);
  return canvas.toDataURL();
}

Given that it takes some time to read the file, we are working with promises. Although we have the barcode data rite away, I don’t display it until the clipped image is loaded.

const scanVideo = () => {
  const video = document.getElementById('video');
  if(readyToDetect && video && !video.paused && video.srcObject) {
    readyToDetect = false;
    detector.detect(video).then(codes => {
      barcodes = codes;
      barcodes.forEach(({ format, cornerPoints, rawValue, boundingBox }) => {
        if(!detected.find(d => d.rawValue === rawValue)) {
          const image = new Image();
          detected.push({
            format,
            rawValue,
            image
          });
          unwarpClipAsURL(video, cornerPoints, boundingBox).then(src => {
            image.src = src;
            showDetected();
          })
        }
      });
    }).catch(err => {
      console.log(err);
    }).finally(() => readyToDetect = true);
  }
}

Should we make it look better?

I made it look a bit pretty. I added a separate page that lets you delete the record or copy it to your clipboard. I wired up indexDb to store the images.

Now for the fun part – the phone… I published the page on GitHub

• Source: https://github.com/lewismoten/Barcode-Detection
• Demo: https://lewismoten.github.io/Barcode-Detection/

Well guess what? The iPhone doesn’t support the Barcode Detection API. Specifically Safari doesn’t support it. Safari on both iOS and MacOS doesn’t work. What about Chrome on iOS? Same problem.

Hmm… I’m kinda stuck. This is a great feature, but its not supported. It looks like it may have come out in 2020. Four years is a long time of no support. It looks like Firefox doesn’t support it either. Only Chrome, Edge, and Opera.

I could look to see if any libraries can detect QR codes in images.