We left off with a successful broadcast of all packets passing CRC checks when transferring a binary file. Unfortunately, the final image didn’t look much like the original and was missing a lot of data. In addition, our progress bar was no longer working.

The stream manager is now working with a byte array instead of bits. As each successful packet arrives, it is unpacked and inserted into the byte array. Our original file is 487 bytes, but we are only receiving 224 bytes. It looks like the logic to determine the maximum number of packets to transfer is not providing the correct numbers.

export const getPacketCount = (bitCount) => 
  canSendPacket() ? Math.ceil(bitCount / getPacketEncodedBitCount()) : 0;

export const getPacketEncodedBitCount = () => getPacketEncodingBlockCount() * PACKET_ENCODED_BLOCK_SIZE;

export const getPacketEncodingBlockCount = () =>
  IS_ENCODED ? Math.floor(getPacketMaxBitCount() / PACKET_ENCODED_BLOCK_SIZE) : getPacketMaxBitCount();

export const getPacketMaxBitCount = () => (2 ** PACKET_SIZE_BITS) * 8;

Lots of function calls. I can see where the problem is. The bit count is the number of decoded data bits. It doesn’t take packet headers into consideration. We shouldn’t be using getPacketEncodedBitCount unless we address the headers. Instead, we need to use packetDataByteCount. Let’s rename the function for clarity.

export const packetsNeededToTransferBytes = (byteCount) => 
  canSendPacket() ? Math.ceil(byteCount / getPacketDataByteCount()) : 0;

export const getPacketDataByteCount = () => {
  const availableBitCount = getPacketEncodedBitCount();
  const blocks = Math.floor(availableBitCount / PACKET_ENCODED_BLOCK_SIZE);
  let decodedBits = blocks * PACKET_DECODED_BLOCK_SIZE;
  decodedBits -= getPacketHeaderBitCount()
  // We only transfer full bytes within packets
  return Math.floor(decodedBits / 8);
}

export const getPacketHeaderBitCount = (padAsBytes = true) => {
  const bitCount = PACKET_CRC_BIT_COUNT +
    PACKET_SEQUENCE_NUMBER_BIT_COUNT +
    PACKET_SIZE_BITS;
  if(padAsBytes && bitCount % 8 !== 0) {
    return bitCount + (8 - (bitCount % 8))
  }
  return bitCount;
}

This will get is very far. Mind you, it doesn’t consider the packetization headers themselves. Let’s test it anyway.

Wow. So much information was retrieved from that test that tells me I moved in the correct direction. The first test resulted in a garbled up image. I thought may packets had failed towards the end of the stream. It turns out that only 1 packet failed. Rather than resetting the data, I played the whole audio stream again. Even more packets failed this time – but the one that failed last time came through just fine and completed the image. Now that’s exactly the situation I was looking for.

This means that one-way communications (ie – playing an audio cassette tape) can re-transmit the entire signal, and the receiver can fill in the missing pieces that didn’t come through the last time. Two days ago I was trying again and again many times, hopeful that the full image would come through. I got it now in just two attempts.

The other problem I was trying to address was the sampling rate. You can see here that the sampling rates at the top of the image are between 4 to 7. Previously they were between 0 to 2 towards the end of the transfer, with the majority being 0. My hunch about memory being the culprit was correct.

Looking at the data coming through as ASCII, I’m noticing the packet header at the beginning of the data stream instead of the file. You’ll notice that our GIF89a tag is not at the beginning. This version of GIF (Graphics Interchange Format) had its specification released in 1989 by CompuServe. I had often thought the “a” was for “Animation” since animations were introduced. However, GIF87a did not have animations. The “a” stands for “Amendment”.

All of this extra information is okay. Our function to get the data takes care of packet headers already.

export const getDataBytes = () => {
  const dataSize = getTransferByteCount();
  const dataSizeTrusted = isTransferByteCountTrusted();
  const headerByteCount = getStreamHeaderByteCount();
  if(dataSizeTrusted) {
    return DATA.subarray(headerByteCount, headerByteCount + dataSize);
  } else {
    return DATA.subarray(headerByteCount);
  }
}

So back to the packet header calculations and its affect. You’ll notice that at the end of our transmission, we had a packet of all zeros…

Our changes had opened up a can of worms in changes that need to be addressed. We now need to send in a bit count instead of a byte count, and we need to let the packet packer handle the packetization headers.

Just for good measure, let’s go back to our calculations and add in the packet header.

Something is odd, and time is being wasted. It’s time to add a new panel to request failed packets individually.

Thinking things over, I realized I could speed things up if I just automate the request by default.

Well that took awhile. I got the interface wired up to show that CRC checks are not available until the necessary information has been received. The size comes through first as you only need the first 3 bytes of the underlying data stream (16 bit size header and 8 bit crc header). The CRC Check itself takes a long time because you not only need the size, but you also need all of the data that is being transferred. In addition, if the CRC size fails, we are unable to determine how much data we need to perform the CRC check on, so it remains invalid if the size crc fails.

Now that we have trust built in regarding the size, I was able to remove high packet numbers from the failed requests. It just meant that at minimum, the packet sequence number was corrupted. Afterwards I was able to get the total number of packets expected based off of the packetStats, and use the number of failed/successful packet id’s to identify how much as transferred so far.

export const getPercentReceived = () => {
  if(!isSizeTrusted()) return 0;
  const { packetCount } = PacketUtils.packetStats(getSize());
  if(packetCount === 0) return 0;
  const received = FAILED_SEQUENCES.length + SUCCESS_SEQUENCES.length;
  return received / packetCount;
}

Previously the progress bar was based on how many sampling periods were expected. As a result, the progress bar is no longer smooth. It jumps a little as each packet arrives – but now it represents the total packets completed. I would also like for the right hand side of the progress bar to represent the percent of packets that failed. This way I can tell more easily how badly the transfer is going.

Image Creator

Just taking a little break, I started playing with some AI prompts to create some images related to the project using Microsoft Designer Image Creator.

Prompt: A cartoon of an air molecule holding a microphone and dancing to make audio waves of binary data

Prompt: A cartoon of a giant hand reaching down to an air molecule and wobbling it to make sounds

Prompt: a cartoon of an air wobbler creating digital frequencies over audio waves

Prompt: a cartoon of an air molecule acting like a modem by vibrating vigorously and creating digital frequencies with sound waves

It seems like the AI is focused mostly on blue balls of balls for an air molecule. Just saying “Air Wobbler” alone confused it enough to get creative as to what an unfamiliar thing would look like given the name and context.

Progress Bars

Back to the grind stone. We’ve gotten pretty far. I’ve gotten the progress bar working again. Let’s see what I can do about showing a percentage of errors.

Rather than letting the stream calculate how much it received, I’m going to expose the number of expected, failed, and successful packets.

export const countFailedPackets = () => FAILED_SEQUENCES.length;
export const countSuccessfulPackets = () => SUCCESS_SEQUENCES.length;
export const countExpectedPackets = () => {
  if(!isSizeTrusted()) return 0;
  return PacketUtils.packetStats(getSize()).packetCount;
}

There we go. My progress bars now support multiple values. I added a bit of an extra check to make sure that the combined total never goes above 100%;

  addProgressBar = (id, ...percents) => {
    const progressBar = document.createElement('div');
    progressBar.className = 'progress-container';
    let sum = 0;
    for(let i = 0; i < percents.length; i++) {
      let percent = percents[i];
      percent = clamp(percent, 0, 1 - sum);
      sum += percent;
      const bar = document.createElement('div');
      bar.id = this.childId(`${id}-${i}`);
      bar.className = 'progress-bar';
      bar.style.width = `${clamp(percent, 0, 1) * 100}%`;
      progressBar.append(bar);
    }
    this.append(progressBar);
  }
  setProgressById = (id, ...percents) => {
    let sum = 0;
    for(let i = 0; i < percents.length; i++) {
      let percent = percents[i];
      percent = clamp(percent, 0, 1 - sum);
      sum += percent;
      const element = document.getElementById(this.childId(`${id}-${i}`));
      if(!element) throw new Error(`Unable to find ${id}`);
      element.style.width = `${clamp(percent, 0, 1) * 100}%`;
    }
  }

Since the progress bar only moves after the packets are evaluated, I decided to break up the choppiness of the progress by with a transition.

.progress-bar {
  height: 100%;
  min-height: 14px;
  background-color: yellow;
  position: relative;
  width: 0;
  display: inline-block;
  transition: width 1s ease;
}
.progress-bar:nth-child(2) {
  background-color: red;
}

Prompt for prompt

I asked Chat GPT to create a prompt.

Prompt: create an image prompt to create a logo for a web page that creates audio signals so that people can have their devices talk to each other over sound waves when they hold the devices together. The logo should give an idea that a digital signal is sent over sound waves.

OpenAI ChatGPT:

Design a logo for a web page that facilitates communication between devices using sound waves. The web page allows people to hold their devices together, enabling them to exchange digital signals over sound waves.

Consider incorporating elements that convey the concept of digital signal transmission through sound waves. You might include visual representations of sound waves, digital signals, and devices interacting with each other.

The logo should be visually appealing, memorable, and scalable to various sizes. Use colors and shapes that evoke a sense of technology and connectivity.

Feel free to use any design tools or mediums you prefer, such as digital illustration software, vector graphics, or hand-drawn sketches. Be creative and innovative in your design approach to effectively communicate the concept of device-to-device communication via sound waves.

I dropped the last paragraph and pasted the whole thing into Image Creator.

These images are much busier, but look a bit cooler. I see lots of hands and mobile devices.

I was hoping for something more simpler. Nothing is really calling out to me.

It’s time to get some food…

Retransmission

So this is what we’ve been leading up to. We have all of our packets. We know which specific ones failed. Our logic has been updated to apply packets based on packet headers. All that is left is to make a request. Well… we aren’t going to talk back with sound waves. We are just going to request the packets via memory.

Ack! It works, but it … works. I have two different problems. One problem is that it sometimes gets in a loop where it keeps requesting packet 35. There is no packet 35! So it fails and tries again, and again. The other problem is failing header packets. Let’s say we’ve received packet 0 and it has all of the bytes necessary to determine the data length, data crc, and data length crc. Unfortunately that packet fails the CRC check. Ok… in that case, we don’t add it to our received data. Next we receive packet 1. The logic confirms that we have enough data for all of our headers to be received. It further confirms that the CRC performed on the size is valid as a CRC-8 check on 0 will return 0. The data length is evaluated as… 0. Packet 1 is considered to be out of range and dropped! The process continues to repeat for all remaining packets. The entire transmission is ignored if the headers fail to transfer. The answer is to not only check to see if enough bytes were received, but also that the packets didn’t fail.

It looks like the first problem is that we are calculating a packet number that is higher than how many packets we have. I have a hunch that the last packet is full of zeros… actually, we’ve already seen it is full of zeros earlier.

Everything after that blue packet line was zero. Actually, lets go ahead and make that image have a background. It’s hard to see much else with a white web page background.

I got most of the retransmission worked out as well as the automation. It’s been a long day working through the problems.

Here is a video demonstrating the requests of individual packets in action.