Taking a Picture - One Pixel at a Time

A camera that only has a one-pixel sensor. It moves an APDS-9960 across two axes to create a full-color image.

Things used in this project

Hardware components

SparkFun APDS-9960 x1

Jumper wires (generic) x1

Arduino Mega 2560 & Genuino Mega 2560 x1

NEMA17 Stepper Motor x2

Adafruit SD card reader x1

DFRobot Dual Stepper Motor Driver Shield x1

Software apps and online services

Arduino IDE

Microsoft VS Code

Hand tools and fabrication machines

Hot glue gun (generic)

3D Printer (generic)

CNC Router

Story

Previous Project

In my recent project, I created a machine that could create images using only one LED. It used techniques found in light painting to move an LED across two axes. 

But what about doing the opposite? Is it possible to create an image using only a single RGB sensor? That is what I wanted to find out.

The Plan

I wanted to use the same two-axis CNC machine I had built previously due to its simplicity and ease-of-use. 

For the RGB color sensor, I needed something reliable and consistent, which led me to pick the APDS-9960 gesture and color sensor. 

The machine would move the sensor across the X-axis first, taking readings at set intervals, and then move down one interval to the next row. After doing this repeatedly, the readings would be written to a file on an SD card, which can then be read by a Python script and a recreated image shown. 

New File Format

Bitmaps are great, but there is a major drawback to them: large metadata headers. On a regular windows bitmap file, the metadata header takes up 54 bytes, but most of this information is useless. That’s why I made the Arduino Image Format (.AIF). It is extremely simple, with the first two bytes for width, the next two for height, and one byte for color depth. The rest of the file is for the image array, which allows for two dimensional arrays to be written to a file with ease. 

Because data is written in Most Significant Byte format, any data over 1 byte must be broken up into multiple bytes. Since the width and height numbers are two bytes each, some bitwise functions are necessary. For example, imagine writing the number 1234. Converted to binary, it becomes 00000100 11010010. Then the first byte needs to be isolated, so a bit-mask of 11111111 00000000 (0xFF00) is necessary. 

00000100 11010010
&
11111111 00000000
-----------------
00000100 00000000

Using the bitwise operator & (AND) makes it become 00000100 00000000. 

00000100 00000000
>> 8
-----------------
00000100

Then the bits can be shifted to the right 8 times using >>, leaving 00000100. For the second half, all that is needed is a mask of 11111111 (0xFF), leaving 11010010. 

00000100 11010010
&
00000000 11111111
-----------------
00000000 11010010 

The Camera Itself

The first program runs on the Arduino Mega 2560, and it controls the two-axis CNC machine, along with the APDS-9960. It begins by initializing the two stepper motors, along with the APDS-9960 sensor in RGB color mode. 

Next, it creates a file on the SD card called “pixelImg.aif” and writes some basic information to it. This includes the width, height, and color depth. After creating the file, it begins to create the photograph. 

The number of pixels is defined by the physical distance each axis is, along with the distance between each reading. This gives a resolution of pixels/total distance. For instance, making the X axis 150mm and the Y axis 100mm with a spacing of.2mm gives a resolution of 25px/mm2 and a total size of 750 x 500px. After each read, three bytes get written to the file in RGB format, with one byte per color. Once the image is finished, the machine gets repositioned to the origin point (0, 0). 

Taking a Picture

For the test, I wanted to create a small yet recognizable image, which meant having a resolution over 1px/mm2 and a size over 50 x 50 pixels. I opted to use a size of 60px x 300px and a physical distance of 60mm x 150mm, giving a resolution of 1x.5px/mm2. Before taking the picture, I had to ensure the device wouldn’t be susceptible to accidental bumps or strong vibrations. Then, I switched it on and started the scan. The speed could be adjusted by changing the RPM parameter near the top of the file. 

I chose this subject to take a picture of:

Viewing the Image

Since the machine creates a custom file, it is also necessary to create a custom program to read it. It starts by opening the image file and reading the metadata header. Since the size data is encoded in unsigned 16-bit data chunks, it is necessary to use int.from_bytes(imgFile.read(2), byteorder='big', signed = False) to convert the two bytes into an int object. Then, a new blank image is created with PIL’s Image.new() function. The picture is then generated by looping through the rows and then columns, reading three bytes per pixel for each corresponding color. img.putpixel((col, row), (r, g, b)) is then used to change the color of the blank image’s pixel to the color read in the file. 

Once all the data has been read, the image is then shown and saved to a file for later processing and viewing. Depending on the lighting circumstances, post processing might be needed. That was the case with one of my images. I started by adjusting the contrast and increasing the brightness to bring out the wide gamut of colors. 

Next, I adjusted the colors to be more realistic. 

It went from this image:

To this image:

This picture looks very similar to the subject it scanned, so I would consider it a success. 

Custom parts and enclosures

CNC Machine

Schematics

Motor Driver Pinout

SD Card Wiring

Code

"Camera" CodeC/C++
Run this on the Arduino Mega

#include <Wire.h>
#include <SparkFun_APDS9960.h>
#include <SD.h>
#include <SPI.h>
#include "DRV8825.h"

#define MOTOR_STEPS 200
#define RPM 60
#define MICROSTEPS 32

#define MM_X 60
#define MM_Y 150

//pin definitions
#define STEPPER_X_DIR 7
#define STEPPER_X_STEP 6
#define STEPPER_X_EN 8
#define STEPPER_Y_DIR 4
#define STEPPER_Y_STEP 5
#define STEPPER_Y_EN 12

#define X 0
#define Y 1

#define X_DIR_FLAG -1 //1 or -1 to flip direction
#define Y_DIR_FLAG 1 //1 or -1 to flip direction

#define STEPS_PER_MM (3.75 * MICROSTEPS) //steps needed to move 1mm
#define PIXELS_W 60
#define PIXELS_H 300
#define SPACE_BETWEEN_POSITIONS_X 1
#define SPACE_BETWEEN_POSITIONS_Y .5

#define MULTIPLIER 2 //Scale each color by this amount

#define SD_CS 22

int currentPositions[] = {0, 0};

File image;

DRV8825 stepperX(MOTOR_STEPS, STEPPER_X_DIR, STEPPER_X_STEP, STEPPER_X_EN);
DRV8825 stepperY(MOTOR_STEPS, STEPPER_Y_DIR, STEPPER_Y_STEP, STEPPER_Y_EN);
SparkFun_APDS9960 apds = SparkFun_APDS9960();

uint8_t light_levels[6] = {};
uint16_t red_light = 0;
uint16_t green_light = 0;
uint16_t blue_light = 0;

unsigned long pixel_count = 0;

void setup(){
  Serial.begin(115200);
  SD.begin(SD_CS);
  init_apds();
  init_file();
  take_picture();
}

void loop(){
  
}

void take_picture(){
  unsigned long start_time = millis(); 
  for(int row=0; row<PIXELS_H; row++){
    for(int col=0; col<PIXELS_W; col++){
      moveToPosition(col, row);
      //delay(50);
      readLightLevels();
      writeLightLevels();
      pixel_count++;
      if(pixel_count > 1000){
        image.close();
        image = SD.open("pixelImg.aif", FILE_WRITE);
        pixel_count = 0;
      }
    }
  }
  image.close();
  moveToPosition(0, 0);
  Serial.print("Image finished! It took "); Serial.print((millis() - start_time) / 1000); Serial.println(" seconds to complete.");
}

void writeLightLevels(){
  image.write(light_levels, 3);
}

void readLightLevels(){
  if (  !apds.readRedLight(red_light) ||
        !apds.readGreenLight(green_light) ||
        !apds.readBlueLight(blue_light) ) {
    Serial.println("Error reading light values");
  } else {
    Serial.print("Red: ");
    Serial.print(red_light);
    Serial.print(" Green: ");
    Serial.print(green_light);
    Serial.print(" Blue: ");
    Serial.println(blue_light);
  }
  light_levels[0] = red_light * MULTIPLIER;
  light_levels[1] = green_light * MULTIPLIER;
  light_levels[2] = blue_light * MULTIPLIER;
}

void moveToPosition(int x, int y){
  int newPosX = (x-currentPositions[X])*STEPS_PER_MM*X_DIR_FLAG*SPACE_BETWEEN_POSITIONS_X;
  int newPosY = (y-currentPositions[Y])*STEPS_PER_MM*Y_DIR_FLAG*SPACE_BETWEEN_POSITIONS_Y;
  stepperX.move(newPosX);
  stepperY.move(newPosY);
  currentPositions[X] = x;
  currentPositions[Y] = y;
  Serial.print("Stepper positions: "); Serial.print(currentPositions[X]); Serial.print(", "); Serial.println(currentPositions[Y]);
}

void init_file(){
  if(SD.exists("pixelImg.aif")){
    SD.remove("pixelImg.aif");
  }

  //Data in MSB format
  uint16_t image_w = PIXELS_W;
  uint16_t image_h = PIXELS_H;
  image = SD.open("pixelImg.aif", FILE_WRITE);
  image.write((0xFF00 & image_w) >> 8);
  image.write(0xFF & image_w);
  image.write((0xFF00 & image_h) >> 8);
  image.write(0xFF & image_h);
  image.write(3); //bit-depth of 3 bytes (24-bits)
}

void init_apds(){
  if ( apds.init() ) {
    Serial.println(F("APDS-9960 initialization complete"));
  } else {
    Serial.println(F("Something went wrong during APDS-9960 init!"));
  }
  
  // Start running the APDS-9960 light sensor (no interrupts)
  if ( apds.enableLightSensor(false) ) {
    Serial.println(F("Light sensor is now running"));
  } else {
    Serial.println(F("Something went wrong during light sensor init!"));
  }
  
  // Wait for initialization and calibration to finish
  delay(500);
}

void init_steppers(){
  stepperX.begin(RPM);
  stepperX.setEnableActiveState(LOW);
  stepperX.enable();
  stepperX.setMicrostep(MICROSTEPS);
  stepperY.begin(RPM);
  stepperY.setEnableActiveState(LOW);
  stepperY.enable();
  stepperY.setMicrostep(MICROSTEPS);
}

Python Image Parser

from PIL import Image, ImageDraw

with open("PIXELIMG.AIF", 'rb') as imgFile:
    img_w, img_h = 0, 0
    img_w = int.from_bytes(imgFile.read(2), byteorder='big', signed = False)
    print(img_w)
    img_h = int.from_bytes(imgFile.read(2), byteorder='big', signed = False)
    if int.from_bytes(imgFile.read(1), byteorder='big', signed = False) == 3:
        img = Image.new("RGB", (img_w, img_h))
        for row in range(img_h):
            for col in range(img_w):
                r, g, b = int.from_bytes(imgFile.read(1), byteorder='big', signed = False), int.from_bytes(imgFile.read(1), byteorder='big', signed = False), int.from_bytes(imgFile.read(1), byteorder='big', signed = False)
                img.putpixel((col, row), (r, g, b))
        img.show()
        img.save("finished_img.bmp")

(This article copied from hackster.io, Author: Arduino “having11” Guy)