Histogram Equalization in Vivado HLS

In this article I will be explaining the Program for Histogram Equalization. This program is written for Xilinx FPGA’s using the Vivado HLS Software.

Vivado High Level Synthesis is used to write RTL code using C and C++, this code is then synthesized to an FPGA friendly VHDL or Verilog Program.

Furthermore, Vivado HLS also provides the OpenCV Image Processing functions as a part of it’s HLS Video Library hls_video.h .

Preparations before proceeding with the code –

Make sure that you have the Board Support files for the part for which you are developing this program. I will be working on the MYIR’s Z-Turn Board with Zynq XC7Z010.

So when using HLS for first time , I add the Board Support Files for MYIR’s Z – Turn Board from the Git Repository “ZTurn Stuff” .

Histogram Equalization in Python

In Python, the process of Histogram equalization is fairly simple as well as self-explanatory and requires no more than 5 lines of code.

import cv2 
img = cv2.imread(imageSource, 0)
equ = cv2.equalizeHist(img)
cv2.imwrite(imageDest, equ ) 

Histogram Equalization in Vivado HLS (using C++)

Following is the program for Histogram Equalization in C++

Note – This is not a conventional C++ program that uses Open CV, it is specifically designed to work on a Xilinx FPGA

core.hpp

#include  "hls_video.h"
#include <ap_fixed.h>
#define MAX_WIDTH  2000
#define MAX_HEIGHT 2000
typedef hls::stream<ap_axiu<32,1,1,1> >           AXI_STREAM;
typedef hls::Mat<MAX_HEIGHT,   MAX_WIDTH,   HLS_8UC3> RGB_IMAGE;
typedef hls::Mat<MAX_HEIGHT,   MAX_WIDTH,   HLS_8UC1> GRAY_IMAGE;

void getHistEq(AXI_STREAM& INPUT_STREAM, AXI_STREAM& OUTPUT_STREAM, int rows, int cols);

core.cpp

#include "core.hpp"
void getHistEq(AXI_STREAM& INPUT_STREAM, AXI_STREAM& OUTPUT_STREAM, int rows,int cols) {

#pragma HLS INTERFACE axis port=INPUT_STREAM
#pragma HLS INTERFACE axis port=OUTPUT_STREAM

	RGB_IMAGE img_main(rows, cols);

	GRAY_IMAGE img_gray(rows, cols);

	GRAY_IMAGE img_grayCopy1(rows, cols);
	GRAY_IMAGE img_grayCopy2(rows, cols);

	GRAY_IMAGE img_grayhisteq1(rows, cols);
	GRAY_IMAGE img_grayhisteq2(rows, cols);

	RGB_IMAGE img_fin(rows, cols);

#pragma HLS dataflow
	hls::AXIvideo2Mat(INPUT_STREAM, img_main);
	hls::CvtColor<HLS_BGR2GRAY>(img_main, img_gray);
	hls::Duplicate(img_gray, img_grayCopy1, img_grayCopy2);

	hls::EqualizeHist(img_grayCopy1,img_grayhisteq1);
	hls::EqualizeHist(img_grayCopy2,img_grayhisteq2);
	hls::CvtColor<HLS_GRAY2RGB>(img_grayhisteq2, img_fin);
	hls::Mat2AXIvideo(img_fin, OUTPUT_STREAM);

}

test_core.cpp

#include <hls_opencv.h>
#include "core.hpp"
#include <iostream>
using namespace std;
int main(int argc, char** argv) {
	IplImage* src;
	IplImage* dst;
	AXI_STREAM src_axi, dst_axi;
	src = cvLoadImage("D:/Blog/VivadoHLSProjects/ImageProcProj1/ImageAssets/testImage1.jpg");
	dst = cvCreateImage(cvGetSize(src), src->depth, src->nChannels);
	IplImage2AXIvideo(src, src_axi);
	getHistEq(src_axi, dst_axi, src->height, src->width);
	AXIvideo2IplImage(dst_axi, dst);
	cvSaveImage("D:/Blog/VivadoHLSProjects/ImageProcProj1/ImageAssets/testImage2.jpg", dst);
	cvReleaseImage(&src);
	cvReleaseImage(&dst);
}

As you can see, Programming a Histogram Equalization Code for an FPGA is a little more complex than its peer in Python . However, it can be broken down to small pieces to understand what is happening here!

Let’s begin with the core.hpp file.

#include  "hls_video.h"

As I said earlier hls_video is the HLS Library that provides all the Open CV Functions.

#include <ap_fixed.h>

ap_fixed is the HLS Library that provides Arbitrary Precision Data Types, a note on why you should use this can be found in the following HLS Documentation (pg 104)-

https://www.xilinx.com/support/documentation/sw_manuals/xilinx2014_2/ug871-vivado-high-level-synthesis-tutorial.pdf

The note says that –

When you revise C code to use arbitrary precision types instead of standard C types,one of the
most common changes you must make is to reduce the size of the data types. In this case, you
change the design to use 8-bit, 24-bit, and 18-bit words instead of 32-bit float types. This results
in smaller operators, reduced area, and fewer clock cycles to complete.

Moving on further,

#define MAX_WIDTH  2000
#define MAX_HEIGHT 2000

These macros just define the maximum possible width and height of the image that we will be working with.

typedef hls::stream<ap_axiu<32,1,1,1> >           AXI_STREAM;

To read an Image File on an FPGA we need to use an AXI Streaming Interface, the datatype of this stream is thus defined and given a name as AXI_STREAM by us here .

typedef hls::Mat<MAX_HEIGHT,   MAX_WIDTH,   HLS_8UC3> RGB_IMAGE;
typedef hls::Mat<MAX_HEIGHT,   MAX_WIDTH,   HLS_8UC1> GRAY_IMAGE;

Here, The DataType for an RGB Image is defined as hls::Mat<MAX_HEIGHT,   MAX_WIDTH,   HLS_8UC3> and for a GrayScale image it is defined as hls::Mat<MAX_HEIGHT,   MAX_WIDTH,   HLS_8UC1>, thus assigning the DataType names as RGB_IMAGE and GRAY_IMAGE respectively.

As you can see the type HLS_8UC3 is for RGB i.e. 3 Channel and HLS_8UC1 is for GrayScale i.e. 1 Channel

And finally, following is the function Prototype of the top level function of the core that we are going to synthesize.

void getHistEq(AXI_STREAM& INPUT_STREAM, AXI_STREAM& OUTPUT_STREAM, int rows, int cols);

Core.cpp File

Input Image is read by the FPGA program through INPUT_STREAM and the output Image is written through the OUTPUT_STREAM.

rows is the number of rows in the image matrix i.e. Height of the image in pixels andcolsis the number of columns in the image matrix i.e. Width of the image in pixels

Explanation of the Pragmas

HLS Specific Pragma Compiler directives are used to exercise some functionalities during the synthesis.

“The INTERFACE pragma specifies how RTL ports are created from the function definition during interface synthesis. “
(Source – https://www.xilinx.com/html_docs/xilinx2019_1/sdaccel_doc/hls-pragmas-okr1504034364623.html#jit1504034365862)

Thus, the following lines specify the creation of AXI Input port and the Output Port for INPUT_STREAM and OUTPUT_STREAM respectively.

#pragma HLS INTERFACE axis port=INPUT_STREAM
#pragma HLS INTERFACE axis port=OUTPUT_STREAM

“The DATAFLOW pragma enables task-level pipelining, allowing functions and loops to overlap in their operation, increasing the concurrency of the register transfer level (RTL) implementation, and increasing the overall throughput of the design.”
(Source – https://www.xilinx.com/html_docs/xilinx2019_1/sdaccel_doc/hls-pragmas-okr1504034364623.html#jit1504034365862)

#pragma HLS dataflow

Defining Variable

Variables of DataType RGB_IMAGE and GRAY_IMAGE are defined –

	RGB_IMAGE img_main(rows, cols);

	GRAY_IMAGE img_gray(rows, cols);

	GRAY_IMAGE img_grayCopy1(rows, cols);
	GRAY_IMAGE img_grayCopy2(rows, cols);

	GRAY_IMAGE img_grayhisteq1(rows, cols);
	GRAY_IMAGE img_grayhisteq2(rows, cols);

	RGB_IMAGE img_fin(rows, cols);

Conversion From AXI Stream , Processing the Image and Converting Back

 hls::AXIvideo2Mat(INPUT_STREAM, img_main);

The above line stores the RGB Image from AXI Stream to img_main

Converting the RGB Image to a GrayScale Image

hls::CvtColor<HLS_BGR2GRAY>(img_main, img_gray);

Note that each operation performed by the hls_video library consumes the Source Image, simply speaking the source image diappears that is the Source Image becomes null

The variable img_main thus becomes empty after the RGB to Grayscale conversion operation.

EqualizeHist the function of hls is meant to work in an efficient manner on a Video data, it equalizes the histogram of the current image frame based on the histogram calculated for the previous image frame. This is done by storing the Histogram data of the image in a static variable by that function.

But, since we only have a single image, the EqualizeHist function returns a black image i.e a matrix full of 0 ‘s because initially the histogram data in static variable is0

Therefore, to get the Histogram Equalized Image we apply the EqualizeHist function on the same image twice !

But remember that hls makes the source variable empty after an operation is performed on it, so you cannot use the same variable to EqualizeHist the second time otherwise it will throw an error. In order to tackle this I make two copies of the image –

  hls::Duplicate(img_gray, img_grayCopy1, img_grayCopy2);

The above line of code “consumes” img_gray and as an output stores this data in img_grayCopy1 as well as img_grayCopy1.

Now we call EqualizeHist on img_grayCopy1 which returns a blank image as an output but stores the histogram information in a static variable internally.

hls::EqualizeHist(img_grayCopy1,img_grayhisteq1);

Then we call EqualizeHist on img_grayCopy2 which uses the previously calculated Histogram Information and Equalizes our image, the output is then stored in img_grayhisteq2

  hls::EqualizeHist(img_grayCopy2,img_grayhisteq2);

The single channel img_grayhisteq2 image is then converted to a 3 channel image(RGB) to suit the type of AXI Stream

hls::CvtColor<HLS_GRAY2RGB>(img_grayhisteq2, img_fin);

Finally, we convert the output image img_fin back to the AXI Stream Format

hls::Mat2AXIvideo(img_fin, OUTPUT_STREAM);

Testbench file Test_Core.cpp

Test Bench files are used to test the functionality of our core programatically, these files aren’t synthesized but only used to test.

The testbench file is pretty easy to understand,

First we define the variables for source and destination image as well as the AXI Stream that we create for testing

	IplImage* src;
	IplImage* dst;
	AXI_STREAM src_axi, dst_axi;

Then we load the src image and also initialize the dest variable

	src = cvLoadImage("D:/Blog/VivadoHLSProjects/ImageProcProj1/ImageAssets/testImage1.jpg");
	dst = cvCreateImage(cvGetSize(src), src->depth, src->nChannels);

After that we convert the src image to AXI Stream, because that is the format that our Core understands

IplImage2AXIvideo(src, src_axi);

Finally, The toplevel function of the core is called, its output which is in AXI Format is converted back to image and this image is saved to a location.

 getHistEq(src_axi, dst_axi, src->height, src->width);
  AXIvideo2IplImage(dst_axi, dst);
  cvSaveImage("D:/Blog/VivadoHLSProjects/ImageProcProj1/ImageAssets/testImage2.jpg", dst);

Results

Test Bench Output
Source Image

0

Leave a Reply