Welcome! Here is the wiki of Vivado HLS Video Library.

Introduction


HLS Video Library is a C/C++ library provided with Vivado HLS to help accelerate computer vision/image processing appliations on FPGA. It includes commonly used data structures, OpenCV interfaces, AXI4-Stream I/O, and video processing functions. HLS Video Library uses OpenCV libraries as reference model, most video processing functions has the similar interface and equaivalent behavior with corresponding OpenCV functions. The pre-built OpenCV libraries (with FFmpeg support) are also shipped with Vivado HLS on different platforms, so users are able to use OpenCV directly without extra effort.

In short, there are two header files that can help users to start using HLS Video Library:
#include <hls_video.h>
include this file in top design to access all specific video data structures and image/video processing functions, they are all synthesizable. When using the synthesizable part of HLS Video Library, it is required to use the hls namespace or hls:: scoped.
#include <hls_opencv.h>
include this file in testbench to access pre-built OpenCV libraries in Vivado HLS and the its interface functions to HLS Video Library. Most of them are not synthesizable.

More information on HLS Video Library usage, please refer to XAPP1167[1].

More information on OpenCV installation, please refer to OpenCV Installation.

More information on Vivado HLS, please refer to UG902[2] .

Video Library Data Structures


There are some template classes provided HLS Video Library to representing the basic data structures like image, pixel, etc. Most of the naming and usage of them are similar to OpenCV, with specialization and optimization for FPGA. Also, the commonly used window buffer and line buffer are included, which allows users to manage the memory more effeciently.

Video Library Functions


OpenCV interface functions


In a typical video system using OpenCV functions, most of the algorithm will remain on the processor and remain using OpenCV functions. Only those parts of the algorithm which require acceleration in the FPGA fabric will be synthesized and therefore updated to use the HLS video library.
Since the AXI4 Streaming protocol is commonly used as the interface of video processing component, the OpenCV interface functions are provided to ensure the data transfer between the OpenCV code running on the processor and the synthesized hardware function running on FPGA fabric. Using the interface functions to transform the data before passing it to the function to be synthesized ensures a high-performance system. In addition to transforming the data, the functions also include the means of converting OpenCV data formats to and from the HLS Video Library data types, e.g. hls::Mat. To use the OpenCV interface functions, the header file hls_opencv.h must be included. These functions would be used in the code which remains on the processor.

AXI4-Stream I/O Functions


Most of the Video Library functions for accelerating video processing is using hls::Mat as the data type of an image. In synthesizable video processing component, the I/O functions which converting hls::Mat to/from AXI4-Stream compatible data type (hls::stream) are provided. To use the AXI4-Stream I/O functions, the header file hls_video.h must be included. These I/O functions should be invoked inside the hardware function, and they are synthesizable.

Video Processing Functions


The video processing functions provided in the HLS Video Library are specifically for manipulating video images. Most of these functions are designed for accelerating corresponding OpenCV functions, which have similar signature and usage[3] .

Roadmap & Feedback


What will be the next feature in HLS Video Library? Any specific functions interested? Found bugs?
We value your input. Reach us by:
  1. Discussion Area in wiki
  2. http://www.wiki.xilinx.com/Contact

Subject Author Replies Views Last Message
No Comments



References


  1. ^ XAPP1167: Accelerating OpenCV Applications with Zynq-7000 All Programmable SoC using Vivado HLS Video Libraries
  2. ^ UG902: Vivado Design Suite User Guide: High-Level Synthesis
  3. ^ OpenCV documentation