1 Revision History


This wiki page complements the 2016.3 version of the Zynq UltraScale MPSoC Embedded Vision SDSoC Platform, specifically addressing use of the SDx tool with Xilinx's zcu102 board as an Embedded Vision Platform. This platform is a version of the Zynq UltraScale MPSoC Base TRD, tailored for embedded vision applications. For versions of the Zynq UltraScale MPSoC Base TRD, refer to the Zynq UltraScale+ MPSoC Base TRD overview page. For other versions of this design, refer to the Zynq UltraScale MPSoC Embedded Vision SDSoC Platform overview page.

Change Log:
  • Initial version of the Embedded Vision Platform for 2016.3 SDx
  • Includes 2D Filter and Optical Flow sample applications



2 Overview


The Zynq UltraScale+ MPSoC Embedded Vision SDSoC Platform is a reference design running on a combination of APU (SMP Linux), RPU (bare-metal) and PL.

The design consists of the following video data paths:
  • Two video capture pipelines:
    • one capturing video from a test pattern generator (TPG) implemented inside the PL
    • the other capturing video from an external HDMI source via FMC daughter card (optional)
  • A memory-to-memory processing pipeline implementing a 2D-convolution filter with programmable coefficients, or alternatively, an optical flow processing pipeline.
  • A simple command line based user interface operating over a serial terminal emulator, such as Teraterm.

The design demonstrates the value of offloading computation intensive tasks like the 2D-convolution filter, or the optical flow algorithm, from the PS onto PL, thereby freeing APU resources and accelerating execution.

This wiki contains information about:
  • How to set up the ZCU102 evaluation board and run the reference design.
  • How to build the sample applications based on the provided source files.

Additional material that is not hosted on the wiki:

Note: Certain material in this reference design is separately licensed by third parties and may be subject to the GNU General Public License version 2, the GNU Lesser General License version 2.1, or other licenses.



3 Software Tools and System Requirements


3.1 Hardware


Required:
  • ZCU102 evaluation board (rev D or newer) / power cable
  • Monitor with DisplayPort input supporting one of the following resolutions:
    • 3840x2160 or
    • 1920x1080 or
    • 1280x720
  • Display Port cable (DP certified)
  • micro-USB cable, connected to laptop or desktop for the terminal emulator.
  • SD card

Optional:
  • Avnet FMC-HDMI-CAM daughter card / HDMI cable
  • HDMI video source with input resolution set to:
    • 1080p60 for 1080p60 display or
    • 720p60 for 720p60 display

3.2 Software Tools


Required:

3.3 Licensing


You will need only the SDSoC license to build the design. You can evaluate for 60-days or purchase it here.

Steps to generate the license:
  1. Log in here with your work E-mail address (If you do not yet have an account, follow the steps under Create Account)
  2. Generate a license from “Create New Licenses” by checking "SDSoC Environment, 60 Day Evaluation License"
license.png

  1. Under system information, give the host details.
  2. Proceed until you get the license agreement and accept it.
  3. The License (.lic file) will be sent to the email-id mentioned in the login details.
  4. Copy the license file locally and give the same path in the SDSoC license manager.

3.4 Compatibility


The reference design has been tested successfully with the following user-supplied components.

DisplayPort Monitor:
Make/Model
Native Resolution
Viewsonic VP2780-4K
3840x2160 (30Hz)
LG 27MU67-B
3840x2160 (30Hz)
Acer S277HK
3840x2160 (30Hz)
Dell U2414H
1920x1080 (60Hz)
GeChic On-Lap1303H
1920x1080 (60Hz)

DisplayPort Cable:
  • Cable Matters DisplayPort Cable-E342987
  • Monster Advanced DisplayPort Cable-E194698

HDMI Source:
Make/Model
Resolutions
Roku 2 XS
1080p60, 720p60
TVix Slim S1 Multimedia Player
1080p60, 1080p24



4 Design File Hierarchy


The Zynq UltraScale+ MPSoC Embedded Vision Platform zip file is released with the binary and source files required to create Xilinx SDx projects and build the sample applications. SD card images are included, with everything pre-built, that enable the user to run the video demonstration and software applications on the zcu102 board, running under the Linux operating system. It includes all binaries necessary to configure and boot the ZCU102 evaluation board. The top-level directory structure is shown in the figure below.

file_hier.png



5 Installation and Operating Instructions


5.1 Board Setup


Required:
  • Connect power supply to J52.
  • Connect DisplayPort cable to P11; connect other end to monitor.
  • Insert SD card (FAT formatted) with binaries matching your board and silicon rev copied from one of the following directories:
    • For rev D with ES1 silicon, 2D filter: zcu102-ev-2016-3/sd_card/es1/filter2d
    • For rev D with ES1 silicon, optical flow: zcu102-ev-2016-3/sd_card/es1/optical_flow
    • For rev 1.0 with ES2 silicon, 2D filter: zcu102-ev-2016-3/sd_card/es2/filter2d
    • For rev 1.0 with ES2 silicon, optical flow: zcu102-ev-2016-3/sd_card/es2/optical_flow
  • Connect micro-USB cable to J83 USB UART connector; use the following settings for your terminal emulator:
    • Baud Rate: 11520
    • Data: 8 bit
    • Parity: None
    • Stop: 1 bit
    • Flow Control: None
Optional:
  • Connect the FMC-HDMI-CAM daughter card to the HPC0 FMC slot.
  • Connect the HDMI cable to HDMI IN port of FMC-HDMI-CAM; connect the other end to HDMI source.
  • Vadj needs to be set to 1.8V (default) for correct operation of the daughter card. To do this, please follow the instructions explained in Answer Record AR 67127
    The steps required are listed in section 7.1 below.

Jumpers & Switches:
  • Set boot mode according to your board and silicon revision
    • For rev B/C/D with ES1 silicon: SW6[4:1] - on,off,on,off
    • For rev 1.0 with ES2 silicon: SW6[4:1] - on,off,off,off

board_setup_es2_ev.png

5.2 Run the Application


Tutorial - run the 2D Filter sample application:
  • Copy all files from the release package directory ./zcu102-ev-2016-3/sd_card/es-/filter2d onto your SD card and insert it into the SD card slot on the zcu102 board.
  • Power on the board; make sure INIT_B, done and all power rail LEDs are lit green.
  • Two terminal sessions should be started on different COM ports, each using the settings mentioned above under Board Setup. With the USB-UART cable connected and the board powered up, you can locate two COM ports that are responsive.
    - APU: Linux Boot and Debug messages, and the linux command line prompt.
    - System Controller: controls power to the FMC connector.
  • During boot-up you should see a series of status messages appear on the APU terminal. After ~20 seconds, the display will turn on and you will be prompted to log in. In response to both "login:" and "Password:", type "root" :
Xilinx-ZCU102-2016_3 login: root
Password: root
 
  • Now use the run_filter2d.sh shell script to run the application. It allows you to select the resolution you wish to run :
# run_filter2d.sh
 
***********************************
*     Zynq UltraScale+ MPSOC      *
* Embedded Vision Platform 2016.3 *
*     2D Filter Application       *
***********************************
 
Select video resolution:
 
1 --> 3840x2160
2 --> 1920x1080
3 --> 1280x720
  • Select "1", and the test pattern appears on the DP monitor, at resolution 3840x2160, and the console displays the command-line menu:
Video Control application:
------------------------
DRM module name: xilinx_drm
HDMI output resolution: 3840x2160
 
--------------- Select Video Source ---------------
1 : Test Pattern Generator  (*)
2 : HDMI Input
 
--------------- Select Filter Type ----------------
3 : 2D Filter  (*)
 
--------------- Toggle Filter Mode ----------------
4 : Filter OFF/ON  (OFF)
 
--------------- Exit Application ------------------
0 : Exit
 
 
Enter your choice :
  • Activate the HW accelerated 2D edge-detection filter with command "4"
Enter your choice : 4
 
 
--------------- Select Video Source ---------------
1 : Test Pattern Generator  (*)
2 : HDMI Input
 
--------------- Select Filter Type ----------------
3 : 2D Filter  (*)
 
--------------- Toggle Filter Mode ----------------
4 : Filter OFF/ON  (ON)
 
--------------- Exit Application ------------------
0 : Exit
 
 
Enter your choice :
  • The 3840x2160 60hz test pattern, processed in real time by the 2D edge detection filter, appears on the display.
  • Repeating the command "4" turns off the processing.
  • Exit the app with command "0".

  • Restart the app at 1080p resolution using the run_filter2d.sh shell script and selecting "2" :
  • The test pattern appears at 1920x1080 60hz resolution.
  • As before, you may toggle the filter on/off with command "4".
  • If you have connected the optional FMC-HDMI-CAM daughter card and plugged in a 1080p 60hz HDMI source to its input plug, you may switch to that input with command "2".
  • Toggle the filter on/off with command "4".

  • If your HDMI source is 720p you may run the app at that resolution using the run_filter2d.sh shell script and selecting "3".

Run the Optical Flow sample application:
  • To run the Optical Flow sample application, power off the board, copy all files from the release package directory ./zcu102-ev-2016-3/sd_card/es-/optical_flow onto your SD card and insert it into the SD card slot on the zcu102 board.
  • Repeat the above tutorial steps.
  • Note that you are restricted to 1920x1080 and 1280x720 formats when running the optical_flow app. The default 3840x2160 resolution is not supported by the optical_flow sample app. The shell script for optical flow allows only the valid resolutions.
# run_optical_flow.sh
 
***********************************
*     Zynq UltraScale+ MPSOC      *
* Embedded Vision Platform 2016.3 *
*     Optical Flow Application    *
***********************************
 
Select video resolution:
 
1 --> 1920x1080
2 --> 1280x720
 
  • When Optical Flow processing is activated, the output shows bright colors where there is the greatest motion from one input frame to the next, and black where there is no motion. The optical flow algorithm returns 2 signed numbers at each pixel position, representing up or down motion in the vertical direction, and left or right motion in the horizontal direction. The brightness of the output, from black up to bright color, indicates the magnitude of the motion, and the color indicates the direction. +/- vertical motion is mapped onto the V color component, and +/- horizontal motion is mapped onto the U color component. To see a nice graph of the range of colors this produces, refer to the wikipedia page on YUV colors.




6 Tool Flow Tutorials


Build the 2D Filter sample application:
Firstly, the SDx Development Environment, version 2016.3, must be installed and working on your host computer, either the Linux or the Windows version. Further, it is assumed that you have already downloaded the Zynq UltraScale+ MPSoC Embedded Vision Platform zip file and extracted its contents. Wherever the zip file has been placed in your file system, the top of the extracted design file hierarchy is called zcu102-ev-2016-3. This directory is where the SDx workspace should be created.

These steps are virtually identical whether you are running the Linux or Windows version of SDx 2016.3.

  • Run SDx
  • Create your workspace in ./zcu102-ev-2016-3
  • From the menu, select File → New →Xilinx SDx Project, this opens the New Project dialog box, first is the "Create a New SDx Project" page.
  • Fill in Project name: filter2d, leave the "Use default location" box checked, hit Next>, this opens the "Choose Hardware Platform" page.
  • Select the platform. The very first time you do this, you must hit Add Custom Platform..., and browse to ./zcu102-ev-2016-3/zcu102_es1_ev, hit OK, note that "zcu102_es1_ev (custom)" now appears in the Name column.
  • Select your newly added custom platform "zcu102_es1_ev (custom)", hit Next>, this opens the "Choose Software Platform and Target CPU" page.
  • Leave everything as is, hit Next>, this opens the "Templates" page.
  • Select 2D Filter, hit Finish
  • The dialog box closes, and you now see the SDx Project Settings pane in the center of the SDx GUI. Locate the "Active build configuration:" in the upper right corner of the pane, which says "Debug" - click it and select Release. Your window should now look something like this:
SDx_project_settings.png

  • In the left hand "Project Explorer" pane, select the filter2d project, click right on it, then select Build Project. In the small Build Project dialog that opens, you may hit the "Run in Background" button. That causes the small dialog box to disappear, though you can still see a progress icon in the lower right part of the GUI, showing that work is in progress. Select the Console tab in the lower central pane of the GUI to observe the steps of the build process as it progresses. A number of warnings appear, of a benign nature, such as "unreferenced label", highlighted with a pale color. Any actual errors that occur would be highlighted with a brighter color and would cause the build process to stop. The build process may take tens of minutes, up to several hours, depending on the power of your host machine, whether you are running on Linux or Windows, and of course the complexity of your design. By far the most time is spent processing the routines that have been tagged for realization in hardware - note the "HW functions" window in the lower part of the SDx Project Settings pane. In our example above, the routine filter2d_sds is tagged to be built in hardware. The synthesis of the C code found in filter2d_sds.cpp into RTL, and the Placement and Routing of that RTL into the programmable logic in the Zynq MPSoC, are the steps that take most of the time.
  • Once the Build completes, you will find an sd_card directory has been created, with all the files you need to copy to your SD card and run your application on the zcu102 board, as explained in section 5.2 above. This directory is found in:
    • .\zcu102-ev-2016-3\filter2d\Release\sd_card

Build the Optical Flow sample application:
  • The Optical Flow project may be created and built in the same way just explained for the 2D Filter project. All the steps are analogous. For the Project name, use optical_flow.
  • Several projects may reside in the same workspace. Notice in the Project Explorer pane shown above, three projects are present: filter2d, optical_flow, and stereo_vision.



7 Other Information


7.1 Known Issues


  • The board doesn't boot and the INIT_B LED stays red.
    • Frequency: Rare
    • Workaround: Power cycle the board
  • SDSoC accelerator code runs very slowly in pure software implementation when Debug configuration is used
    • Frequency: Common
    • Workaround: Set project build configurations to Release which sets sdsoc compiler to optimize most (-O3)
  • Linux 'reboot' command not working
    • Frequency: Common
    • Workaround: Power cycle the board
  • HDMI input is not working, GUI displays error message FMC not connected
    • Frequency: Common
    • Workaround: Based on AR 67127
      • Connect a cable to the USB-UART port, open a terminal emulator and connect to COM3 of the quad-UART (use the same terminal settings as listed in 5.1) -> the MSP430 System Controller can be controlled through this serial terminal
      • Power up the board without the FMC card plugged in to the HPC0 slot -> VADJ will be set by the MSP430 System Controller to 1.8V (default)
      • Use the MSP430 System Controller to turn off VADJ -> the VADJ_FMC LED will turn off
      • Plug in the FMC card to the HPC0 slot
      • Use the MSP430 System Controller to set VADJ to 1.8V -> the VADJ_FMC LED will turn back on
      • Press the POR_B button to reset and reboot the system
  • When building the optical_flow sample app, you will see a number of warnings, such as: "WARNING: [SDSoC 0-0] No size pragma specified for pointer argument f0Stream". These can be ignored.

7.2 Limitations


  • The application only supports the following display resolutions: 3840x2160p30, 1920x1080p60, and 1280x720p60.
  • The application does not support audio.
  • HDMI input is only available via Avnet FMC-HDMI-CAM daughter card connected to the HPC0 slot; the on-board HDMI connectors are not used.
  • HDMI input only supports 1920x1080p60 and 1280x720p60 video resolutions and the resolution must match the display resolution.
  • HDMI input does not support HDCP content protected sources.
  • Make sure the DisplayPort cable is plugged in when you power on the board, otherwise the framebuffer console will default to 1024x768 resolution and the application will not start.
  • DP-to-HDMI adapters are not supported, see AR 67462



8 Support


To obtain technical support for this reference design, go to the:
  • Xilinx Answers Database to locate answers to known issues
  • Xilinx Community Forums to ask questions or discuss technical details and issues. Please make sure to browse the existing topics first before filing a new topic. If you do file a new topic, make sure it is filed in the sub-forum that best describes your issue or question e.g. Embedded Linux for any Linux related questions. Please include "ZCU102 Base TRD" and the release version in the topic name along with a brief summary of the issue.