IMPORTANT:

This TRD requires the 2017.2 Vivado version that is bundled with the 2017.2 SDSoC tools. The stock 2017.2 Vivado version will result in an error, see Section 6.2 Limitations for more information.



1 Revision History


This wiki page complements the 2017.2 version of the Base TRD. For other versions, refer to the Zynq UltraScale+ MPSoC Base TRD overview page.

Change Log:
rev2:
  • Fixed DM6 Vivado project by adding missing local IPs and constraints files
rev1:
  • Update all projects, IPs, and tools versions to 2017.2
  • Update to 2017.2 xfOpenCV libraries version
  • Use dsa for hardware platform
  • Use data flow for xf::Mat top level arguments in xfOpenCV functions
  • Move xfOpenCV libraries from sample to platform includes
  • Add MIPI CSI-2 RX capture pipeline with support for Sony IMX274 image sensor
  • Various fixes and clean-up



2 Overview


The Zynq UltraScale+ MPSoC Base Targeted Reference Design (TRD) is an embedded video processing application running on a combination of APU (SMP Linux), RPU (bare-metal) and PL.

The design supports the following video interfaces:
  • Sources:
    • Virtual video device (vivid) implemented purely in software
    • USB webcam connected to the PS (optional)
    • Test pattern generator (TPG) implemented in the PL
    • HDMI Rx capture pipeline implemented in the PL
    • MIPI CSI-2 Rx based image sensor pipeline implemented in PL + FMC
  • Sinks:
    • DP Tx display pipeline in the PS
    • HDMI Tx display pipeline implemented in the PL

The following processing accelerators are implemented as memory-to-memory pipelines in the PL:
  • 2D-Convolution filter with programmable coefficients
  • Dense optical flow algorithm

The TRD demonstrates the value of offloading computation intensive tasks from the PS onto PL, thereby freeing APU resources. The APU load is plotted on the GUI to compare a pure software vs hardware accelerated implementation.
The RPU is used to monitor the live memory throughput of the design by reading the built-in AXI performance monitors (APM) inside the PS. The data is sent to the APU via the OpenAMP communication framework and plotted on the GUI.

This wiki contains information about:
  • How to setup the ZCU102 evaluation board and run the reference design.
  • How to build all the TRD components based on the provided source files via detailed step-by-step tutorials.

Additional material that is not hosted on the wiki:
  • Zynq UltraScale+ MPSoC Base TRD user guide UG1221: contains information about system, software and hardware architecture.



3 Software Tools and System Requirements


3.1 Hardware


Required:
  • ZCU102 evaluation board
    • rev 1.0 with ES2 silicon or
    • rev 1.0 or rev D2 with production silicon
  • Monitor with DisplayPort or HDMI input supporting one of the following resolutions:
    • 3840x2160 or
    • 1920x1080 or
    • 1280x720
  • Display Port cable (DP certified) or HDMI cable
  • Micro-USB cable, connected to laptop or desktop for the terminal emulator
  • Xilinx USB3 micro-B adapter
    • adapter shipped with ZCU102 rev 1.0 + production silicon
    • adapter needs to be purchased separately for ZCU102 rev 1.0 + ES2 silicon or rev D2 with production silicon
  • USB mouse
  • SD card

Optional:
  • HDMI video source with output supporting one of the following resolutions:
    • 3840x2160 or
    • 1920x1080 or
    • 1280x720
  • USB webcam
  • USB 3.0 hub (supplied with ZCU102 kit)
  • Leopard LI-IMX274MIPI-FMC (only supported on rev 1.0 boards)

3.2 Compatibility


The reference design has been tested successfully with the following user-supplied components.

Monitors:
Make/Model
Native Resolution
Viewsonic VP2780-4K
3840x2160 (60/30Hz)
Acer S277HK
3840x2160 (60/30Hz)
Dell P2415Q
3840x2160 (30Hz)
Dell U2414H
1920x1080 (60Hz)
GeChic On-Lap1303H
1920x1080 (60Hz)

HDMI Sources:
Make/Model
Resolutions
Nvidia Shield TV
3840x2160, 1920x1080
OTT TV BOX M8N
3840x2160, 1920x1080, 1280x720
Roku 2 XS
1920x1080, 1280x720
TVix Slim S1 Multimedia Player
1920x1080, 1280x720

USB Webcams:
Make/Model
Supported Resolutions
Supported Formats
Logitech HD Pro Webcam C920
1920x1080 (5fps), 1280x720 (10fps)
YUYV
Logitech HD Webcam C525
1920x1080 (5fps), 1280x720 (10fps)
YUYV

DisplayPort Cables:
  • Cable Matters DisplayPort Cable-E342987
  • Monster Advanced DisplayPort Cable-E194698

Storage Devices:
  • Crucial BX200 2.5in SATA SSD 240GB
  • San Disk UltraFit USB3.0 Flash Drive 16 GB

3.3 Software


Required:

3.4 Licensing


  • Important: Certain material in this reference design is separately licensed by third parties and may be subject to the GNU General Public License version 2, the GNU Lesser General License version 2.1, or other licenses.
    The Third Party Library Sources zip file provides a copy of separately licensed material that is not included in the reference design.
  • You will need only the SDSoC license to build the design which includes all the required IP licenses. You can evaluate for 60-days or purchase it here.

Steps to generate the license:
  1. Log in here with your work E-mail address (If you do not yet have an account, follow the steps under Create Account)
  2. Generate a license from “Create New Licenses” by checking "SDSoC Environment, 60 Day Evaluation License"
    license.png
  3. Under system information, give the host details.
  4. Proceed until you get the license agreement and accept it.
  5. The License (.lic file) will be sent to the email-id mentioned in the login details.
  6. Copy the license file locally and give the same path in the SDSoC license manager.



4 Design Files


4.1 Design Modules


The reference design is split into 10 design modules DM1 to DM10:
  • DM1 – APU SMP Linux
  • DM2 – RPU0 FreeRTOS Application
  • DM3 – RPU1 Bare-metal Application
  • DM4 – APU/RPU1 Inter Process Communication
  • DM5 – APU Qt Application
  • DM6 – PL Video Capture
  • DM7 – OpenCV-based Image Processing
  • DM8 – PL-accelerated Image Processing
  • DM9 – Two Image Processing Functions
  • DM10 – Full-fledged Base TRD

Each module is described in more detail on the respective tutorial page (see below).

The following table shows the dependency matrix between different modules. For example: DM6 (row) depends on or builds on top of modules DM1 and DM5 (columns).


DM1
DM2
DM3
DM4
DM5
DM6
DM7
DM8
DM9
DM1









DM2









DM3









DM4
+

+






DM5
+








DM6
+



+




DM7
+



+
+



DM8
+



+
+
+


DM9
+



+
+
+
+

DM10
+
+
+
+
+
+
+
+
+

4.2 Design Components


The top-level directory structure shows the the major design components organized by execution unit (APU, RPU0, RPU1). A pre-built SD card image is provide for DM10 along with a basic README and legal notice file.
zcu102-[es2-]base-trd-2017.2
├── apu
│   ├── perfapm-client
│   │   ├── perfapm-client
│   │   └── perfapm-client-test
│   ├── petalinux_bsp
│   ├── sdsoc_pfm
│   │   └── zcu102_[es2_]base_trd
│   │       └── samples
│   │           ├── filter2d
│   │           └── filter2d_optflow
│   └── video_app
│       ├── video_lib
│       └── video_qt2
├── images
│   └── dm10
├── IMPORTANT_NOTICE_CONCERNING_THIRD_PARTY_CONTENT
├── README
├── pl
│   └── zcu102_[es2_]base_trd
├── rpu0
│   └── heartbeat
│       ├── heartbeat
│       ├── heartbeat_bsp
│       └── zcu102_base_trd_wrapper_hw_platform_0
└── rpu1
    └── perfapm-server
        ├── perfapm
        ├── perfapm_bsp
        ├── perfapm-ctl
        ├── perfapm-server
        └── zcu102_base_trd_wrapper_hw_platform_0

The below figure shows the relevant design components for DM10 as well as inter-dependencies and generated output products.

project_dep_2017.1.png

The below table shows which design components are used in which design modules. A graphical view for each design module is provided on the respective design module tutorial page.

Design Component
Design Module
DM1
DM2
DM3
DM4
DM5
DM6
DM7
DM8
DM9
DM10
apu/perfapm-client/perfapm-client



Y





Y
apu/perfapm-client/perfapm-client-test



Y






apu/petalinux_bsp
Y


Y
Y
Y
Y
Y
Y
Y
apu/videao_app/video_lib




Y
Y
Y
Y
Y
Y
apu/videao_app/video_qt2




Y
Y
Y
Y
Y
Y
apu/zcu102_[es2_]base_trd/samples/filter2d






Y
Y


apu/zcu102_[es2_]base_trd/samples/filter2d_optflow








Y
Y
pl/zcu102_[es2_]base_trd





Y




rpu0/heartbeat

Y







Y
rpu1/perfapm-server/perfapm


Y
Y





Y
rpu1/perfapm-server/perfapm-ctl


Y







rpu1/perfapm-server/perfapm-server



Y





Y



5 Tutorials


5.1 Board Setup


Required:
  • Connect power supply to 12V power connector.
  • Connect USB mouse via USB hub to the USB3 micro-AB connector.
  • Display
    • Connect a DisplayPort cable to DisplayPort connector on the board; connect the other end to a monitor OR
    • Connect an HDMI cable to HDMI Tx connector (top) on the board; connect the other end to a monitor
    Note: Certain monitors have multiple HDMI ports supporting different HDMI standards. Make sure you choose an HDMI 2.0 capable port (if available) for 4k60 performance
    Note: Make sure you only connect either DP or HDMI Tx on the board, not both, otherwise the design might malfunction
  • Connect micro-USB cable to the USB-UART connector; use the following settings for your terminal emulator:
    • Baud Rate: 115200
    • Data: 8 bit
    • Parity: None
    • Stop: 1 bit
    • Flow Control: None
  • Insert SD card (FAT formatted) with binaries copied from $TRD_HOME/images/dm10 directory.

Optional:
  • Connect a USB webcam to the USB hub.
    Note: The USB webcam needs to output YUYV pixel format. Other formats are not supported in this design.
  • Connect an HDMI cable to HDMI Rx connector (bottom) on the board; connect the other end to an HDMI source
  • Connect the LI-IMX274MIPI-FMC module to the HPC0 FMC connector on the board
    Note: The design only supports this FMC on rev 1.0 boards. Vadj needs to be set to 1.2V for correct operation of the daughter card. If the FMC card does not seem functional, please follow the instructions explained in Answer Record AR67308 to check and/or set Vadj.

Jumpers & Switches:
  • Set boot mode to SD card:
    • Rev 1.0: SW6[4:1] - off,off,off, on
    • Rev D2: SW6[4:1] - on, off on, off
  • Configure USB jumpers for host mode
    • J110: 2-3
    • J109: 1-2
    • J112: 2-3
    • J7: 1-2
    • J113: 1-2

board_setup_btrd_2017.2.png

To run the pre-built SD card image for design module 10, follow the instructions here .

5.2 Build and Run Flow


The following tutorials assume that the $TRD_HOME environment variable has been set as below.

For rev 1.0 or rev D2 with production silicon:
% export TRD_HOME=</path/to/downloaded/zip-file>/rdf0421-zcu102-base-trd-2017-2
For rev 1.0 with ES2 silicon:
% export TRD_HOME=</path/to/downloaded/zip-file>/rdf0429-zcu102-es2-base-trd-2017-2

For some modules, the $PETALINUX environment variables needs to be set as well. This is done automatically when you source the PetaLinux settings.sh script (see PetaLinux installation guide).

For the individual tutorials, follow the links below:



6 Other Information


6.1 Known Issues


  • SDSoC accelerator code runs very slow in pure software implementation when Debug configuration is used
    • Frequency: Always
    • Workaround: Set project build configurations to Release which sets sdsoc compiler to optimize most (-O3)
  • HDMI Tx:
    • Certain monitors link up in HDMI 1.4 mode by default instead of HDMI 2.0 which limits the frame rate to 30 fps at a resolution of 3840x2160. Look for a similar message in the dmesg boot log: dp159_program(rate = 2970000000) for HDMI 1.4
      • Frequency: Common
      • Workaround: Connect a DP and HDMI cable to your monitor and select DP input. Power cycle the board. The application will start up up targeting DP Tx. Exit the application, unplug the DP cable from the board, and switch to the HDMI port on your monitor. You should see a blue HDMI standby screen. Run the run_video.sh command from the serial terminal. The application will now start up targeting HDMI Tx. Look for a similar message in the dmesg boot log to verify HDMI 2.0 mode: dp159_program(rate = 5332800000) for HDMI 2.0
    • Certain monitors don't detect a signal when operated in HDMI 2.0 mode and targeting a resolution of 3840x2160. Look for a similar message in the dmesg boot log: dp159_program(rate =5940000000) for HDMI 2.0
      • Frequency: Common
      • Workaround: None, this will be fixed in a future release. Please refer to the compatibility matrix for known working hardware.
    • After exiting the application, the blue HDMI standby screen does not come up and the screen stays dark when re-starting the application.
      • Frequency: Common
      • Workaround: Start the application using a different resolution e.g. run_video.sh -r 1920x1080. Exit the application and start again with the desired resolution.
  • Leopard LI-IMX274MIPI-FMC:
    • The design currently does not auto-detect if the Leopard IMX274 MIPI FMC is plugged in and the corresponding CSI video source is always listed in the selection menu. If the FMC card is not present, the following error message will appear when selecting CSI as video source: write to sensor failed: input/output error
  • Accelerator frame rate drops if HDMI Tx 4K60 is selected. The Filter2d frame rate maxes out at 4K 30 fps and the optical flow frame rate maxes out at 4K 35 fps
    • Frequency: Always
    • Workaround: None

6.2 Limitations


  • The application only supports the following display resolutions: 3840x2160, 1920x1080 and 1280x720.
  • The application does not support audio.
  • USB webcams:
    • Need to support the YUYV pixel format.
    • Most webcams don't support resolutions > 1080p using uncompressed video. Make sure you start the GUI in 1080p or 720p mode using the -r switch if you want to use your webcam as video source.
    • For USB2 webcams, it is expected to see increased latency and low frame rate.
  • Do not connect a DisplayPort cable and HDMI Tx at the same time.
  • Make sure the DisplayPort or HDMI Tx cable is plugged in when you power on the board.
  • DP-to-HDMI adapters are not supported, see AR 67462
  • HDMI Rx:
    • Does not support YUV 4:2:0 input.
    • Does not support HDCP encrypted input.
    • Does not support hotplug or dynamic resolution changes while the application is running.
  • Leopard LI-IMX274MIPI-FMC:
    • The Leopard IMX274 MIPI FMC is only supported on rev 1.0 boards. If used on a rev D2 board the following error message will appear when selecting CSI as video source: write to sensor failed: input/output error
  • Vivado 2017.2:
    • The design is not compatible with Vivado 2017.2. Instead, the Vivado version bundled with SDx 2017.2 must be used or you will see the following message design was created in a ‘future version of Vivado’ and the build will fail.



7 Support


To obtain technical support for this reference design, go to the:
  • Xilinx Answers Database to locate answers to known issues
  • Xilinx Community Forums to ask questions or discuss technical details and issues. Please make sure to browse the existing topics first before filing a new topic. If you do file a new topic, make sure it is filed in the sub-forum that best describes your issue or question e.g. Embedded Linux for any Linux related questions. Please include "ZCU102 Base TRD" and the release version in the topic name along with a brief summary of the issue.