Overview

The Linux Video Mixer driver is an early access DRM kernel driver designed to provide support for the Xilinx LogiCORE IP Video Mixer (v2.0). The Video Mixer is a configurable IP core than can blend up to eight video layers in addition to an optional logo layer into a single output video stream.

DRM_Impl_Diagram_v2.png


Location

The driver is currently located in a special branch of the standard Xilinx Linux kernel: https://github.com/Xilinx/linux-xlnx/tree/2017.3_video_ea

Supported IP Features

The following is a list of IP constraints for which there is support in the driver and for which verification within the context of the listed reference designs has been performed (see below).
Note that the Mixer (v1.0) driver was made available in a special branch for 2017.1 and 2017.2 and that Mixer v2.0 is only available in 2017.3 (wherein support for Mixer v1.0 has been removed):

IP Feature

2017.1/2017.2

2017.3

Output Stream
RGB, YUV422
RGB
Samples per Clock
2
Maximum Data Width
8
8,10
Maximum Number of Columns
3840
Maximum Number of Rows
2160
Number of Layers
8
Layer Video Formats
RGB8, RGBX8, YUYV8,
RGBA8, BGRA8, Y_UV8,
Y_UV8_420
[2017.1/2 list +] YUV8,
YUVX8, Y8,
UYVY8, BGRX8,
RGBX10, YUVX10,
RGB565, Y_UV10,
Y_UV10_420, Y10
Layer Alpha
Yes
Layer Scaling
Yes
Layer Interface
Memory
Logo Layer
Yes
Logo Layer per Pixel Alpha
Yes

Unsupported IP Features
The following list of IP constraints either has no driver support or has not yet been verified to work in any existing technical reference design:
1. Video Stream Output Formats: YUV 4:4:4, YUV 4:2:2, YUV 4:2:0
2. Maximum Data Width: 12, 16
3. Samples per Clock: 1, 4
4. Layer Video Formats: RGB565
5. Layer Interface Type: streaming
6. Logo Transparency Color

Known Issues

  • Only DRM Encoder drivers implementing the encoder slave interface are supported. The DRM bridge interface is not yet supported.

  • If the DRM primary plane uses a video format other than some form of RGB, the Linux Framebuffer Console driver will fail during boot (see more about specific DRM Implementation Overview below). This can be avoided in one of the following ways:
    • Configure the hardware design to output a video stream format of RGB and ensure the driver is configured accordingly via the device tree.
    • Disable the Framebuffer Console driver within the kernel (FRAMEBUFFER_CONSOLE)
    • Assign a layer of the mixer that is configured to read from some form of RGB in memory to be the DRM primary plane (see DRM Implementation Overview below)

Kernel Configuration

This version of the driver is an extension of the existing DRM driver. As such, the Xilinx DRM driver must be enabled: CONFIG_DRM_XILINX_XVMIXER=[y|m]

XVMixer_Kernel_Config.png


Device Tree Configuration
Comprehensive documentation may be found within the kernel branch under:
<linux_root>/ Documentation/devicetree/bindings/drm/xilinx/xilinx_video_mixer.txt

DRM Implementation Overview

The Linux driver is an implementation of the Direct Rendering Manager (DRM) framework. The Video Mixer IP is modeled as a DRM CRTC object and each Mixer layer (including the video output stream layer) is a DRM Plane. DRM Planes can assume one of three logical roles:
  • Primary: this serves as the mode-setting layer and determines the background screen image
  • Overlay: a layer designed to be superimposed over the primary layer
  • Cursor: a special layer designed to represent a screen pointer or equivalent
By default, the Mixer’s various layers will be mapped to the above DRM Plane roles as follows:
Mixer IP Layer
DRM Plane Type
Output stream layer
Primary
Layer 1-7
Overlay
Logo
Cursor
The default assignment of the layer serving as DRM Primary can be overridden in the device tree with the optional property xlnx,layer-primary. This property can be used to assign the role of “primary” to any one of the Mixer overlay layers present in the design. This property cannot be used on the logo layer. If present, the output stream layer will be assigned the role of “overlay” layer. As an overlay plane, the output stream layer will only respond to video format changes; its size will always correspond to the current size of the assigned primary layer.
There are several cases wherein assignment of the primary layer to a mixer overlay layer might be advantageous:
  • Your mixer instance is configured to stream out YUV 4:2:2 but you wish to present user space drivers with an ARGB memory interface. You could configure the Mixer with an overlay layer that accepts ARGB memory formatted data and assign this as the DRM primary layer. User space is able to write ARGB and conduct normal mode setting/resolution changes through this layer yet the final output, via the Mixer, will be transformed into YUV 4:2:2.
  • Your use case is such that using overlays superimposed over a background primary plane does not work. Instead, you wish the background layer to be the top layer in z-order position and utilize per-pixel alpha values to create open regions in the background through which content can be framed underneath. This might be done in the case where the primary plane is used to render video/audio controls and the video is displayed beneath this superimposed gui background. By assigning the primary plane to be the uppermost layer, the remaining layers can be used, effectively, as underlays.

From within Vivado, Mixer layers can be configured with the following optional properties:
  • layer alpha
  • layer scaling (1x, 2x or 4x)
These properties, if described in the device tree, will be represented as DRM plane properties. To understand more about these capabilities, please refer to the Video Mixer Product Guide [PG243].

Additionally, the Video Mixer supports generation of solid background color when either the AXI streaming input is not connected or the layer is otherwise disabled. On initialization, this color is programmed to default to blue. The color may be configured using a value representing packed RGB little-endian format via the DRM plane property bg_color. This property is attached to the primary plane.

Test Procedure


To verify the proper configuration and operation of the IP, a suitable hardware design will need to include at a minimum:
1) Video DMA IP to supply an input stream to the Mixer IP (layer 0) (e.g. VDMA or Framebuffer DMA)
2) Video Mixer IP
3) HDMI Tx

modetest

Modetest is a test tool which can be found as part of the libdrm suite of test tools. We will use this tool to ensure proper configuration and operation of the Mixer IP. Modetest can be used to activate overlay layers and alter layer properties (e.g. layer alpha, layer scaling, background color)

Test 1 - Ensure DRM driver has been properly loaded and is configured

root@mixer_proj:~# modetest -M xilinx_drm_mixer
Output should include information about the Encoder, Connector, CRTC (the Mixer), Planes (Mixer layers). All Mixer layers will be deactivate by invoking modetest so the screen should become a solid hue of blue (the default background color).

Sample output (edited for brevity and clarity):
Encoders:
id      crtc    type    possible crtcs  possible clones
36      35      TMDS    0x00000001      0xffffffff
 
Connectors:
id      encoder status          name            size (mm)       modes   encoders
37      36      connected       HDMI-A-1        520x290         40      36
  modes:
        name refresh (Hz) hdisp hss hse htot vdisp vss vse vtot)
  3840x2160 30 3840 4016 4104 4400 2160 2168 2178 2250 flags: phsync, pvsync; type: driver
  3840x2160 30 3840 4016 4104 4400 2160 2168 2178 2250 flags: phsync, pvsync; type: driver
  3840x2160 25 3840 4896 4984 5280 2160 2168 2178 2250 flags: phsync, pvsync; type: driver
 
  <snip>
 
  640x480 60 640 656 752 800 480 490 492 525 flags: nhsync, nvsync; type: driver
  720x400 70 720 738 846 900 400 412 414 449 flags: nhsync, pvsync; type: driver
  props:
        1 EDID:
                flags: immutable blob
                blobs:
 
                value:
                        00ffffffffffff004c2dd30c44534d30
                        131a010380341d782a1255a9544d9f25
                        0c5054bfef80714f810081c081809500
                        a9c0b300010108e80030f2705a80b058
                        8a0009252100001e000000fd00184b1e
                        873c000a202020202020000000fc0055
                        3234453539300a2020202020000000ff
                        00485450483530313132310a2020011b
                        020334f04d611203130420221f105f60
                        5d5e23090707830100006d030c002000
                        803c20106001020367d85dc401788003
                        e30f0104023a801871382d40582c4500
                        09252100001e023a80d072382d40102c
                        458009252100001e011d007251d01e20
                        6e28550009252100001e565e00a0a0a0
                        29503020350009252100001a000000a0
        2 DPMS:
                flags: enum
                enums: On=0 Standby=1 Suspend=2 Off=3
                value: 0
 
CRTCs:
id      fb      pos     size
35      44      (0,0)   (1024x768)
  1024x768 60 1024 1048 1184 1344 768 771 777 806 flags: nhsync, nvsync; type: driver
  props:
 
Planes:
id      crtc    fb      CRTC x,y        x,y     gamma size      possible crtcs
26      0       0       0,0             0,0     0               0x00000001
  formats: RA24
  props:
        5 type:
                flags: immutable enum
                enums: Overlay=0 Primary=1 Cursor=2
                value: 2
        23 scale:
                flags: range
                values: 0 2
                value: 0
        24 alpha:
                flags: range
                values: 0 256
                value: 256
27      0       0       0,0             0,0     0               0x00000001
  formats: YUYV
  props:
        5 type:
                flags: immutable enum
                enums: Overlay=0 Primary=1 Cursor=2
                value: 0
        23 scale:
                flags: range
                values: 0 2
                value: 0
        24 alpha:
                flags: range
                values: 0 256
                value: 256
28      0       0       0,0             0,0     0               0x00000001
  formats: UYVY
  props:
        5 type:
                flags: immutable enum
                enums: Overlay=0 Primary=1 Cursor=2
                value: 0
        23 scale:
                flags: range
                values: 0 2
                value: 0
        24 alpha:
                flags: range
                values: 0 256
                value: 256
29      0       0       0,0             0,0     0               0x00000001
  formats: AR24
  props:
        5 type:
                flags: immutable enum
                enums: Overlay=0 Primary=1 Cursor=2
                value: 0
        23 scale:
                flags: range
                values: 0 2
                value: 0
        24 alpha:
                flags: range
                values: 0 256
                value: 256
30      0       0       0,0             0,0     0               0x00000001
  formats: GREY
  props:
        5 type:
                flags: immutable enum
                enums: Overlay=0 Primary=1 Cursor=2
                value: 0
        23 scale:
                flags: range
                values: 0 2
                value: 0
        24 alpha:
                flags: range
                values: 0 256
                value: 256
31      0       0       0,0             0,0     0               0x00000001
  formats: XR24
  props:
        5 type:
                flags: immutable enum
                enums: Overlay=0 Primary=1 Cursor=2
                value: 0
        23 scale:
                flags: range
                values: 0 2
                value: 0
        24 alpha:
                flags: range
                values: 0 256
                value: 256
32      0       0       0,0             0,0     0               0x00000001
  formats: NV12
  props:
        5 type:
                flags: immutable enum
                enums: Overlay=0 Primary=1 Cursor=2
                value: 0
        23 scale:
                flags: range
                values: 0 2
                value: 0
        24 alpha:
                flags: range
                values: 0 256
                value: 256
33      0       0       0,0             0,0     0               0x00000001
  formats: BG24
  props:
        5 type:
                flags: immutable enum
                enums: Overlay=0 Primary=1 Cursor=2
                value: 0
        23 scale:
                flags: range
                values: 0 2
                value: 0
        24 alpha:
                flags: range
                values: 0 256
                value: 256
34      35      44      0,0             0,0     0               0x00000001
  formats: BG24
  props:
        5 type:
                flags: immutable enum
                enums: Overlay=0 Primary=1 Cursor=2
                value: 1
        25 bg_color:
                flags: range
                values: 0 16777215
                value: 16711680
 
Frame buffers:
id      size    pitch


Test 2 - Activate an overlay layer


We will activate an overlay plane (RGB in this case) and position it to the top left corner while the background color is being generated using the following command:
root@mixer_proj:~# modetest -M xilinx_drm_mixer -P 35:640x480+0+0@BG24
Output should indicate the plane id that was activated:
testing 640x480@BG24 overlay plane 33
Additionally, the plane should be presented with diagonally stripped color pattern on screen.


Test 3 - Scale the layer (if enabled for the layer)

From within another console window (and/or if the previous test was run in the background), adjust the layer scale property using modetest.
The plane id (33 in case of the example above) will be needed to adjust overlay properties like scale, alpha or background color
root@mixer_proj:~# modetest -M xilinx_drm_mixer -w 33:scale:1
Note that the range of possible values for a property appears in the output of modetest. For example, in the case of plane id 33:
33      0       0       0,0             0,0     0               0x00000001
  formats: BG24
  props:
        5 type:
                flags: immutable enum
                enums: Overlay=0 Primary=1 Cursor=2
                value: 0
        23 scale:
                flags: range
                values: 0 2
                value: 0
        24 alpha:
                flags: range
                values: 0 256
                value: 256
 
Assuming the up-scaled version of the plane image will fit within the screen, the plane data should be doubled in size by setting the scale property to '1'.

Test 4 - Change layer alpha (if enabled for the layer)

Changing the layer alpha will make an existing overlay layer appear more or less transparent. An alpha value of '0' will render the overlay invisible and a value of '256' will be completely opaque.
With an existing layer being displayed (see test 2), change the alpha property to '0' to render the layer invisible.
root@mixer_proj:~# modetest -M xilinx_drm_mixer -w 33:alpha:0
 
The layer should disappear.

Changing the alpha property back to 256 by repeating the above command with a value of 256 should render it visible again.
root@mixer_proj:~# modetest -M xilinx_drm_mixer -w 33:alpha:256

Test 5 - Change the background color

The Mixer generates a background color when the primary layer is inactive. By default, this is blue. The color is controlled by an internal RGB-based register and is represented by modetest as a decimal value.
The most significant bits represent 'blue' and the least 'red'. As such, by default, only the upper 8 bits are set to generate a solid blue (0xFF0000) resulting in a default value of 16711680.

Change the background color to solid red:
root@mixer_proj:~# modetest -M xilinx_drm_mixer -w 34:bg_color:255
The background color should be a pure red and the new value of the bg_color property will be 255 (0x0000FF).

Test 6 - Change the output resolution

To change Mixer output to a new resolution, modetest must be invoked with the connector id and new resolution. In this example, we change to output 1024x768:

root@mixer_proj:~# modetest -M xilinx_drm_mixer -s 37@35:1024x768@BG24
The output should be an SMPTE color bar pattern on the screen in the new resolution specified (note: an optional refresh rate can be added to the above command when multiple options are available via the monitor's EDID).
setting mode 1024x768-75Hz@BG24 on connectors 37, crtc 35
 

vbltest

vbltest is a test tool which is part of the libdrm suite of test tools. It is used to ensure vertical blanking interrupts are properly sent by the DRM driver.
root@mixer_proj:~# vbltest -M xilinx_drm_mixer
starting count: 0
freq: 60.49Hz
freq: 60.00Hz
freq: 60.00Hz
freq: 60.00Hz
 
The exact frequency output reported should correspond to the display refresh rate (60 Hz in this example). Simply terminate the test when satisfied.