Hardware Acceleration
Realtek has developed a couple of hardware to system burden on UI related issues, such as image blending and LCD panel driving. The following modules are included:
LCDC: LCD Controller which supports RGB, SDI and DBIB interfaces.
IDU: Image Decode Unit, which decodes the compress data for PPE/MCU to handle. It reduces the memory space of image storing and supports multiple encoding methods. IDU is recommended to work together with image tool produced by Realtek.
PPE: Pixel Process Engine, which blends layers together.
1. LCDC (LCD Controller)
LCDC provides several interfaces for developers to drive display panel, and integrates an internal DMA for data transmission. It avoids concentration on timing and releases more resources for other tasks. Furthermore, LCDC provides some practical functions for display, such as pixel format conversion, adjusting byte order and TE signal synchronization.
LCDC supports the interfaces listed below:
SDI
SDI is develop based on SPI (Serial Peripheral Interface), and extended by importing extra control signals. SDI can also drive LCD panel without GRAM through use QSPI. SDI can set command, address and data in advance to avoid the circumstance that user needs to construct the data packet, and user no longer needs to adjust lanes and timing for each phase.
DPI
The display panel using DPI interface usually contains no GRAM. It requires main controller continuously sending data and synchronize signals. LCDC has an infinite mode, which can automatically sending synchronize signals and data to screen with internal DMA. It avoids the requirement of screen refreshing with interrupt frequently.
DBIB
DBIB is similar with SDI, it can set command, address and data in advance to help user to control timing and construct data packet.
2. IDU (Image Decode Unit)
Realtek provides image tool to help compressing images and uses IDU to decompress these images on SoC. Currently IDU supports pixel formats of 16 bits, 24 bits and 32bits, and there's no limitation on width and height.
IDU the following compression methods.
RLE (Run Length Encode): A lossless method.
FastLZ: A lossless method.
YUV Conversion: Support YUV444, YUV422 and YUV 411 sampling methods. It can abandon 4 bits at most for further compression. It is a lossy compression method, particularly suitable for photos.
3. PPE (Pixel Process Engine)
PPE is a hardware designed for image blending. It can mix front ground and background with transformation described by any valid matrix following the formula: Result = Aplha_FG × FG + (1-Alpha_FG ) × BG
There can be more blending formula on different platforms.
During the blending procedure, PPE can convert pixel formats, filter color key and produce a scissor. After each pixel is processed, PPE will store the output in specified memory for further handling.
PPE has different capability on different platform, which is listed below.
The transformation of front ground can be describe by a combination of multiple 3 by 3 matrices, some typical transformations are demonstrated below:
Perspective transformation can be used to imitate 3D scenes:
HoneyGUI
HoneyGUI is a GUI developed by Realtek, primarily designed to run on embedded SoC. It delivers rich and smooth visual effects while consuming minimal chip resources:
A great variety of powerful widgets available.
Features include vector, 2.5D and 3D graphics.
Designed for compatibility, it can run on multiple platforms and also supports development using a PC simulator.
Offers a graphical development tool called RVD to reduce development costs.
Detailed development documentation is provided. For more information, please refer to https://docs.realmcu.com/gui/en/latest/index.html.
1. Widget Introduction
HoneyGUI provides over 50 rich and diverse widgets to meet various needs in embedded GUI development. Below are examples of its main widget categories:
Display Widgets
Image Widget: Basic widget for displaying images, supports movement, scaling, and rotation.
Text Widget: Enables diverse text rendering with features like text layout, scrolling, and anti-aliasing.
Video Widget: Plays video streams in formats such as H.264 and MJPEG.
Canvas Widget: Designed for 2D graphics drawing, supports software-accelerated rendering and multiple image output formats.
Gesture Widgets
Button: Responds to click, press, and release events, offering simple and intuitive interaction, ideal for triggering functional operations.
Seekbar: Allows sliding to adjust progress, suitable for applications like media players or brightness controls.
Container Widgets
View Widget: Supports real-time view creation with multiple transition effects, effectively reducing memory consumption.
List Widget: Creates lists in various styles, supports real-time updating and scrollbar integration.
2. Demonstration
View Transition Effects:
List Application Menu:
Cellular Application Menu:
Explosion Animation Countdown:
Music Player:
Visual Programming
1. In-depth Exploration of Realtek Visual Designer: A Next-Generation UI Design Tool
Realtek Visual Designer is a professional tool dedicated to simplifying the graphical user interface design process for Realtek HoneyGUI embedded systems. It supports usage on Windows operating systems and is designed to help developers quickly build simple, user-friendly, and powerful user interfaces. This tool is suitable for a variety of application areas such as smart home panels and smart dashboards, and it significantly enhances design efficiency while ensuring both aesthetics and practicality of the interface.
Realtek Visual Designer supports multiple series of Realtek IC, including the RTL87x2 and RTL87x3 series, offering both Bluetooth Low Energy and Dual-Mode functionalities. Users can effortlessly build complex graphical interfaces through intuitive drag-and-drop components. Once the design is complete, projects can be exported as .bin files and directly flashed onto Realtek HoneyGUI embedded devices, ensuring a smooth transition from design to deployment.
For engineers focused on software development with Realtek integrated circuits, mastering Realtek Visual Designer is essential. This tool not only improves development efficiency but also ensures the ability to create high-quality graphical interfaces on the Realtek platform.
2. Revealing Key Features
Drag-and-Drop UI Editor: Simplify Operations, Unleash Creativity
Transform complex UI design into an easy 'puzzle game' without requiring extensive programming skills. Through simple drag-and-drop operations, various UI components can be precisely added to the design canvas, and their appearance can be easily adjusted using the components' properties view.
Rich Component Library: Rapid Construction, Unified Visual Style
Provides a variety of standard components including buttons, labels, images, switches, dropdown menus, text areas, and keyboards, offering a solid foundation for creating outstanding user interfaces. Realtek can also develop highly integrated and customized components based on customer needs to meet unique application scenarios.
Low-Code Development: Easy Implementation, Increased Efficiency
By combining a small amount of C/JavaScript code, complete functionalities can be realized, significantly speeding up the process compared to traditional development methods. This helps development teams focus on optimizing business logic and user experience. It reduces dependency on underlying code, lowers development thresholds, allows non-technical personnel to participate, and saves manpower costs. Additionally, the simplified code writing method makes subsequent modifications and updates more convenient, enhancing project flexibility and adaptability.
Multifunctional Resource Management: Supports Multiple Formats, Efficient Utilization
Supports importing various common image formats and classifies them efficiently; with image compression function to effectively reduce the size of exported files, ideal for embedded systems with strict Flash storage requirements. Supports importing third-party font files to meet different design style needs and ensures global reuse of fonts through unified management, ensuring style consistency and reducing workload and error probability.
GUI Simulation: Pre-validate, Ensure Precise Decisions
Supports simulation running on Windows platforms, allowing developers to validate GUI functionalities before porting, identify, and fix functional defects or logical errors. This process significantly shortens the development cycle compared to discovering and fixing issues after porting.
Fast Export and Porting: Simplify Processes, Improve Migration Efficiency
One-click export functionality packages project files and resources into a format directly flashable to Realtek HoneyGUI embedded devices, simplifying the process and making it easier for developers to complete project porting and deployment.
3. Step-by-Step Tutorial
1) Download: Start Your Design Journey
Visit the official website at RealMCU to download the Realtek Visual Designer standalone version. After extraction, you can begin your design journey.
2) Create a Project: Build Your Creative Platform
After double-clicking to open the tool, select the IC type you want to develop, and set the project name, solution path, and screen resolution. Click the 'Create Project' button to create a new project, and a blank design canvas will be presented to you, ready for your creative expression.
3) Interface Design in Practice: From Conception to Presentation
Explore the design capabilities of Realtek Visual Designer by creating a simple clock face.
Step 1: Drag an Image component from the toolbox panel on the left onto the canvas to serve as the background image for the clock face. Adjust the position and size of the image to cover the entire canvas, then select a pre-imported background image.
Step 2: Drag a Clock component onto the canvas and set the background images and rotation center for the hour, minute, and second hands.
4) Export and Integration: Bringing Designs to Life
After completing the design, click the 'Export' button in the menu bar to package the project files and resources into a binary file.
Then, click the 'Simulate' button in the menu bar to perform a simulation.
Once the simulation is verified to be correct, the exported files can be flashed into the Realtek HoneyGUI embedded device.
4. Showcase of Successful Cases
Using Realtek Visual Designer, developers can easily design intuitive and aesthetically pleasing smart home control panel interfaces.
The following are functional pages included in the smart home panel example:
1) Home Page
Add 'Text' components to display information such as the current date and weather conditions.
Use 'Image' components to set the background of the homepage.
2) Smart Lighting Control Page
Place multiple 'Switch' components to control lights with different functions.
3) Smart Air Conditioning Control Page
Use 'Button' components to adjust cooling, heating, air swing, and dehumidification modes of the air conditioner.
Add 'Text' components to display the air conditioner temperature and current mode.
Use 'Slider' components to enable drag adjustment of the air conditioner temperature.
4) Smart Curtain Control Page
Integrate 'Image Slider' components to adjust the curtain's opening and closing progress and state through a set of curtain state images.
Use 'Button' components to control the opening and closing of the curtains.
These examples showcase the rich functionality and ease of operation of Realtek Visual Designer in designing smart home panels. If you have any other ideas or suggestions, feel free to discuss and exchange thoughts at any time.
LVGL
LVGL (Light and Versatile Graphics Library) is a free and open-source C language graphics library that provides all the necessary tools to create embedded GUI with easy-to-use graphical elements, aesthetic visual effects, and low memory usage.
The Realtek platform now supports LVGL, combining years of accumulated display solutions and image processing technology with LVGL's open ecosystem to build a complete development system.
1. Hardware Acceleration
The Realtek display solution includes three independent image-related hardware units: the drawing unit PPE, the decompression unit IDU, and the JPEG unit JPU.
PPE is a Realtek-developed image processing unit that can significantly enhance image rendering efficiency. It is deeply integrated as an independent drawing unit under the LVGL rendering interface and can handle most drawing tasks, supporting image matrix rendering.
IDU is a Realtek-developed image decoding unit that supports various compression algorithms, significantly reducing resource storage overhead and resource loading time. It is used in LVGL as a standard decoder in conjunction with the image converter.
JPU is a standard JPEG processing unit that can perform JPEG decompression and compression. It can be used as a standard decoder to decompress JPEG images and can also achieve efficient video solutions through upper-layer encapsulation.
2. Resource Tools
Realtek has packaged and extended the resource conversion tools based on LVGL, adding a visual tool interface and several new features.
Image Converter
Compared to the original LVGL image converter, the Realtek image converter offers the following advantages:
1) Visual operation interface, supporting real-time image preview and conversion configuration.
2) Supports batch image conversion and individual image configuration.
3) Added support for Realtek platform hardware compression formats, reducing resource storage overhead.
4) Supports the separation of image data and descriptive information, better adapting to the storage management model of embedded platforms.
5) Supports video solution conversion.
6) Supports one-click conversion and packaging, improving development efficiency.
Font Converter
Compared to the original LVGL font converter, the Realtek font converter offers the following advantages:
1) Visual operation interface, making the font conversion process clear and easy to operate efficiently.
2) Supports multi-font, multi-size, and configuration label design, significantly enhancing the flexibility and efficiency of project development.
3) Added support for Realtek platform GPU rendering features, fully leveraging hardware acceleration capabilities.
4) Supports the separation of font data and descriptive information, better adapting to the storage management model of embedded platforms.
5) Supports the import and export of configuration schemes, facilitating project migration and team collaboration.
3. Innovative Extension -- 2.5D Transition Effects and Page Caching Mechanism
Leveraging the drawing capabilities of the 2.5D GPU and the LVGL core framework, two sets of page management solutions are integrated based on tileview and screen, both including 2.5D transition effects and a dynamic page compression caching mechanism to ensure smooth frame rates and stunning effects during page transitions.
Lightweight 3D Engine
A self-developed lightweight 3D engine (Lite3D) is integrated into LVGL, allowing various 3D effects to be rendered without modifying the LVGL core code. For detailed information, please refer to the 3D engine chapter.
Waterfall Flow Cards
Honeycomb List
4. Design Tools
Currently, LVGL has a rich designer ecosystem, including several mainstream tools like SquareLine Studio, EEZ Studio, and LVGL Editor. These designers provide visual interface building, efficient widget configuration, and automatic code generation, greatly simplifying the development process of embedded GUIs. With these tools, developers can further accelerate interface design, improve development efficiency, and better meet diverse cross-platform application needs.
Realtek has fully supported SquareLine Studio. Based on the software and hardware features of the Realtek platform, design optimization suggestions for LVGL projects have been proposed, forming a complete project development process. Users can directly design UIs with SquareLine Studio, export C code and resources, and utilize the Realtek-provided resource converter to achieve efficient features such as image compression and GPU font acceleration. The entire integration process covers design, code generation, resource conversion, and adaptation, ensuring developers can quickly and efficiently implement a complete LVGL solution on the Realtek platform. Detailed guidance is also provided for special resource naming, migration adaptation, and advanced feature extensions, minimizing the usage threshold and enhancing development efficiency.
Lite3D Engine
Lite3D is a lightweight 3D rendering engine designed for embedded systems and resource-constrained devices. It supports software rendering and hardware acceleration extensions, featuring:
Lightweight: Small code footprint for easy integration.
Cross-platform: Compatible with HoneyGUI, LVGL, and other GUI frameworks, and can run on FreeRTOS and Zephyr environments.
Flexible Rendering: Provides a software rendering pipeline with support for GPU acceleration extensions.
1. Key Features
Model Support: Parses OBJ files, extracting vertex, texture, and material data.
Coordinate Transformation: World space → Camera space → Screen space (with perspective projection).
Rendering Pipeline: Vertex processing → Backface culling → Scanline rasterization → Depth buffering.
Performance Optimization: MVE (Matrix-Vector Engine acceleration).
Hardware Extensions: Provides GPU acceleration interfaces (e.g., PPE adaptation layer).
2. Processing Pipeline
Lite3D Engine Processing Pipeline
3. Demo Presentation
Font
HoneyGUI is a graphical user interface display solution developed by Realtek. It supports multiple font display engines, enabling users to implement diverse and customizable font display solutions in their devices and applications, applicable to various smart wearable devices and IoT devices.
1. Introduction
The font display solution utilizes the text controls and font rendering engine in HoneyGUI to provide a rich set of features:
Offers 15 text layout modes to meet diverse layout needs.
A schematic of all layout modes is shown below:
Layout Modes Schematic
The actual rendering effects of text under several layout modes are as follows:
Single-line layout (LEFT/CENTER/RIGHT) display effects
Multi-line layout (MULTI_LEFT/MULTI_CENTER/MULTI_RIGHT) and word wrap display effects
Chinese and vertical layout (VERTICAL_LEFT) display effects
Supports text scrolling effects in four directions to enhance visual dynamism.
Scrolling text (SCROLL_X/SCROLL_Y) effects
Supports custom animation effects, adding creative elements to text display.
Animated text effects
Compatible with a variety of EMOJI symbols, enriching forms of expression.
Emoji text display effects
Supports text rotation, translation, scaling, perspective, and transparency effects.
Multi-line text transition display effects
Supports multi-level anti-aliasing to ensure high quality and clarity of text display.
Utilizes alpha channel anti-aliasing technology, representing pixel opacity with 1/2/4/8-bit data to smooth glyph edges and reduce aliasing effects, providing clear and smooth text display. The image shows rendering effects of four anti-aliasing levels, from top to bottom: 1-bit (2-level), 2-bit (4-level), 4-bit (16-level), and 8-bit (256-level) anti-aliasing.
Multi-level Anti-aliasing Display Effects
In addition to providing rich text display features, HoneyGUI excels technically, showcasing highlights in compatibility, flexibility, and development support.
Compatible with UTF-8, UTF-16, and UTF-32 standard Unicode encodings.
Supports diverse rendering with multiple languages, fonts, and sizes.
Supports 32-bit true color, presenting more vivid color performance.
Compatible with standard TTF font files and custom font library files.
Provides extensive technical documentation and a modular coding structure, facilitating the porting of custom font libraries.
2. Font Solutions
HoneyGUI offers comprehensive solutions for different font rendering methods, including bitmap font solutions and vector font solutions, to meet various application needs. Both solutions support all the features mentioned above.
Bitmap Font Solutions
Realtek has developed a complete bitmap font generation tool that allows customization of character indexing within any range, and supports multi-level anti-aliasing and compression functions. This ensures high-quality text display while effectively reducing file size. Leveraging HoneyGUI's text controls, bitmap fonts can be flexibly inserted into any interface, achieving various custom display effects with extremely high rendering speed. The font library solution offers extensive technical and porting documentation, making it easy for users to port font library code when they need to use custom font library solutions.
Vector Font Solutions
Vector font solutions provide higher display quality, with accompanying font tools capable of cropping and encoding vector fonts, ensuring the precision of vector information while reducing file size. The vector font engine within HoneyGUI has undergone deep algorithm optimization for the Realtek platform, ensuring both quality and efficiency. Additionally, vector fonts support a complete image rendering matrix, enabling diverse image transformation effects, allowing text to achieve a special presentation akin to artistic fonts.
Artistic Font Effects
Performance Comparison of Bitmap and Vector Font Solutions
Visit: HoneyGUI GitHub: https://github.com/realmcu/HoneyGUI
Video Solutions
1. High-Efficiency Video Playback Based on JPU (JPEG Process Unit)
This solution utilizes the JPU for smooth video playback by sampling videos into JPEG frame sequences, or using MJPG-formatted video files. The JPU provides high-speed decoding of video frames. JPEG encoding effectively compresses each frame while maintaining visual quality, achieving a compression ratio of approximately 1:10 and significantly reducing the required storage space for video files.
During Playback:
The JPU rapidly decodes JPEG frame sequences into RGB/YUV formats, with key features including:
Efficient Decoding Speed: Ensures smooth playback experience.
Support for Multiple YUV Sampling Formats: Compatible with various JPEG images.
8-bit Depth Decoding: Supports 24-bit true color and grayscale output.
Multiple Color Output Formats:
Common RGB formats: RGB565, RGB888, ARGB8888 (with customizable global transparency).
YUV formats: planar YUV, packed YUV422, packed YUV444.
Region of Interest (ROI) Decoding: Allows partial decoding for specified regions, enhancing performance and flexibility.
Advantages:
High Compression Ratio: Up to 1:10 compared to original image frames, effectively reducing storage and transmission overhead.
Flexible Frame Control: Supports dynamic adjustment of video playback speed, start point, and order, meeting diverse application requirements.
2. High-Efficiency Video Playback Based on Software H.264 Decoder
This solution achieves smooth video playback by using a software H.264 decoder. Raw videos are converted to H.264 encoding, then played using the decoder. H.264 delivers smaller file sizes for the same visual quality, saving storage space and bandwidth, and supporting high-quality, low-bitrate video streaming.
During Playback:
The H.264 decoder decodes each frame to RGB/YUV formats, with key features including:
High-speed Decoding: Ensures smooth playback.
Lightweight and Efficient Decoder: Low system resource consumption.
8-bit Depth Decoding: Provides 24-bit true color video output.
Multiple Color Output Format Support:
Common RGB formats: RGB565, RGB888, ARGB8888.
Common YUV formats: planar YUV.
H.264 240x240 video decoding and scaling up to 410x502 (Note: Source: Planet Earth one minute rotate - Loop with Matte 17619487 Stock Video at Vecteezy)
Advantages:
Extremely High Compression Ratio: Achieves up to 1:100 compression compared to raw frame data, greatly reducing storage and transmission needs.
Adaptable to Various Application Scenarios: Especially suitable for storage and bandwidth-sensitive applications such as video streaming and mobile device playback.
Application Scenarios
Our company's graphics solution is designed to provide powerful, efficient, and low-power graphics processing capabilities for various smart terminal devices. Through this solution, our partners can achieve smooth and stunning graphical user interfaces (GUI) and innovative human-machine interaction experiences on their products, thereby significantly enhancing product competitiveness and user value. Below are typical application scenarios of this solution in several key areas.
1. Smart Wearables: The Information and Interaction Hub on Your Wrist
For wearable devices like smartwatches, which have extreme requirements for power consumption and size, this solution can create a feature-rich and smooth user experience.
Smooth and Expressive UI: Supports high frame rate dynamic watch faces, smooth list scrolling, and stunning transition animations. Users can intuitively operate through the touchscreen and physical buttons, enjoying an interaction experience comparable to smartphones.
Powerful Application Ecosystem Support: With efficient computing and graphics rendering capabilities, it can smoothly run map navigation, data visualization health apps, casual games, and various third-party mini-programs, greatly expanding the functional boundaries of smartwatches.
Seamless Information Synchronization and Push: As an extension of a smartphone, it can display incoming calls, messages, schedule reminders, and other notifications on the screen in real-time and clearly, ensuring users do not miss any important information.
2. Smart Appliances and Central Control: The Visual Steward of the Home Environment
This solution provides an intuitive graphical interface for smart appliances and home control devices, making complex device management and environment control simple and easy to use.
Refined Device Management Interface: On devices like smart refrigerators and ovens, it provides a friendly visual interface where users can intuitively adjust the temperature of different compartments of the refrigerator, select cooking modes, or check the status of ingredients through the touchscreen.
Unified Smart Home Control Hub: Serving as the central control panel of the home, it allows centralized management of smart devices that follow protocols like Matter and Mesh through a graphical interface. Users can control the entire home's lighting, curtains, and air conditioning with a touch of the screen and view device status in real-time.
Scene and Automation Configuration: Supports visual editing and one-click activation of custom scenes such as 'Home Mode' and 'Cinema Mode', integrating cumbersome single-step operations into intelligent and automated processes to enhance the quality of life.
3. Two-Wheel Electric Vehicle Dashboard: The intelligent riding companion
This solution upgrades the traditional electric vehicle dashboard into a smart information terminal that integrates information display, navigation, and vehicle condition control.
Real-time Core Riding Data Display: Provides precise reports on key information such as instantaneous speed, remaining battery, and estimated range through a high-contrast, high-definition graphical interface to ensure riding safety.
Convenient Visual Navigation and Communication: By linking with a mobile app via BT/Wi-Fi, the map navigation interface from the phone can be projected onto the dashboard in image frame format (e.g., displaying 360x240 pixel turn prompts at 2FPS), while also pushing important calls and messages so the rider's eyes can stay on the road.
Deep Vehicle Personalization Configuration: Offers a graphical settings menu allowing users to engage in richer interactions, such as adjusting power mode, lighting settings, and viewing riding data statistics, achieving deep customization of the vehicle.
4. Customizable Keyboard: The Interactive Console on the Desktop
This solution adds an interactive screen to peripherals like mechanical keyboards, evolving them from input tools into desktop information centers and efficiency tools.
Real-time System Information Dashboard: Monitor system performance indicators like CPU/GPU temperature, usage rate, and network speed in real-time on the keyboard screen without switching application windows.
Enhanced Gaming and Professional Application Experience: Deep integration with games or productivity software (like Photoshop, Premiere) allows displaying key in-game states (such as health, skill cooldown) or software's shortcut toolbar on the screen, improving operational efficiency.
Extreme Personalization and Visual Expression: Users can customize the screen to display GIF animations, team logos, or personal signatures, and easily configure macro key functions through a visual interface, creating a unique personalized device.
5. Voice remote control with screen: A multimodal intelligent interaction entrance
By adding a display screen to the voice remote control, this solution addresses the pain points of unclear feedback in traditional voice interaction and interruption of the main screen experience.
Precise Visual Voice Feedback: Users' voice commands are converted into text and displayed on the remote control screen in real-time, allowing immediate confirmation of recognition accuracy and significantly enhancing the success rate of voice interaction and user trust.
Non-intrusive 'Second Screen' Experience: Users can browse search results, check weather, and control smart home devices on the remote control's small screen, while completely unaffected by the content being played on the TV's main screen, achieving multitasking.
Intuitive Graphical Advanced Settings: For complex settings like TV picture quality and sound equalizer, users can adjust using sliders and virtual knobs on the remote control's touchscreen, making operation intuitive and efficient.
苏公网安备32059002006558号