Background#
I have a cloud rhinoceros Box 3.0 Pro touch screen live broadcasting all-in-one machine that has been idle for many years. This device supports dual 1080P HDMI input, built-in Douyin, Taobao, Kuaishou live streaming, and RTMP streaming functions. However, our previous usage scenario was quite special; some platforms could not conveniently perform RTMP streaming, and we mostly used a computer connected to multiple USB capture cards and OBS Studio virtual cameras, which is why this device has been unused for many years.
Recently, I happened to be developing a similar hardware product, so I began to re-evaluate this device. On one hand, I hope to learn and understand its hardware and software solutions; on the other hand, I also hope to modify the software to allow its unique hardware to play a role in other scenarios.
Hardware#
By unscrewing the screws hidden under the anti-slip mat on the back, the rear cover can be easily opened. The design of the mainboard is relatively simple, with components laid out on one side, and it adopts a core module + baseboard form.
For projects with ample PCB space like this, using a core board or core module indeed has many advantages. On one hand, the most complex designs, such as high-frequency RF parts, high-speed DDR memory, and storage, are all integrated into the core module, greatly reducing the workload of hardware design and debugging, which helps bring the product to market faster; on the other hand, the baseboard can use fewer layers and simpler processes, keeping high multilayer boards to a minimum area and effectively controlling costs.
From the disassembly diagram, we can see the main chip solution of the cloud rhinoceros Box 3.0 Pro:
- Core module: Quectel SC60 module, equipped with Qualcomm Snapdragon 625 (MSM8953) SoC, 2GB RAM, and 16GB eMMC.
- Ethernet: ASIX AX88179 USB 3.0 to Gigabit Ethernet
- MIPI-DSI to HDMI Tx bridge: Lontium LT8912B
- HDMI Rx to MIPI-CSI bridge: Toshiba TC358840XBG *2
- Power management: Texas Instruments BQ25890
- USB HUB: Genesys Logic GL3523
At first, I guessed it used the most common Rockchip RK3399 solution, but after disassembly, I found it was a Qualcomm solution. Upon further reflection, considering that live broadcasting devices may need to stream via mobile networks, the Qualcomm platform with its baseband advantages becomes very attractive, making the choice of the Qualcomm solution reasonable.
For its core function of HDMI capture, the cloud rhinoceros Box uses two Toshiba TC358840XBG bridge chips to achieve dual signal capture. Although TC358840XBG is classic, it has some limitations; it requires Dual Link MIPI-CSI-2 with a total of 8 lanes to achieve 4K 30fps video capture (the two links capture the left and right halves of the 4K image, respectively), while devices like the cloud rhinoceros Box that use Single Link CSI-2 can only support up to 1080p capture. In contrast, newer chips, such as Lontium's LT6911UXE, can achieve a maximum lane speed of 2.5Gbps per MIPI lane, supporting 4K 60fps capture with just one 4-lane MIPI port, making the 4K capture solution much simpler. However, TC358840XBG has open-source drivers available, which are friendly to individual developers, while the publicly available information on LT6911UXE is currently very scarce.
In hardware design, one can feel that the peripheral limitations of MSM8953 pose some challenges to the design. Since MSM8953 does not have a built-in Ethernet MAC and no PCIe interface, achieving wired network access basically relies on USB network cards. However, MSM8953 only has one USB 3.0 interface. This leads to a series of chain reactions: on one hand, to support both a USB network card and an external USB 3.0 Type-A interface simultaneously, a USB HUB chip (GL3523) must be introduced. On the other hand, the Type-C interface, primarily used as a Device, and the USB HUB used as a Host need to use USB switch chips to switch the physical connection of signals, corresponding to the PI3USB302 and TS3USB221A on the mainboard. This means that once the system switches to Host mode (connecting the network card and Type-A port), the Type-C interface is completely disconnected, and functions relying on this interface, such as ADB and Fastboot, can no longer be used.
Software#
From the device interface, it can be seen that the cloud rhinoceros Box runs an Android system. Compared to a pure Linux solution, using the Android system can simplify development by leveraging its high abstraction of hardware and significantly improve application development efficiency through a mature application development ecosystem based on Java.
However, this Android system is deeply restricted; it can only run the official cloud rhinoceros APP, cannot directly launch any application, there is no place to open ADB, the notification bar is locked, and standard Android interfaces such as system settings are also hidden. It seems that trying to lift the restrictions through simple system interface operations is not feasible.
Thus, the next step plan was formulated: to use Qualcomm's 9008 Emergency Download Mode (EDL) to extract the system partition image from the device's eMMC storage, then mount the image for modification, replace the built-in app, and finally re-flash the modified image.
After trying, I found that the reserved button S2 on the mainboard is connected to the force_boot_from_usb pin (GPIO37) of MSM8953. By holding this button during power-up, the device can enter 9008 mode. Using the Qualcomm Premium Tool, I exported the system partition image, and then I could freely mount and modify this image on the PC.
Since the cloud rhinoceros APP can launch specific third-party applications like Douyin, Kuaishou, and Taobao live streaming, I replaced the APK file corresponding to Taobao live streaming with a modified package name QuickShortcutMaker APK for a switch. After flashing the modified system.img back to the device via 9008 mode, upon rebooting, I saw that the original "Taobao Live" icon on the desktop had changed to the QuickShortcutMaker icon. Opening QuickShortcutMaker allowed me to freely launch any application and system component installed in the system, including the hidden native Android settings.
To gain complete system control, rooting is essential. I used Magisk to patch the device's boot partition image, adding the su command to the system, and after flashing, this device successfully obtained root privileges.
The USB issue I was previously worried about did occur. Although I successfully enabled the USB debugging option in the settings, I still could not see the ADB device when connecting the Type-C port to the computer. This indicates that the USB controller is currently in Host mode, used for the built-in USB network card and Type-A port.
However, forcing a mode switch myself is quite tricky. One potentially feasible but unverified path is to cut off the switching control signal line of the USB switch chip on the hardware side, ensuring that the USB signal physically always points to the Type-C interface; on the software side, set /sys/kernel/debug/msm_otg/mode to peripheral, making the USB controller operate in Device mode.
However, I accidentally discovered that pressing another reserved button S3 on the mainboard unexpectedly brought up USB debugging, which should be a debugging method reserved by the manufacturer. This saved us the trouble of making complex software and hardware modifications ourselves.
Later, to sort out the process of switching USB modes, I extracted and decompiled the device tree from boot.img. In the usb_detect node of the device tree, I found that qcom,gpio-usbdetect additionally defines a key-gpio, whose GPIO number corresponds to the vol_up key, and the S3 button on the board corresponds to this vol_up. Therefore, the manufacturer should have modified the gpio-usbdetect driver in the kernel so that pressing the S3 button can also trigger an interrupt and execute the USB switch switching and USB Role (Host/Peripheral) switching logic.
vol_up {
label = "volume_up";
gpios = <0xbe 0x55 0x01>;
linux,input-type = <0x01>;
linux,code = <0x73>;
debounce-interval = <0x0f>;
};
usb_detect {
compatible = "qcom,gpio-usbdetect";
interrupt-parent = <0x11e>;
interrupts = <0x00 0xc6 0x00>;
interrupt-names = "vbus_det_irq";
key-gpio = <0xbe 0x55 0x01>;
usb-switch-gpio = <0xbe 0x1c 0x00>;
};
By observing the kernel logs (dmesg), we can see the relevant logs of usb_role_switch before and after pressing S3:
[ 1089.480345] [yunxi_bq25890]: vbus_detect_delay_work: key_level: 0
[ 1089.480374] [yunxi_bq25890]: usb_role_switch , is_host: 1
[ 1329.977896] --== bq25890_irq_handler
[ 1339.940379] [yunxi_bq25890]: vbus_detect_delay_work: key_level: 0
[ 1339.940410] [yunxi_bq25890]: usb_role_switch , is_host: 0
HDMI Capture#
HDMI capture is the most core and unique function of this device. My goal is to bypass the heavily restricted official cloud rhinoceros APP and directly utilize this hardware capability.
The TC358843XBG chip outputs video data through the MIPI CSI-2 interface, which is consistent with the way general sensors connect to SoCs. The difference is that TC358843XBG directly outputs YUV or RGB format data, rather than the Bayer RAW format commonly found in cameras.
After installing the open-source Open Camera application, it can directly recognize and preview the images from the two HDMI inputs, which correspond to two cameras. This way, real-time monitoring and video recording of HDMI input can be done directly through Open Camera. Once again, it is impressive that Android's good hardware abstraction brings strong compatibility, allowing many functions to be developed without repeating for each specific platform, and leveraging the vast ecosystem, existing and even open-source applications can be directly utilized.
By checking the debug information of Open Camera and combining it with the decompilation analysis of the official cloud rhinoceros APP, it can also be confirmed that the cloud rhinoceros Box uses the older android.hardware.Camera API (Camera1 API), which does not support the more powerful and flexible Camera2 API.
Others#
Boot Process#
Due to some errors during the partition backup writing process, I gained a rough understanding of the boot process of the MSM8953 platform while troubleshooting:
- Power on and Boot ROM (PBL):
- After pressing the power button, the processor starts executing from a predefined address in the internal ROM (Boot ROM), running the Primary Boot Loader (PBL).
- PBL reads the partition table (GPT) on the boot medium such as eMMC, finds the
sbl1(Secondary Boot Loader 1) or backupsbl1bakpartition, loads it into on-chip memory (IMEM), and jumps to execute.
- SBL1:
- SBL1 first initializes DDR memory, then loads and verifies the components needed for subsequent boot stages:
tz.mbn: TrustZone related components (QSEE, HYP, TEE, etc.).devcfg.mbn: Device configuration data.rpm.mbn: Resource Power Manager (RPM) firmware, loaded into the internal memory (TCM) of the RPM core.aboot.mbn: Application Boot Loader (APPSBL), usually Little Kernel (LK).
- Sends a start signal to RPM and releases its reset state.
- Requests a Warm Reset by writing
RMR_EL3, and the CPU restarts in AArch64 EL3 privilege level, entering the TrustZone execution environment.
- SBL1 first initializes DDR memory, then loads and verifies the components needed for subsequent boot stages:
- QSEE/ TrustZone:
- Performs security-related initialization in the EL3 environment.
- Switches the CPU back to AArch32 EL1 and jumps to the entry point of
aboot.mbn(LK).
- Aboot (LK):
- LK (little kernel) provides Fastboot flashing mode.
- Initializes necessary hardware, such as the MIPI-DSI controller and display panel, reads the logo file from the
splashpartition, and displays the boot screen. - Loads
boot.img(normal boot) orrecovery.img(recovery mode) and verifies it. - Prepares the kernel runtime environment, switches to AArch64 EL1, and jumps to the kernel's entry address.
- Kernel:
- Identifies and mounts partitions such as
system,vendor,userdata,persist, etc. - Loads and starts other coprocessor firmware, such as Modem (baseband), WCNSS (Wi-Fi/Bluetooth).
- Enters userspace, starting the first user-space process
init, and booting Android.
- Identifies and mounts partitions such as
Device Tree Extraction#
The device tree describes hardware information, and the kernel reads it during boot to configure the corresponding drivers. On Qualcomm platforms, the DTB (Device Tree Blob) is usually appended to the end of the kernel image (zImage).
Using the unpackbootimg tool to separate zImage from boot.img, and then using the split-appended-dtb tool obtained from moetayuko/split-appended-dtb to extract all appended DTB files from boot.img, a total of 69 DTB files were extracted.
$ file boot.img
boot.img: Android bootimg, kernel, ramdisk, page size: 2048, cmdline (console=ttyHSL0,115200,n8 androidboot.console=ttyHSL0 androidboot.hardware=qcom msm_rtb.filter=0x237 ehci-hcd.park=3 lpm_levels)
$ ./unpackbootimg -i boot.img -o ./
BOARD_KERNEL_CMDLINE console=ttyHSL0,115200,n8 androidboot.console=ttyHSL0 androidboot.hardware=qcom msm_rtb.filter=0x237 ehci-hcd.park=3 lpm_levels.sleep_disabled=1 androidboot.bootdevice=7824900.sdhci earlycon=msm_hsl_uart,0x78af000 buildvariant=user
BOARD_KERNEL_BASE 80000000
BOARD_PAGE_SIZE 2048
$ ls
boot.img boot.img-base boot.img-cmdline boot.img-pagesize boot.img-ramdisk.gz boot.img-zImage
$ ./split-appended-dtb boot.img-zImage
Found 69 appended dtbs, please check the output.
Subsequently, using the dtc (Device Tree Compiler) tool to decompile the binary DTB files into human-readable text format DTS (Device Tree Source).
#!/bin/bash
for i in $( ls *.dtb ); do
echo decompile $i
dtc -I dtb -O dts $i -o output/$i.dts
done
There are many device trees, but only one will be used. LK will select the most matching one to load based on the hardware ID (such as qcom,msm-id, qcom,board-id) passed in by SBL. By checking the log information output by LK during the boot process via the serial port, it can be found that it matched the device tree with ID <293 8 0 0x0>.
[2110] Found an appended flattened device tree (Qualcomm Technologies, Inc. MSM8953 + PMI8950 MTP - 293 8 0 0x0)
[2120] Add DTB entry 293/00000008/0x00000000/0/10016/0/0/0/a11c281f/3e740
[2120] Device tree exact match the board: <293 8 0 0x0> == <293 8 0 0x10001>
Accordingly, the corresponding device tree DTS file is:
/ {
model = "Qualcomm Technologies, Inc. MSM8953 + PMI8950 MTP";
compatible = "qcom,msm8953-mtp\0qcom,msm8953\0qcom,mtp";
qcom,msm-id = <0x125 0x00>;
interrupt-parent = <0x01>;
qcom,board-id = <0x08 0x00>;
qcom,pmic-id = <0x10016 0x00 0x00 0x00>;
...
};
From the DTS file, a wealth of information can be obtained, including special GPIO allocations, screen initialization sequences, etc. This is very valuable for system porting. For example, with the following MIPI DSI screen initialization information, lighting up this screen becomes easy.
qcom,mdss_dsi_yunxi_1200p_video {
qcom,mdss-dsi-panel-name = "yunxi 1200p video mode dsi panel";
qcom,mdss-dsi-panel-controller = <0x1a4>;
qcom,mdss-dsi-panel-type = "dsi_video_mode";
qcom,mdss-dsi-panel-destination = "display_1";
qcom,mdss-dsi-panel-framerate = <0x3c>;
qcom,mdss-dsi-virtual-channel-id = <0x00>;
qcom,mdss-dsi-stream = <0x00>;
qcom,mdss-dsi-panel-width = <0x4b0>;
qcom,mdss-dsi-panel-height = <0x780>;
qcom,mdss-dsi-h-front-porch = <0x32>;
qcom,mdss-dsi-h-back-porch = <0x3e>;
qcom,mdss-dsi-h-pulse-width = <0x08>;
qcom,mdss-dsi-h-sync-skew = <0x00>;
qcom,mdss-dsi-v-back-porch = <0x08>;
qcom,mdss-dsi-v-front-porch = <0x0a>;
qcom,mdss-dsi-v-pulse-width = <0x06>;
qcom,mdss-dsi-h-left-border = <0x00>;
qcom,mdss-dsi-h-right-border = <0x00>;
qcom,mdss-dsi-v-top-border = <0x00>;
qcom,mdss-dsi-v-bottom-border = <0x00>;
qcom,mdss-dsi-bpp = <0x18>;
qcom,mdss-dsi-color-order = "rgb_swap_rgb";
qcom,mdss-dsi-underflow-color = <0xff>;
qcom,mdss-dsi-border-color = <0x00>;
qcom,mdss-dsi-on-command = [39 01 00 00 01 00 02 b0 00 39 01 00 00 01 00 06 b3 14 08 00 22 00 39 01 00 00 01 00 02 b4 0c 39 01 00 00 01 00 03 b6 3a d3 39 01 00 00 01 00 02 b7 00 39 01 00 00 01 00 07 b8 07 90 1e 00 1e 32 39 01 00 00 01 00 07 b9 07 82 3c 00 3c 87 39 01 00 00 01 00 07 ba 07 9e 20 00 20 8f 39 01 00 00 01 00 19 ce 7d 40 43 49 55 62 71 82 94 a8 b9 cb db e9 f5 fc ff 01 38 02 02 44 24 01 39 01 00 00 01 00 02 c0 ff 39 01 00 00 01 00 25 c1 04 61 00 20 8c a4 d6 ff ff ff ff 7f 73 ef b9 f6 ff ff ff ff bf 54 22 02 00 00 00 00 00 00 62 03 00 22 00 01 39 01 00 00 01 00 0a c2 31 f7 80 00 09 00 04 00 00 39 01 00 00 01 00 04 c3 00 00 00 39 01 00 00 01 00 0d c4 70 00 00 00 00 00 00 00 00 05 04 00 39 01 00 00 01 00 15 c6 6f 73 75 0a 14 0b 16 00 00 00 00 00 00 00 00 00 00 1c 17 09 39 01 00 00 01 00 1f c7 08 0e 19 22 31 3f 49 58 3c 44 4e 5b 64 6c 76 04 0c 13 1e 2d 3d 47 58 3c 44 50 5d 66 6e 76 39 01 00 00 01 00 14 c8 01 00 00 00 00 fc 00 00 00 00 00 fc 00 00 00 00 00 fc 00 39 01 00 00 01 00 0d cb 3f c0 0f f0 03 00 03 00 03 00 00 c0 39 01 00 00 01 00 02 cc 11 39 01 00 00 01 00 06 d0 44 81 bb 15 94 39 01 00 00 01 00 1a d3 0b 33 bf bb b3 33 33 37 00 01 00 a0 98 a0 00 39 39 33 3b 37 72 07 3d bf 99 39 01 00 00 01 00 03 44 07 7f 39 01 00 00 01 00 06 2a 00 00 04 af 00 39 01 00 00 01 00 06 2b 00 00 07 7f 00 39 01 00 00 01 00 02 d6 01 39 01 00 00 01 00 02 3a 77 39 01 00 00 01 00 02 2c 00 39 01 00 00 01 00 02 51 e6 39 01 00 00 01 00 02 53 24 39 01 00 00 01 00 02 55 01 05 01 00 00 32 00 02 29 00 05 01 00 00 78 00 02 11 00 39 01 00 00 32 00 02 35 00];
qcom,mdss-dsi-off-command = [05 01 00 00 32 00 02 28 00 05 01 00 00 78 00 02 10 00];
qcom,mdss-dsi-on-command-state = "dsi_lp_mode";
qcom,mdss-dsi-off-command-state = "dsi_hs_mode";
qcom,mdss-dsi-h-sync-pulse = <0x01>;
qcom,mdss-dsi-traffic-mode = "burst_mode";
qcom,mdss-dsi-lane-map = "lane_map_0123";
qcom,mdss-dsi-bllp-eof-power-mode;
qcom,mdss-dsi-bllp-power-mode;
qcom,mdss-dsi-lane-0-state;
qcom,mdss-dsi-lane-1-state;
qcom,mdss-dsi-lane-2-state;
qcom,mdss-dsi-lane-3-state;
qcom,mdss-dsi-panel-timings = <0xf23a2800 0x6c6e2c3e 0x2e030400>;
qcom,mdss-dsi-t-clk-post = <0x02>;
qcom,mdss-dsi-t-clk-pre = <0x2d>;
qcom,mdss-dsi-bl-min-level = <0x01>;
qcom,mdss-dsi-bl-max-level = <0xfff>;
qcom,mdss-dsi-dma-trigger = "trigger_sw";
qcom,mdss-dsi-mdp-trigger = "none";
qcom,mdss-dsi-bl-pmic-control-type = "bl_ctrl_pwm";
qcom,mdss-dsi-reset-sequence = <0x01 0x14 0x00 0x02 0x01 0x14>;
qcom,mdss-dsi-panel-timings-phy-v2 = <0x241f0809 0x50304a0 0x241f0809 0x50304a0 0x241f0809 0x50304a0 0x241f0809 0x50304a0 0x241c0809 0x50304a0>;
qcom,mdss-dsi-bl-pmic-pwm-frequency = <0x64>;
qcom,mdss-dsi-bl-pmic-bank-select = <0x00>;
qcom,mdss-dsi-pwm-gpio = <0x1a5 0x04 0x00>;
qcom,panel-supply-entries = <0x1a3>;
linux,phandle = <0x1a9>;
phandle = <0x1a9>;
};
};
References#
Loading Process of DTB Files on Qualcomm Platform - CSDN Blog
Using 9008 to Flash/Backup Single Partition - Bilibili
Qualcomm Chip Boot Process - asges Lin - Blog Garden
Software Needed for Device Flashing | Ningning's Blog
Analysis of Android Device Boot Logs - Mounting Android Boot Logs - CSDN Blog