Thursday, June 5, 2014

Turning my old Intel Atom 330 PC into a Dreambox transcoding server

A few weeks ago I booked my holiday to France. This takes place during the World Cup Soccer Tournament in Brazil. Because I'm a huge soccer fan and I don't know if there are TV's with the right (sports) channels in the accommodations where I'm staying ( this happened before! :-( ) I investigated the possibility to use my Dreambox at home to serve a stream over the internet and watch it on my Tablet or Smartphone.

After doing some research on Google I came across some commercial Windows applications that do transcoding and are manageable on other devices. But since I don't run Windows these applications aren't useful for me. For Linux I found out that VLC and FFmpeg are capable to do the transcoding task. I tested these applications directly with the Dreambox streams over http and this worked like a charm. The only benefits of VLC is that it has a http streaming server in it and FFmpeg requires an extra setup to act like a http streaming server. So the VLC option is, in my case, the easiest.

Another challenge was to get the transcoding process stable and usable on my old Intel Atom 330 PC (my Dreambox doesn't have a transcoding chip so this can't be done on my Dreambox directly) which is always running and act as a home server. Because transcoding is a very CPU intensive process this can be an issue. After a lot of tweaking of the VLC transcoding parameters I finally managed to get a poor quality, but usable over low bandwidth!, stream which is acceptable to watch on my Tablet and Smartphone. I also tested this on my Cubietruck which has a dual-core ARM Cortex-A7 CPU, but without success. The ARM CPU is to slow to produce the poor quality stream with the same VLC parameters. Maybe the Radxa Rock Quad board does (it has a ARM Cortex-A9 Quad Core CPU), but I don't own it so I can't test it (please let me know if it does).



So having the stream running it's time to find a way to easily change channels and launch VLC to do the transcoding process. Then the idea came up to start the OpenETV (Open Enigma TV) project. The basic design of OpenETV is to talk to the Dreambox Enigma2 webinterface and control VLC. This all needs to be tied through a webinterface so it can be controlled on every device with a webbrowser on it.

I decided to write the project code in Python because of time issues (I only have a couple of evenings) and portability to easily use it cross-platform (there are many Dreambox owners who use Windows so they also can benefit of this project). By now all the basics mentioned above are implemented in OpenETV and I already use it on my Smartphone and Tablet over a VPN to my home network. It can be controlled with the build-in web interface and it's possible to watch the streams with, for example, VLC.

So bottom line: I'm now ready for my vacation! ;-)

For those who want to try/use it, I made it Open Source (GPL2/3 licensed) and it's free downloadable on GitHub:

The project page is located at:

Below is a simple setup description on how to install and use OpenETV on CentOS 6 or Fedora 20. This is also covered in the project README. If you want to run it on other distro's the steps below are similar except for the package installation part.

Because of time issues OpenETV does not support the following important features:
  • Enigma username/password authentication
  • OpenETV web authentication

I will add them later or feel free to do it yourself and send me a patch :-)


Setup and run OpenETV on CentOS 6 or Fedora 20

To install the VLC packages on CentOS 6 you need to add the rpmfusion repo:
$ sudo yum localinstall --nogpgcheck \
http://download1.rpmfusion.org/free/el/updates/6/x86_64\
/rpmfusion-free-release-6-1.noarch.rpm

If you are running Fedora 20 do the following to add the rpmfusion repo:
$ sudo yum localinstall http://download1.rpmfusion.org/free\
/fedora/rpmfusion-free-release-20.noarch.rpm

Install the VLC rpms:
$ sudo yum install -y vlc

Install the required Python libs:
$ sudo yum install -y libxml2-python

Get the OpenETV application:
$ git clone https://github.com/joeyloman/openetv.git

Configure OpenETV:
$ cd openetv

Edit openetv.py with your favorite editor and configure the following options:

Transcoding options:
As mentioned above, transcoding is a heavy CPU intensive process so you need a PC with enough power to suit your needs. In the source code header there are 2 transcoding options in the VLC stream section, named vlc_stream_options:
  1. Poor quality (this one I use on my old Intel Atom 330 fileserver and stream it over a 3G network to my Smartphone)
  2. Good quality (this one I tested on my Macbook Pro Core i7)

Choose one of them by comment/uncomment the selected option. Or if you have other needs you can configure the VLC stream options by yourself.

Enigma host:
Make sure the Enigma WebIf is enabled on your Dreambox or clone. If so, configure the ip address and the tcp port of your device in the following options, for example:

enigma2_host     = "192.168.0.5"
enigma2_port     = 80

Start OpenETV:
$ ./openetv.py start

Note: it's not recommended to start openetv as root. Instead create a service account and run openetv with reduced privileges.

Launch your browser and go to:
http://<openetv-host>:8081

Select your bouquet and channel, and click on the "start stream" button. Then launch, for example, VLC on your client device and connect it to:
http://<openetv-host>:8080

If something went wrong you can check the openetv.log file for errors. If it doesn't have enough information you can set the debug flag in openetv.py to 1 and restart it. Then repeat the browser steps and recheck the logfile for more details.

Note: OpenETV does not support Enigma user authentication at the moment. If it's enabled on your device it cannot get the bouquets.

Saturday, May 10, 2014

Running a KVM host and guest(s) on a Cubietruck


A couple of weeks ago I bought a Cubietruck to mess with ARMv7 and the ARM virtualization extension. One of my first goals was to get KVM working and run a guest with bridged network support. Because it took a lot of research, troubleshooting and kernel builds I decided to share this on my blog.

Because I'm a Fedora user this howto is based on Fedora 20 (which is the current stable at the moment).

Things you need:
  •   Cross compile build host with (in my case) Fedora 20 on it
  •   Cubietruck
  •   SD-card with minimal 8GB capacity
  •   UART TTL USB to COM Cable (usefull to control/follow the boot process and managing the console)

Prepare the build host

First create the build environment on your Fedora build host by installing these packages:
sudo yum install -y gcc-arm-linux-gnu ncurses-devel \
uboot-tools libusb libusb-devel git make wget

Create the disk image

Create a 2GB disk image with 2 partitions (later on we are gonna resize it to the full capacity of the SD card). The first partition needs to be 100MB minimal and the second contains the rest:
fallocate -l 2G fedora20-ct.img
sudo kpartx -a fedora20-ct.img
sudo fdisk /dev/loop0
n
p
1
2048
+100M
n
p
2
<ENTER>
<ENTER>
w

sudo kpartx -u fedora20-ct.img

Create the filesystems (the first partition must be ext2 which is supported by u-boot to boot from):
sudo mkfs.ext2 /dev/mapper/loop0p1
sudo mkfs.ext4 /dev/mapper/loop0p2

Mount the filesystems:
sudo mkdir /mnt/ct-boot
sudo mkdir /mnt/ct-root
sudo mount /dev/mapper/loop0p1 /mnt/ct-boot
sudo mount /dev/mapper/loop0p2 /mnt/ct-root

Note: you can also use a SATA disk for the Fedora rootfs, just mount it on /mnt/ct-root instead and configure it in uEnv.txt which is covered below in this post.

Kernel compilations

Get the latest sunxi-devel branch kernel:
git clone git://github.com/linux-sunxi/linux-sunxi.git -b \
sunxi-devel

Cross-compile the kernel with the Cubietruck host configuration:
cd <kernel sourcedir>
cp <path to>/config.txt-ct-host .config
make ARCH=arm oldconfig

If you want to add extra kernel options or drivers:
make ARCH=arm menuconfig

Build the kernel:
LOADADDR=0x40008000 make ARCH=arm \
CROSS_COMPILE=/usr/bin/arm-linux-gnu- -j4 uImage dtbs

Copy the kernel and dtb file to the boot mount:
sudo cp arch/arm/boot/uImage \
arch/arm/boot/dts/sun7i-a20-cubietruck.dtb /mnt/ct-boot

Note: you can also download the pre-compiled host kernel and dtb file from here:

Cross-compile the kernel with the QEMU guest configuration (this is handy because compiling on the cubietruck isn't very fast):
cd <kernel sourcedir>
cp <path to>/config.txt-ct-guest .config
make ARCH=arm clean
make ARCH=arm oldconfig

If you want to add extra kernel options or drivers:
make ARCH=arm menuconfig
  
Build the kernel:
make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnu- \
-j4 zImage dtbs

Copy the kernel and dtb file to the boot mount:
sudo cp arch/arm/boot/zImage \
arch/arm/boot/dts/vexpress-v2p-ca15-tc1.dtb /mnt/ct-boot

Notes:
- You can also download the pre-compiled guest kernel and dtb file from here:
- This kernel doesn't boot with SELinux support otherwise the guest will fail to boot (maybe I will fix this later).
- Above kernels are compiled with loadable module support, but they don't contain any modules in the configurations.

Build and install the u-boot bootloader

Get Hans de Goede's patched u-boot which supports ARM HYP mode:
git clone https://github.com/jwrdegoede/u-boot-sunxi.git -b \
sunxi-test

Cross-compile u-boot:
cd <u-boot source>
make cubietruck_config
make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnu- -j4 

Install the u-boot bootloader:
sudo dd if=u-boot-sunxi-with-spl.bin of=/dev/loop0 bs=1024 seek=8

Create the file "/mnt/ct-boot/uEnv.txt" with the following contents:

loglevel=9
# uncomment this if you want to use SD storage:
root=/dev/mmcblk0p2
# uncomment this if you want to use SATA storage:
#root=/dev/sda2
console=ttyS0,115200
extraargs=rw rootwait panic=10 rootfstype=ext4 rootflags=discard

Create the file "/mnt/ct-boot/boot.cmd" with the following contents:

setenv bootargs console=${console} root=${root} loglevel=${loglevel} ${panicarg} ${extraargs} 
ext2load mmc 0 0x46000000 uImage
ext2load mmc 0 0x4b000000 sun7i-a20-cubietruck.dtb
env set fdt_high ffffffff
bootm 0x46000000 - 0x4b000000

Build the boot.scr image (this contains above boot parameters which are used by u-boot):
sudo mkimage -C none -A arm -T script -d \
/mnt/ct-boot/boot.cmd /mnt/ct-boot/boot.scr

Copy the Fedora 20 files

Get the Fedora 20 ARM image:

Mount the third Fedora 20 ARM image partition:
sudo kpartx -a Fedora-Minimal-armhfp-20-1-sda.raw
sudo mkdir /mnt/f20-arm
sudo mount /dev/mapper/loop1p3 /mnt/f20-arm

Set the default root password to "fedora" in the image by changing the root line in
"/mnt/f20-arm/etc/shadow" with:

root:$6$TdrjRNbI$8ra4nFKjqj0s0cFBtEcKxugGy5VD6RheEJHEDC8CJakdB2UT2pT3kjH2uHwy9RzjyKHafb92veemXSKRxH.pf/:16198:0:99999:7:::

Note: this is handy when the Fedora initial setup fails due memory errors or if your system boots in emergency mode. 

Create the file "/tmp/rsync-excludes" which is used by rsync:

/dev
/proc
/sys
/media
/mnt
/run
/tmp

Rsync the files:
cd /mnt/f20-arm
sudo rsync -aH --exclude-from=/tmp/rsync-excludes * /mnt/ct-root
rm -f /tmp/rsync-excludes

Clear the fstab and mount the first partition on /boot2:
sudo mkdir /mnt/ct-root/boot2
sudo sh -c  "echo '/dev/mmcblk0p1 /boot2 ext2 defaults 1 1' >\ /mnt/ct-root/etc/fstab"

Make sure to relabel the SELinux file contexts on the filesystems during boot. Otherwise you can't login:
sudo touch /mnt/ct-root/.autorelabel

Unmount all partitions and remove the loop devices:
cd <path to images>
sudo umount /mnt/f20-arm
sudo kpartx -d Fedora-Minimal-armhfp-20-1-sda.raw
sudo umount /mnt/ct-root
sudo umount /mnt/ct-boot
sudo kpartx -d fedora20-ct.img

Write the image file to your SD card using dd:
dd if=<path to images>/fedora20-ct.img of=/dev/mmcblk0

Boot your Cubietruck

Put the SD-card in your Cubietruck and power it on.
The kernel only shows output on the serial interface so use the UART TTL USB to COM cable to access the console.

You can use screen for example to connect to the serial console:
sudo screen /dev/ttyUSB0 11520

If HYP mode is activated you should see in the kernel messages:
    Brought up 2 CPUs
    SMP: Total of 2 processors activated.
    CPU: All CPU(s) started in HYP mode.
    CPU: Virtualization extensions available.

Also /dev/kvm should exist. If this is not the case, see the FAQ below.

If you get the console login prompt with a python memory error above the initial setup program failed. Follow these steps to setup you system manually:

Login as root with the password "fedora" (which you set in one of the previous steps above) and create a user, sudo rights etc.

Disable the initial setup:
sudo systemctl disable initial-setup-text

Set your timezone:
sudo rm -f /etc/localtime
sudo ln -s /usr/share/zoneinfo/<area directory>/<location file> /etc/localtime

Setup the Fedora host to your needs

Set the hostname:
sudo sh -c  "echo 'cubietruck.example.org' > /etc/hostname"

Configure the network and a bridge for the guest

Stop and disable NetworkManager (this is because NetworkManager doesn't support bridge interfaces at the moment):
sudo systemctl stop NetworkManager
sudo systemctl disable NetworkManager

Enable network:
sudo systemctl enable network
    
Configure the ethernet interface by adding the following content to "/etc/sysconfig/network-scripts/ifcfg-eth0":

DEVICE=eth0
ONBOOT=yes
BRIDGE=br0
NM_CONTROLLED=no

Configure the bridge interface by adding the following content to "/etc/sysconfig/network-scripts/ifcfg-br0":

DEVICE=br0
TYPE=Bridge
BOOTPROTO=dhcp
ONBOOT=yes
DELAY=0
NM_CONTROLLED=no

Note: this configures the use of dhcp, you can also configure a static ip address if you like.

Start the network:
sudo systemctl start network
    
Disable netfilter on the bridge by adding the following content to "/etc/sysctl.d/bridge.conf":

net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

Activate the sysctl parameters:
sudo sysctl --system

Resize the second partition to the full size of the SD card:
sudo fdisk /dev/mmcblk0
d
2
n
p
2
<ENTER>
<ENTER>

Reboot the system to make the new partition visible.

Resize the rootfs to the full size of the SD card:
sudo resize2fs /dev/mmcblk0p2

Build the latest QEMU

Install the required rpm's to build and run the latest QEMU:
sudo yum install -y gcc cpp gcc-c++ pixman-devel libfdt-devel \
git zlib-devel glib2 glib2-devel uml_utilities

Build and install QEMU (I installed QEMU in /kvm because I don't want to mix it with the Fedora RPM version):
git clone git://git.qemu-project.org/qemu.git
cd qemu
./configure --prefix=/kvm --target-list=arm-softmmu \
--disable-xen --enable-kvm
make
sudo make install

Note: The Fedora 20 version of QEMU doesn't support the Cortex A7 CPU. That's why we have to build the latest version.

Create a KVM guest

Create the /kvm/images directory:
sudo mkdir -p /kvm/images

Copy the Fedora-Minimal-armhfp-20-1-sda.raw image you downloaded and extracted earlier to the Cubietruck and place it in /kvm/images.

Place the following script in the same directory as the Fedora image file and make it executable (for example: "/kvm/bin/start-guest.sh"):

#!/bin/bash
export QEMU_AUDIO_DRV=none

bridge=br0
tap=tap0

# create tap0 interface
tunctl > /dev/null 2>&1

sleep 1

# bridge tap0 with br0
brctl addif $bridge $tap  > /dev/null 2>&1

# bring the tap0 link up
ip link set $tap up > /dev/null 2>&1

echo "Guest is starting in the background.."
echo "Use "telnet localhost 1234" to connect to the console."

# start the guest with the console redirected to telnet
/kvm/bin/qemu-system-arm -enable-kvm -M vexpress-a15 -smp 2 \
-cpu host -m 1024 -kernel /boot2/zImage \
-dtb /boot2/vexpress-v2p-ca15-tc1.dtb -drive if=none,\
file=/kvm/images/Fedora-Minimal-armhfp-20-1-sda.raw,\
cache=writeback,id=foo -device virtio-blk-device,drive=foo \
-serial telnet:localhost:1234,server,nowait \
-append "rw root=/dev/vda3 rootwait physmap.enabled=0 \
console=ttyAMA0,115200n8" -net nic,vlan=0 \
-net tap,vlan=0,ifname=$tap,script=no,downscript=no \
> /dev/null 2>&1

echo "Guest is powered off. Cleaning up the tap interface."

# bring the tap0 link down
ip link set $tap down > /dev/null 2>&1

# remove tap0 from the bridge
brctl delif $bridge $tap > /dev/null 2>&1

# remove the tap0 interface
tunctl -d $tap > /dev/null 2>&1

Make it executable:
sudo chmod +x /kvm/bin/start-guest.sh

Start the KVM guest in the background:
sudo /kvm/bin/start-guest.sh &

Connect to the guest console:
telnet localhost 1234

If you get the console login prompt with a python memory error above the initial setup program failed. Follow these steps to setup you system manually:

Login as root with the password "fedora" (which you set in one of the previous steps above) and create a user, sudo rights etc.

Disable the initial setup:
sudo systemctl disable initial-setup-text

Set your timezone:
sudo rm -f /etc/localtime
sudo ln -s /usr/share/zoneinfo/<area directory>/<location file> \ /etc/localtime

Setup the Fedora guest to your needs

Set the hostname:
sudo sh -c  "echo 'cubietruck-guest.example.org' > /etc/hostname"

Configure the network

Show all connections and get the uuid:
sudo nmcli con

Delete the "Wired" connection:
sudo nmcli con del <uuid>

Add the eth0 connection:

dhcp:
sudo nmcli con add con-name eth0 ifname eth0 type ethernet

static example:
sudo nmcli con add con-name eth0 ifname eth0 type ethernet \
ip4 192.168.0.2/24 gw4 192.168.0.1
sudo nmcli con mod eth0 ipv4.dns "192.168.0.1"
sudo nmcli con down id eth0
sudo nmcli con up id eth0

That's it!



FAQ

If you see that only one CPU is activated in the kernel messages you used the wrong version of the u-boot bootloader:

    CPU1: failed to boot: -38
    Brought up 1 CPUs

Make sure you downloaded the sunxi-test branch of Hans de Goede's git repo:
git clone https://github.com/jwrdegoede/u-boot-sunxi.git -b \
sunxi-test

Then rebuild the u-boot bootloader and install it on the boot device (/dev/mmcblk0).
Or you can use this prebuild version: http://binbash.org/ct/files/u-boot-sunxi-with-spl.bin

If you get the messages following messages during boot:

  Timed out waiting for device ...
    ...
  Dependency failed for /boot ...

You probably forgot to clear the fstab.

My guest kernel doesn't boot, the screen stays black.

If this happens your guest kernel configuration isn't properly configured to boot from QEMU. I built serveral kernels based on the host kernel configuration with many added options. They all failed to boot. Eventually I took the vexpress_defconfig from the kernel source and extend it with all the needed options to run a proper Fedora system.

It's also possible that you used the wrong device tree blob for QEMU to setup the hardware. Always use the vexpress-v2p-ca15-tc1.dtb, the other ones don't work with the vexpress-a15 machine option!

Dont buy an expensive SSD drive for speed!

The internal SATA bus isn't very fast. I bought a Kingston ssdnow v300 drive for the cubietruck which should theoretically achieve a read and write speed of 450MB/s. I did some tests which resulted in a 100MB/s read speed and a 42MB/s write speed. This was also confirmed by others.