Using Siri to control your Linux media center

The Raspberry Pi I bought a few days ago has just arrived. I was looking to existing Raspberry Pi projects to get some ideas on what use I could give to this board. One of the coolest projects I found out there was one that uses SiriProxy and WiringPi to open a garage door.

I thought that while deciding what to do with my Raspberry Pi I could start using it as a media center and let Siri control my multimedia apps.

Thanks to D-Bus talking to Linux apps is extremely easy. So I wrote a very simple SiriProxy plugin that controls the Rhythmbox music player using D-Bus and created a pull request in case upstream is interested in having more sample plugins.

I don’t speak Ruby so probably that won’t be the nicest Ruby code you will ever see (to say the least 🙂 )

To use the plugin just clone the master-rhythmbox branch from my SiriProxy fork on Github and follow the SiriProxy’s README set-up instructions.

Keep in mind that there is a known issue on SiriProxy master and you have to apply the fix found on this pull request.

The last two steps are to install ruby-dbus using Gems and change your .siriproxy/config.yml configuration file to include the Rhythmbox plugin.

$ sudo gem install ruby-dbus
$ echo -e "    - name: 'Rhythmbox'\n      path: './plugins/siriproxy-rhythmbox'" >> ~/.siriproxy/config.yml

That’s all, now you can run siriproxy server and Siri can start playing your favorite songs!

The voice commands currently supported are:

start player: starts the Rhythmbox media player
play, pause, stop, next and previous: to start, pause, stop, go to next and previous track.
shuffle on: turn shuffle mode on
shuffle off: turn shuffle mode off
volume up: turn volume up by 20%
volume down: turn volume down 20%

Device Tree support for IGEPv2 (and IGEP COM Module)

Because I work on an awesome company, I could spend the last few days doing upstream kernel work for the OMAP3 SoC based embedded board I maintain.

Now that SPL/U-Boot and the kernel supports the latest version of the board, the natural next step was to add support for Device Tree (DT).

I didn’t know too much about DT besides it was the new shiny way to describe hardware (instead of the infamous board files) and that it could allow to boot most ARM board using a single kernel image.

So, after spending some time learning about DT, I posted a patch-set that adds DT support for IGEP technology devices (IGEPv2 and IGEP COM Module). Hopefully the patches won’t need too much love before they get merged.

DT is a hot topic on the Linux kernel community right now and it’s been actively developed. Support for it is still premature on some platforms (such as OMAP3). So, these DT don’t have all the hardware support for IGEP that is already implemented using board files.

Currently is working (tested with latest Linux v3.7-rc7 kernel on an IGEPv2 board):
– MMC/SD
– UARTs
– GPIO LEDs
– TWL4030 codec audio
– pinmux/pinconf pinctrl

There are still peripherals not yet supported such as Ethernet, Flash memory and Wifi/BT. Some of these are connected to the OMAP processor through its General Purpose Memory Controller (GPMC) interface and patches to add DT bindings for OMAP GPMC are still flying around. So support for them should be easily added once these patches hit mainline.

It should also work on an IGEP COM Module but since I don’t have this hardware it has been just “compile tested”.

Many thanks to Enric who provided me his initial DT that booted the board, reviewed my patches and gave me lots of feedback and suggestions. Enric’s initial DT and a great explanation on how to boot a kernel with DT can be found on his blog entry.

Also, I’ve updated the oe-core/poky/yocto meta-igep layer to include a mainline Linux kernel recipe. This will make easier to test IGEP support on upstream and to add any remaining hardware enablement bits.

Adding IGEPv2 Rev.C support to mainline u-boot and kernel

I own an IGEPv2 Rev.C board which is a Texas Instrument OMAP3 DM3730 ARM embedded platform very similar to the Beagleboard-MX board but with more peripherals and a 512 NAND flash memory. This board is part of a family of embedded boards known as IGEP-based boards (another popular board from this family is the IGEP COM module).

IGEP’s manufacturer (ISEE) provides a very complete SDK and documentation on how to build a bootable image for the device and use all its features. The SDK is composed of an custom boot-loader (based on TI x-loader and u-boot), a custom kernel (based on v2.6.37 kernel from TI’s arago project), the Linaro ARM cross-toolchain and a root file system with all the necessary software and example code on how to make the best use of the board features. This makes the IGEPv2 a really competitive platform for embedded DIY projects.

Probably ISEE has its reasons to ship a custom kernel and boot-loader instead of their mainline counterparts but I (being a FOSS hacker) wanted to use upstream software on my board. So I started yet another hobby project to do on my (lately very scarce) free time to hack the kernel and u-boot and make them able to boot my IGEPv2 Rev.C board.

Fortunately, ISEE engineers have been pushing support for their IGEP board family to the mainline Linux kernel even though this kernel is not supported by their company. They do this on their free time because they understand the benefits of having their code as close as upstream as possible thus minimizing the size of the patch-set they need to carry on and forward port every time they move to a newer kernel release. But since they do on their spare time, it usually doesn’t support the latest version of their boards (such as the latest IGEPv2 Rev.C I own).

Also, ISEE used to ship u-boot as the IGEP family boot-loader before developing their own boot-loader (igep-x-loader). So it turned out that I only needed to add a few patches to the upstream kernel and u-boot to make them work with my board.

The first issue I had is that previous IGEP boards came with an OneNAND flash memory while newer boards use a NAND flash memory. So the first patch to the kernel was to to give NAND support to the IGEP. With this patch I was able to boot an upstream kernel using ISEE’s custom boot-loader.

The next step was to add the needed support to u-boot. This was a little more complicated since ISEE has not used u-boot for a while and u-boot has added recently a major feature which eliminates the need of a first stage loader (such as TI’s x-loader). Now u-boot has a framework known as SPL (Second Program Loader) that uses the same code base to generate both the first stage and second stage boot-loaders.

So, besides adding NAND support and a compile config option to choose the flash memory type to be used on an IGEP board I added SPL support for IGEP boards and updated the IGEP default kernel command line arguments to use the OMAP-specific serial driver and the EXT4 file system

With these patches my board booted with mainline u-boot and kernel but I found an issue on the IGEP kernel platform code that assumed the boot-loader would set the OMAP mux pin connected to the LAN9221i ethernet chip IRQ line as GPIO input mode (which was true with igep-x-loader but not with u-boot anymore). This should be made on the kernel anyways so the fix was just a trivial one-liner patch to the kernel.

All these patches are upstream now, so to test it just use the latest v3.6-rc4 kernel from Linus and u-boot master branch (git://git.denx.de/u-boot.git).

I had a lot of fun hacking this and my next step is to migrate the IGEP platform code to device trees. So, if anyone else owns an IGEPv2 or an IGEP COM module and is interested on using mainline software instead of ISEE custom one, please contact me so we can share the efforts 🙂

Hardware enablement

Linux 3.4 is close to be released and this version includes a device driver I was working on my free time a few months ago for the Cypress TrueTouch Gen3 family of touchscreen micro-controllers.

So, now these devices found on many embedded devices such as the HP Touchpad, Sony Xperia phone and the Nook Color e-reader are fully supported on Linux and the driver is using the stateful multi-touch protocol type B (identifiable contacts and slots) taking full advantage of the hardware finger tracking capabilities.

Many thanks to Kevin McNeely from Cypress that provided me the last version of Cypress’s Android Gingerbread driver and for answering all my questions about the device operation and hardware registers. First I thought I could just forward port this driver and post it for upstream inclusion. But after posting Cypress’s driver, I had so many change requests from the kernel hackers that I had to basically rewrite the driver (the most important issue was that it used input MT protocol type A instead of B).

The Linux multi-touch and input maintainers, Henrik Rydberg and Dmitry Torokhov helped me a *lot* reviewing my patches, explaining me the correct approach to report the contact slots to the input MT layer and pointing me out lots of issues with the driver. I wish all the Linux subsystem maintainers were as constructive and willing to help as Henrik and Dmitry.

I had fun working with the driver and now the Linux installation on my Nook Color is more close to functional. My next step is to work on the LCD panel driver that’s still not supported. A touch-screen without a panel is not very useful besides running evtest on a console to see how the input evens are reported to user-space 🙂

Finally, if you are a hardware vendor and want your device supported on Linux or have an out-of-tree driver that needs to get merged on the mainline kernel, please drop an email to sales@collabora.co.uk, we will be more than pleased to work with you to make that happen.

Netlink notifier chain

At Collabora we are working on improving the D-Bus IPC system. We are trying different approaches and one of them is implementing the D-Bus routing logic and match rules handling as a Generic Netlink service. Rodrigo wrote a great blog entry on the design of this Netlink solution.

A problem we found is that AF_NETLINK is not connection oriented. Applications don’t establish a connection before transmitting, they just open a socket and start sending messages. So, if an application close its socket, the other peers won’t notice. This also applies to the kernel code since in Generic Netlink, the in kernel service is just another user of the Netlink channel.

The problem is that D-bus is connection oriented, we have to keep track of each connection to the bus and remove it when the socket associate with the connection is closed.

Fortunately, the kernel has a mechanism known as notifier chains that allows kernel code to listen for certain events by registering to a specific notifier chain.

Netlink defines a notifier netlink_chain to notify when a socket is released (NETLINK_URELEASE event). So, we can use that information to remove the D-Bus connection associate with a socket when it is released.

The notifier API is very simple, we only have to define a notifier_block data structure and assign a function handler to the notifier_call function pointer. Once the notifier_block data structure is registered, the notifier_call function handler is called every time the events occurs.

I knew about notifier chains before but haven’t used until today so I thought it was something interesting to share. Here is a small example of how we can use the Netlink notifier chain:

static int dbus_netlink_notify(struct notifier_block *nb, unsigned long state,
                               void *_notify)
{
        struct netlink_notify *notify = _notify;

        if (state == NETLINK_URELEASE && notify->protocol == NETLINK_GENERIC)
                dbus_bus_del_connection(notify->pid);

        return NOTIFY_DONE;
}

static struct notifier_block dbus_netlink_notifier = {
	.notifier_call = dbus_netlink_notify,
};

static int __init dbus_init(void)
{
	int rc;

	rc = genl_register_family_with_ops(&dbus_family, dbus_ops,
					   ARRAY_SIZE(dbus_ops));
	if (rc)
                return rc;

	rc = netlink_register_notifier(&dbus_netlink_notifier);
	if (rc) {
                genl_unregister_family(&dbus_family);
                return rc;
	}

	return 0;
}

Using the Linaro Toolchain on Fedora

Linaro is doing a great job improving Linux on ARM and its toolchain is amazing. I’ve been using it as my primary ARM compiler for some time so I wanted to install it on my Fedora 16 box.

They don’t provide RPMs of their tools so you have to either compile it from source or use Michael Hope LSB compatible prebuilt binaries.

For anyone insterested here is a mini-how:

$ yum install redhat-lsb.i686
$ wget -c http://people.linaro.org/~michaelh/incoming/binaries/gcc-linaro-arm-linux-gnueabi-2012.01-20120112+bzr2334~linux.tar.bz2
$ tar -xvf gcc-linaro-arm-linux-gnueabi-2012.01-20120112+bzr2334~linux.tar.bz2
$ mv gcc-linaro-arm-linux-gnueabi-2012.01-20120112+bzr2334~linux /opt/gcc-linaro
$ echo "export PATH=\$PATH:/opt/gcc-linaro/bin" > /etc/profile.d/gcc-linaro.sh

What is the best way to learn Linux kernel development?

This is a question a I got a lot. Today I saw the question What’s the best way to learn device driver development on Linux? on Quora  so I spent a few minutes answering it.

The answer finally was longer than I thought so I decide that it was worth a post. I don’t know if this is the best approach to learn Linux kernel development and get involved, but is the one I recommend nowadays.

I for example didn’t follow these steps, I got involved first watching Greg KH’s Write and Submit your first Linux kernel Patch excellent video. After that I realized that anyone (even me) could contribute to the Linux kernel so I cloned the linux-next integration tree, compiled and started sending trivial one-liners patches.

But, after getting a job as a Linux Kernel Engineer and being contributing to the kernel for some time now I realized that there is a better and more fun way to get involved. Obviously device drivers is the best way to get involved for two reasons.

First, unless you are a PhD student doing research on operating systems. It is very unlikely that you will came with a brilliant idea an start hacking the Linux kernel virtual memory or block device subsystem. The Linux kernel has 20 years of development and its core is very, very complex. Second, it is more probable that your first job as a Linux kernel developer will be to write device drivers.

But, whatever is the way you chose to get involved, there is a step you can’t skip and is to learn the theory behind operating systems in general and the Linux kernel in particular. So if you don’t know about operating systems, I recommend you to read Willam Stalling’s OS book. This book has a more hardware oriented approach unlike other OS books that focus more on the algorithms and data structures used in operating systems than explaining with examples how things work in the real world.

After having a high level overview of operating systems, you have to learn about Linux kernel development. Robert Love’s Linux Kernel Development book is for me the best one on the subject. You can learn not only about Linux kernel development, but also the motivations behind the technical decisions that lead to the design.

Then you have to learn about device drivers development on Linux, two books that are very good are Linux Device Drivers and Essential Linux Device Drivers. This two books are complementary. The former teaches how to write virtual device drivers for memory based devices, so you can try the examples without the need of special hardware. While the latter shows you how device drives for real devices are actually written. I maintain an up-to-date repository with all LDD3 examples so you can compile and test on recent kernels.

Once you have the theory you can begin with the practice by writing real device drivers. You need a hardware that still is not supported on Linux. The good news is that you probably already have on of this devices.

With the popularity of the Android platform, is very likely that you own an Android device. Even when Android is a Linux-based operating system, Google by a design decision forked the Linux kernel and added some APIs that don’t exist on the Linux kernel. These APIs are used by device drivers and for that reason, a developer has to choose whether to write a device driver for Android or Linux.

So, porting Android device drivers to Linux is an excellent opportunity to learn device driver development skills, the Linux kernel development process and how to work with the Linux community while doing something useful.

A final comment, when asking questions to kernel developers on public mailing lists, remember to do your homework and researched first. They tend to be very polite and collaborative with people that show signs that they tried to understand the problem first before to ask, but can be a little grumpy with people that ask first before even tried to understand the problem.

Also, don’t be afraid to contact the developers that wrote the Android version device driver. They usually are working for the company that build the hardware of were contracted by them. Usually both the company and the original developer also want the driver to be merged on the Linux kernel but they don’t have neither the time nor the resources to do it.

So, if you offer them to help and your time to modify the driver to be in an merge-able state, they will be more than happy to help you finish the task. This is important because you probably will need the hardware datasheet or at least information about the people that know well the hardware.

Happy hacking!

Building your own Nook Color Android kernel

On the last post I told that I’m working on Cypress TrueTouch(tm) touchscreen driver for mainline inclusion.

So to work in a driver first you need to compile a Linux (or Android) kernel. Since I don’t have a fully functional Linux kernel for the Nook Color, I’m developing against an Android kernel.

I want to document how to build your own Android kernel for two reasons. First someone could find it useful. Second, I wanted to document the process for myself. I context switch more than I would like and when I comeback to something I didn’t do in a few days, I have to remember what are the exact commands to build the uImage, what is the defconfig file I use to build each kernel, etc. This happens even more with my hobbyists projects since I don’t work with them on a daily basis.

So these are the steps to build a custom Android kernel for your Nook Color:

1- Installing CyangenMod (CM7) on an SD card:

First you have to decide if you want to install your kernel on your boot partition of your nand memory or using an external SD card. I prefer to only install stable software on my nand so I did an CM7 installation on an SD card and used that for development.

To install a CM7 on your SD card first download and gunzip the generic sd card image.

$ wget http://nook.handhelds.ru/sdimage/generic-sdcard-v1.3.img.gz
$ gunzip generic-sdcard-v1.3.img.gz

Then insert your SD card and copy the image to your SD card:

$ dd if=generic-sdcard-v1.3.img of=/dev/sdX bs=1024

Where X is the block device name for your SD card. To know which block device is associated with your SD card, insert your card and look the kernel log messages with dmesg for example.
After done with writing eject and insert again your SD card on your computer. Download a nightly build of CM7 for the Nook from http://download.cyanogenmod.com/?device=encore I’m using -153 but I guess that a newer or older should work either.

$ wget http://download.cyanogenmod.com/get/cm_encore_full-153.zip

One the file has been downloaded, copy it to the SD card. There should be only one partition on the SD.

$ mount /dev/sdX1 /media/boot
$ cp cm_encore_full-153 /media/boot

That’s all, now unmount the SD card, insert it on the Nook and power off. The generic image will find and update file and automatically start CM7 installation. The process will re-partition your SD card and copy all the necessary files in each of them (x-loader MLO, uBoot u-boot.bin, kernel image and ramdisk in the boot partition, binaries and libraries in the rootfs partition, etc).
Once the installation is complete the Nook will power off and you have a fully functional CM7 installation on your SD card. The Nook Color has boot sequence when the SD card has higher precedence than the nand memory. If the SD card is inserted, the Nook will attempt to boot from there. So with the SD inserted, just turn on again the Nook an CM7 will boot from there.
That is, you now have a system to test your newer kernel.

2- Instal a ARM cross-compiling tool-chain

Since the Nook has an ARM processor and it is unlikely that your host machine is ARM base, you will need to install an ARM cross-compiling tool-chain to compile ARM Android images.
You can install different cross tool-chains both from source code and using pre-compiled packages for your host machine distribution. For example to install Linaro’s GCC cross tool-chain you can use the following command on Debian and Debian-based distros (such as Ubuntu):

$ apt-get install gcc-arm-linux-gnueabi

3- Obtaining and compiling an Android kernel

Now you need CM7 Android kernel, this can be found in Dalingrin github repository.

$ git clone https://github.com/dalingrin/nook_kernel.git

Now you have to checkout a local branch for your work:

$ cd nook_kernel
$ git checkout -b devel origin/encore-32

Now you have a local branch devel to do your kernel development.
Do all the modifications you wish to the Android kernel and to compile your kernel first generate a .config file from the Nook Color defconfig file:

$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-  omap3621_evt1a_defconfig

This will generate a .config file that contains all the symbols needed to compile Nook Color’s board file and needed drivers. You can customize the build with:

$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-  menuconfig

Once your kernel compilation is customized you can generate the kernel image with the following command:

$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-  uImage

Once it finish compiling, the uImage is located in arch/arm/boot/uImage.
The last step is to copy the uImage on the SD card. I prefer to mount the SD boot partition in the Nook Color using ssh or adb commands and copy the uImage using scp, but this setup will be explained in other article. Another option is to remove the SD card from the Nook, insert on your host machine, mounting the boot partition and copy the image:

$ mount /dev/sdX1 /media/boot
$ cp arch/arm/boot/uImage /media/boot

That is, now you can insert the SD card on your Nook Color and boot your customized Android kernel.

Happy Hacking!!

The power of open source

I own an Barnes & Nobles Nook Color e-reader. It is a fantastic device that can be rooted easily and used as a tablet pc.

Besides using with Android I wanted to install Linux on it, but there isn’t a fully functional Linux kernel for the Nook. So I contacted Oleg Drokin (a.k.a verygreen) and William Marone (a.k.a wmarone) that were working in porting Nook Color’s Android drivers to Linux.

I pick the task to work on the Cypress TrueTouch ™ touchscreen driver for upstream inclusion. I knew that Kevin McNeely from Cypress submitted a patch-set for a newer and better designed driver for the TrueTouch family. I contacted Kevin asking if I could take that patch-set as a starting point for the driver and what modifications were needed for the driver to be accepted.

Kevin kindly told me that a requirement was to modify the driver to use a different multi-touch protocol type from the one the driver had. So I modified the driver to use the newer multi-touch protocol type B. Kevin also gave me guidance, advice and code when I didn’t understand the driver internals.

I posted a patch-set which was reviewed and a few issues were found. Again Kevin point me out what modifications are necessaries to fix these issues.

So that is what I’m working on in my free time. I’ll keep modifying the driver and resubmitting the code until the patch is merged or Kevin and the folks from linux-input get tired of me.

This post is now longer than I would like, but I wanted to tell the full story to show the power of open source. I own a hardware that doesn’t meet all my needs, I’m able to hack the device, modify it at my will and find help both from hobbyists and companies like Cypress and its engineers.

I think this is the best moment in history to be an engineer, 20 years ago one could only dream of having a modern operating system such as Linux or Android with its complete source code to read, understand and modify.

I know I want to be a FOSS hacker all my life. I don’t think there could be a better way to build high quality software such as Linux.

10 years anniversary with computers

It is incredible how time flies. I just realized that I have been hanging with computers for ten years now. Looking back I’m kind of happy with my achievements. I got two academic degrees, a B.S in Computer Engineering and a M.S in High Performance Computer.

I have worked in many sectors doing different things. I have administered Linux servers, developed ERP systems for wholesale companies, design and developed enterprise mobile applications and location based services for a telco and even build a data-warehouse and a GIS for a distribution company.

Also, I’ve worked in Academia as a researcher, developing a performance analysis and estimation tool for parallel applications running in Linux clusters and published papers in international conferences.

I also contributed to my favorite open source project, the Linux kernel with 65+ patches pushed to mainline.

But the best of all is that I have finally realized what I want to do, I think I’ve found my path. I realized that I want to become a full-time FOSS hacker. My experience contributing to open source and working both in the industry and the academia, has showed me that it is near impossible to build proprietary (or academic) systems with the same quality of their open source counterparts.

As Eric Raymond said, “With many eyes, all bugs become shallow”. But not only the eyes but also the passion that open source developers put to their work and the high level of scrutiny to the code you post.

Also I see open source as a vehicle to do research, as Dirk Hohndel of Intel said, “open source is the shortest path to innovation”.

Also, the possibility that your work will be used by millions of people really inspire me. The good news is that because of my previous work with the Linux kernel, a company offered me a job to work full time as an Linux embedded engineer. That means I will be doing not only Linux kernel development but also Linux user space development and even boot-loader development!

I had to make a huge decision. Today I quit as a researcher at the Universitat Autònoma de Barcelona and give up my scholarship for PhD studies. I had the pleasure to work with very smart people while doing research. I’d like to thank my advisors, Dr Emilio Luque and Dr. Dolores Rexachs and fellows at the Computer Architecture and Operating Systems department at UAB.
But I think my decision is correct given the path I choose.

I will work hard and do my best to achieve all my goals in this new adventure. As Linus Torvalds said “Its no the great ideas that bring innovation, its sweat and hard work”

So from now on expect me to post more about Linux kernel development, embedded systems, C programming and things like that!