Skip to content

Broadcom CrystalHD Decoder support for FFmpeg and MPlayer

At the end of last year, Broadcom released open-source drivers and a library for their CrystalHD hardware video decoder; You can read the details about that at Jarod Wilson’s blog if you’re interested.

The hardware is particularly attractive because it’s low cost and can be added to any system, regardless of the GPU it uses. It provides MPEG1/2, H.264 and VC-1 decode capabilities in all hardware versions, and the latest 70015 part also adds MPEG4 Part 2 / DivX / XviD support – and, if you care about such things, it does so in a way that means all the infamous patent issues are handled in hardware.

Once the drivers and library were released, we started to see plenty of application support emerge, with XBMC, Xine and MythTV drivers under development early on, and a gstreamer plugin provided by Broadcom with the library. In the last couple of months, a VLC patch has been proposed. But, for all this, there was no movement on FFmpeg or Mplayer; the first being the most widely used codec library around and the later, one of the most widely used media players (and obviously a consumer of FFmpeg). I bought myself a 70012 and later a 70015 with the intention of playing around with them, but when I got some free time last month, I started working on FFmpeg and Mplayer support.

After some experimentation, I did the initial implementation as a native Mplayer decoder, which helped remove FFmpeg as a variable while I tried to get things working correctly, and after some effort, I came up with something that worked pretty well for progressive content. and partially worked for some interlaced content. In my initial discussion on the mplayer mailing list, it became clear that it needed to be an FFmpeg decoder to be maintainable for the long term. So I went back and converted it, which was relatively straight-forward as the APIs are very similar. I’ve now started getting review feedback on the FFmpeg list, and I expect it will be quite a while before it gets in, assuming it ever does, but the code is definitely usable enough to publicise more widely.

What Works

  • MPlayer playback
  • 70015 Hardware: This is the newest part, with extra codec support
  • All the officially supported content types except DivX 3.11
  • Progressive Content
  • Interlaced MPEG2 and H.264 MBAFF Content

You’ll note that I said MPlayer playback works but nothing about FFmpeg even though it’s an FFmpeg decoder. The issue here is that the hardware is pipelined, so it is normal for many frames to be in flight at one time – perhaps as many as 20 under normal conditions, and over a hundred if things start going wrong… This means that the application has to have a concept of lag in the decoding process. Now, while Mplayer uses FFmpeg’s codecs, it handles frame timing and av sync in a completely different way from the actual FFmpeg transcoding application, and it happens to do it in a way that can cope with the hardware behaviour (although I had to increase the number of frames in flight that Mplayer can handle). FFmpeg on the other hand assumes that when it submits a frame for decoding, the frame it gets back is the same one. From the discussion, it sounds like I’ll need to add the same mechanism that Mplayer uses to FFmpeg. So, for now, it’s an FFmpeg decoder that doesn’t work with FFmpeg :-)

The 70012 is the previous generation of hardware and it has some significant differences from the 70015. Beyond the reduced codec support, it has much tighter requirements with respect to keeping the pipeline happily fed, and I haven’t had a chance to investigate what this really means yet. For now, it will kind-of work, but expect something to go wrong with the pipeline and either get input overflow or output underflow pretty quickly.

Interlaced content is tricky because it can appear in many different forms, and if you look at all the other applications out there with CrystalHD support, none of them really even try to handle it. This inherent challenge is compounded by the fact that the frames don’t seem to get marked correctly when returned from the hardware, so it’s hard to tell the cases apart. I have been able to verify that MPEG2 style interlacing (where the hardware returns each field separately) and H.264 MBAFF (where the hardware returns a field pair as one frame, but with dubious flags) work correctly. H.264 PAFF should work the same as MPEG2 but I haven’t found a sample that isn’t a scary DVB broadcast stream, which introduces all sorts of other complications. There’s also a concept of interlaced content that’s marked as progressive (so you just get a field pair frame) and this should just work but I haven’t found a sample yet.

Other things of note

If you’re trying to play very high bandwidth files, make sure your I/O path is actually fast enough to move the data (so don’t expect Bluray content to work over an 802.11g link). And if your file is encoded with features that aren’t supported by the hardware, then all bets are off; the most common example of this would be an H.264 file where the encoder decided to use the highest possible settings – such as 16 b-frames on 1080p content. The CrystalHD appears to try its best to play these files but you’ll see glitches (eg: In the 16 b-frame case, it will just silently fail to resolve references to the additional b-frames)

Performance

As all codec work is done in hardware, the CPU utilization is purely based on the video resolution – almost all the time is spent copying frames back and forth. In my very unscientific tests, my old 2.2GHz Core 2 Duo laptop can play 1080p content at 25% of a core compared to 70-100% for software decoding. Also note that the X server (and window manager if you use a composited desktop) will burn measurable amounts of CPU time to display the frames. It’s supposed to be possible to do 1080p playback on a single-core Atom, but I’m not in a position the test that. Nevertheless, the benefits are clear.

Getting the code

First off, you’ll need the latest driver and userspace library from Jarod’s git tree.

Then you should grab my patched Mplayer and FFmpeg trees from github, and then construct a full Mplayer tree on disk. That means:

  • mplayer/: My mplayer tree
  • mplayer/ffmpeg/: My ffmpeg tree
  • mplayer/ffmpeg/libswcale/: From mplayerhq
  • mplayer/libdvdread4/: From svn at svn://svn.mplayerhq.hu/dvdnav/trunk/libdvdread/src
  • mplayer/libdvdnav/: From svn at //svn.mplayerhq.hu/dvdnav/trunk/libdvdnav/src

Then you should be able to build Mplayer as normal. CrystalHD support should be auto-detected and will be preferred at playback to the software decoder. In theory, mencoder should also work correctly but I haven’t tried it.

Enjoy!

Hacking the Promise NS4600

A couple of weeks ago, I bought myself a Promise NS4600 to replace my old Promise NS4300n NAS. These Promise devices are not particularly special but are cost effective and the new one performs well. As you might expect, both the old and news ones run Linux, but Promise do not allow direct access. Some people found ways in to the NS4300n and even worked out how the plugin format worked so you can extend its functionality fairly cleanly. Initial work had been done on the NS4600 but no one had documented the new plugin format or produced any plugins. So, I took it upon myself to do exactly that – I put together a guide and have linked to my initial plugins from there.

The NS4600 is pretty neat because it’s actually x86 compatible – it uses an Intel EP80579 SoC with a Pentium M core; it’s not a common sight (we’re drowning in high-end Atom based NASs) so the novelty is neat and being able to compile and run code easily is a plus.

The guide is here.

Enjoy!

Tethering Monitor (or an exploration of python and dbus)

The Bluetooth DUN package has been well received, but was hardly a profound programming endeavour. So, I’ve been trying to find suitable inspiration for a more substantial project. This morning, someone made a comment on the Bluetooth DUN page that there’s no way to tell if the phone has a tethered data connection or not – and he’s right: there’s no visual feedback on the phone, unlike Nokia’s Symbian devices or, I imagine, many other phones in the world. With that as a motivation, I decided to try and write a status indicator for tethering.

My first decision was language, and I went with Python as I’ve wanted to use it more and I know how laborious it would be to write this kind of utility in C. I then had to dig around to find out how to write a status area plugin in Python, and luckily there is a way, and it’sfairly well documented.
The biggest source of confusion is that the get_dbus_connection method isn’t exposed in the Python bindings. So, after that took *way* too much time to work out, I had to try and achieve the same thing with direct DBus calls (get a private connection that doesn’t kill the app if it dies), which I reckon I’ve got right.

Once you’ve got the basic stuff sorted out, it becomes really easy to iterate and test – you replace the python file and move .desktop file in and out of a specific directory and Hildon will reload it. Debugging was a real pain because the phone components that I’m talking too don’t exist inside the scratchbox dev environment – so I had to play a trial and error game on the N900 itself, where my only feedback was the icon failing to appear – what fun.

The next challenge was investigating what DBus interfaces to use to find the necessary information. The most important one is com.nokia.csd.GPRS. It’s not documented anywhere, but it’s fortunately introspectable and has obviously named methods and signals, so I was able to establish when a connection is made, suspended or disconnected.

Unfortunately, you see the same set of signals whether the connection is made by the phone itself or a tethered client, so then I had to find a way to detect if the phone was using the connection. I eventually found a way by using com.nokia.icd and com.nokia.icd2 – the first is undocumented and unintrospectable while the second is actually documented. For com.nokia.icd, I was able to use dbus-monitor to find a useful status signal and the get_ipinfo() method I needed had been explored by others. So, now I can avoid false positives from phone initiated connections.

There is one problem that remains, however: It’s possible to tether through the phone at the same time that the phone is using the connection for itself – this is apparently not as amazing as it sounds; all my old phones could do it. In this case, there appears to be no way to notice the tethered connection, so the monitor will not report it. At the moment, I’ve got no good ideas for doing this cleanly – I might be able to poke sysfs or look for pnatd processes, but neither is particularly attractive. But it’s not that common a case, so I consider the program useful before this gets solved.

And what does the program actually do? It shows an icon in the status area that reflects the connection state: disabled, attached, or suspended. I actually think that showing an icon when there’s no connection is a bad use of real eastate, so I’m going to take that out of the next release, but it’s there for now and helpful for confirming that the plugin actually started.

If you’re interested, you can grab it from extras-devel.

Enjoy!

N900 Bluetooth DUN package now in extras repository

Just in time for the new year, I’m pleased to be able to say that the Bluetooth DUN package is now in the Maemo Extras repository. This is the primary location for community packages that have been through a community QA process that tries to ensure the packages are safe for ‘normal’ users. If you don’t have the extras repository turned on, you can do so by following the instructions here.

Bluetooth DUN in packaged form for N900

Over the last couple of weeks, I’ve been working on packaging up my bluetooth dun script for easy consumption. It’s been through a few iterations and is now in the ‘extras-testing’ repository and should be ready to go into the main ‘extras’ repository once it has enough testing feedback.

The extras-testing repository is not intended for un-adventurous users, but if you’re interested in getting my package, it’s the place to go to. In the latest version, it will correct start the service when you install the package, and shut it down when you remove it. The actual upstart script hasn’t needed to change since I fixed the dependency issue.

And on an unrelated note, there’s now a way to reliably trigger the portrait mode ‘hack’ – if you want to try out portrait mode browsing, etc. You can find that here.

Controlling Bluetooth DUN with upstart on the n900: Part 2

As I mentioned in a quick update to my old post; I got a report of DUN not auto starting reliably, if at all. I did some digging and the cause is that the /var/run/sdp socket created by bluetoothd and needed by sdptool is not present when bluetooth-dun runs.

I’ve now updated the script to wait until the socket appears before continuing. (And as upstart is asynchronous, only the DUN service is delayed by the wait).

Now, the mechanism I used for the wait is a crude ‘while-not-exist’ loop with a one second sleep. The dbus script does this so I felt it was morally acceptable. It’s crude and an inotifywait approach would be better but that utility isn’t installed by default. Finally, the delay should really be in the bluetoothd script so that it doesn’t signal readiness until it really is…

Controlling Bluetooth DUN with upstart on the n900

Wow, it’s been a long time since I posted anything. But I’ve got something worth coming out of hibernation for.

Perhaps, unsurprisingly, I got myself an n900 and it’s a great device; I’m using it as my primary phone and it really is impressive.

One thing that doesn’t work out of the box is DUN (Dial-Up Networking) over Bluetooth. DUN is one of the simplest ways to tether a computer to a phone, so it’s a useful feature to have. (The n900 does support DUN over USB by default). Fortunately, it’s very easy to turn on, as documented on the maemo wiki. However, if you want the feature to always be ready to go (say, after you restart your phone), you need to do a little more.

Like modern versions of Ubuntu, the n900 with Maemo 5 uses upstart to control most startup services, such as bluetooth. So, if we want the DUN service to be nicely coordinated, we should start it with upstart too. Here’s my script:


description "DUN over Bluetooth"
author "Philip Langdale"

respawn
console none

start on started bluetoothd
stop on stopping bluetoothd

pre-start script
sdptool add --channel 1 DUN
end script

exec rfcomm -S -- listen -1 1 /usr/bin/pnatd '{}'

post-stop script
sdptool del `sdptool browse local | grep Dial-Up -A 1 -m 1 | tail -n 1 | cut -d ' ' -f 3`
sleep 1
end script

So, what is this doing? As upstart is pretty new, and quite different from old style init-scripts, it’s worth explaining a bit.

The description and author fields are just for documentation. respawn means to restart if the main process exits. console none means don’t log stdout or stderr anywhere.

Now, the start on and stop on directives are the heart of Upstart. They allow you to express dependencies between services, events, and each other. In this case, we want to start the DUN server after bluetoothd is started and stop it as soon as we start stopping bluetoothd. You can express multiple start and stop conditions and the upstart site documents these.

With that done, we can move on to the functional code. From the wiki page, we see that the invocation of rfcomm is the key call. What happens here is rfcomm will wait for an incoming connectio request on channel 1 and then spawn pnatd and connect it to that channel. When the connection is complete, pnatd will exit and then rfcomm will too. Upstart either tracks a particular binary or a script. In either case, it execs the binary or script and watches the resulting process to see when it exits. So, we can conveniently transfer the rfcomm command line to an upstart exec directive.

However, there’s more to do. We have to register the service with sdpd so that clients know we offer DUN, and we have to unregister when the service is terminated. This can be done with the pre-start and post-stop blocks. This also gives us a place to enforce the one second delay suggested by the example script.

Registering the service is easy, but unregistering it is a bit of a chore. The example script can avoid it because it uses the while loop, but for upstart, the entire service is ‘inside’ the loop, so we must unregister to avoid adding an extra registration each time. The problem arises because you can only unregister by the service record ID which is selected at registration time, but not provided back to us. So, we must look for it ourselves. The long command line searches the list of services for DUN and then extracts the ID.

Now, all you have to do is drop the script into /etc/event.d/ and then execute start bluetooth-dun, assuming you name the script “bluetooth-dun”. Obviously, you must be root for both these steps.

You can download the script from here. I’ll probably package it up as a deb in due course, but I don’t have a working scratchbox environment right now.

Enjoy!

Update: It seems that it’s not perfect yet. I’ve had a report, and reproduced, it failing to start when the phone boots, even though it starts reliably if you stop/start bluetoothd. My suspicion is that there’s an additional dependency (maybe the rfcomm kernel module) that needs to be accounted for. I will investigate.

High Speed SD/MMC kernel for Diablo 5.2008.43

I’m rather behind the times, but I’ve now released a matching kernel for Diablo 5.2008.43 with the highspeed SD/MMC support. Make sure to update to 5.2008.43 before installing my kernel or the update will overwrite it.

You can find my custom kernel, patches and instructions here.

High Speed SD/MMC kernel for Maemo 4.1 (Diablo)

I finally got some time to update my development tree to the new Diablo code and built a new kernel with my high-speed SD/MMC patch. Additionally, it will speed up access to the internal 2GB flash (eMMC) in the n810. For those with long memories, the Maemo bug which slows down card access when the CPU is idle is still present although there has been a little bit of activity on the Nokia side recently (They have an internal bug open for it now)

You can find my custom kernel, patches and instructions here.

MMC Specification now available for free

Wow, it’s been a long time since I’ve written anything, but I figure this is a good enough reason to do so. The SD specification has been available for quite a while now, but up until now, the MMC specification has had a $5000 price tag stuck to it. This has kept it out of most people’s hands, although Nokia were kind enough to buy a copy for Pierre. However, they have now made the latest version (4.3) available without charge. I believe this change of heart stems from the standardisation of eMMC through JEDEC (They still charge for their other specs). Despite the focus on eMMC, it is the full specification (minus the section on using MMC over SPI which they have declared obsolete; this isn’t a big deal as there’s enough documentation around explaining how MMC over SPI works).

What does this mean? Not that much in terms of Linux kernel support for MMC; Pierre already used his copy of the 4.2 spec to fix any problems and 4.3 doesn’t really add anything new that affects us. eMMC (Embedded MMC) is mostly just a new form-factor and doesn’t appear differently from a regular card. The one substantive new feature is the introduction of a special ‘boot partition’ which are accessed in a simplified way – presumably this was added to make it easy for a bootloader to load an OS off an eMMC. I don’t really see much demand emerging for this, so we have no immediate plans to support it (and good luck finding a card with a boot partition!) but if the need arose, we’d be able to do it.

So, nothing’s changed in practical terms, but it’s still helpful for us that the spec is now freely available; however, it probably won’t do much to help MMC against the SD juggernaut. Anyone seen a high capacity MMC card despite them being ‘available’ for over two years now? Thought so.