VMPK 0.7.0 released

It has been a while! First of all: thank you very much for the million downloads of VMPK! It happened in 2016, but I wasn't paying attention at the time. I didn't post any article around that event, neither released a new VMPK version since that year, so let's cover both things.

Download statistics

Born in 2008, this project is multiplatform from the beginning, with the goal of emulating a MIDI controller device on software; which means: producing (and consuming) MIDI events. The producing part requires another source of human triggered events: computer keyboard, mouse and touch screen events are supported and available on the three main target operating systems. But producing MIDI events without delivering them to a MIDI Synthesizer makes the program useless, with no sound output at all. This has been the experience of most new VMPK Linux users for years. Windows users have a software synth included in the operating system, installed and ready to use, so they may produce sound as soon as they install VMPK from scratch. But Mac and Linux users need to find and install first a suitable synthesizer. Mac OSX also includes a soft synth library, but it is not a ready-to-use service and needs to be activated by a third party software. It's not very hard to find and install, because Apple has always made MIDI users a niche for their products. Good for them.

I've been approached last year by Microsoft people to package and publish VMPK into their Windows Store for Windows 10. Probably the million downloads has something to do with it. I've tried the packaging part without technical problems thanks to the support of the MS Desktop Bridge team, but the store conditions were awful, arbitrary and unfair, so I am not willing to comply with them. This means that VMPK will be available only outside the Windows Store and there won't be an UWP or Windows Phone version either in the future. I don't want to speak about the Apple's App Store. To the hell with them as well.

Linux users have been often made hostages of their distributions and packagers. In theory, the package repositories of each distribution (equivalent to an app store) are made for easy discovery and install of all the software available for Linux, but this is not true in practice. Software integration from different unrelated sources is also the job of the Linux distributions, which they miserably fail as well. I mean that when saying hostages, because the packagers sometimes act as if they want to force their users to follow their own personal preferences, like the gurus of a sect: Jack is the chosen one!!! PulseAudio is evil!!!

More about that later. Let's listen an enlightening Linus Torvalds first:

OK Linus. I've released this time an AppImage package for 64-bit Linux. OTOH, I'm still using Subversion and not planning to migrate the source repository to Git anytime soon.

What I think it would be a good VMPK distribution? Well, first of all, the latest version should be available for install, optionally with some former versions available to choose if the users needs to compare something (like a bug resolved or introduced in the latest version). Second: main functions should be immediately available: the user must be able to simply get sound at once. For Linux, ALSA sequencer inputs and outputs must also be available without any other extra configuration. About network I/O: the Mac OSX and Windows operating systems automatically activate firewalls blocking this function but, as soon as VMPK starts the first time, the OS asks the user for permission to open the firewall for this software. Linux distributions that include an active firewall should listen and learn something here.

The latest VMPK release includes lightweight soft synths for Linux and Mac OSX, which were available as a Drumstick-RT library back-ends for some time. The Linux soft synth is Sonivox EAS, borrowed from Android OSP and ported to Linux + PulseAudio, which was already mentioned in another post.

It is the default output chosen by VMPK upon install, so you need to use PulseAudio if you want to try VMPK. Why the PulseAudio choice? Because it is the default Linux sound server in most distributions: Ubuntu, Fedora (Red Hat), OpenSUSE, etc. I know that there has been a lot of criticism among the Linux audio community about PA, but honestly: I can't care less about arguments that sound like fanaticism. Someone told me that: “many people in linux-audio avoid PA like the plague.” Well, I'm sure you know that many computer users avoid Linux like a plague, but I don't let a handful of haters to influence my decision of keeping VMPK available for Linux users. And at the end of the day, this is my personal project and I am who decide the road-map. This is also free software. If you don't like something or have good ideas to get it better, you may contribute with code or make your own fork.

Taking Back From Android

Android is an operating system developed by Google around the Linux kernel. It is not like any other Linux distribution, because not only many common subsystems have been replaced by other components, but also the user interface is radically different based on Java language running into a virtual machine called Dalvik.

An example of subsystem removed from the Linux kernel is the ALSA Sequencer, which is a key piece for MIDI input/output with routing and scheduling that makes Linux comparable in capabilities to Mac OSX for musical applications (for musicians, not whistlers) and years ahead of Microsoft Windows in terms of infrastructure. Android did not offer anything comparable until Android 6 (Marshmallow).

Another subsystem from userspace Linux not included in Android is PulseAudio. Instead, OpenSL ES that can be found on Android for digital audio output and input.

But Android also has some shining components. One of them is Sonivox EAS (originally created by Sonic Network, Inc.) released under the Apache 2 license, and the MIDI Synthesizer used by my VMPK for Android application to produce noise. Funnily enough, it provided some legal fuel to Oracle in its battle against Google, because of some Java binding sources that were included in the AOSP repositories. It is not particularly outstanding in terms of audio quality, but has the ability of providing real time wavetable GM synthesis without using external soundfont files, and consumes very little resources so it may be indicated for Linux projects on small embedded devices. Let's take it to Linux, then!

So the plan is: for the next Drumstick release, there will be a Drumstick-RT backend using Sonivox EAS. The audio output part is yet undecided, but for Linux will probably be PulseAudio. In the same spirit, for Mac OSX there will be a backend leveraging the internal Apple DLS synth. These backends will be available in addition to the current FluidSynth one, which provides very good quality, but uses expensive floating point DSP calculations and requires external soundfont files.

Meanwhile, I've published on GitHub this repository including a port of Sonivox EAS for Linux with ALSA Sequencer MIDI input and PulseAudio output. It also  depends on Qt5 and Drumstick. Enjoy!

Sonivox EAS for Linux and Qt:
https://github.com/pedrolcl/Linux-SonivoxEas

Related Android project:
https://github.com/pedrolcl/android/tree/master/NativeGMSynth

Presenting VMPK for Nokia N950

I've been playing with my new and sexy Nokia N950 developer device, and here is the fruit: a newborn VMPK. I've just released a beta for testing, usable but not yet optimized. Please, try it. Your feedback will be appreciated.

Download VMPK & FluidSynth for N950 from sourceforge.net

I've learned two lessons from the Symbian port of VMPK published at Nokia's OVI Store: people expect that if a program looks like a piano, it should sound like a piano. It doesn't matter if the product description says that it doesn't produce any sound by itself. Dozens of comments in OVI Store page confirm that there is no hope that users read the description before downloading a program.

When I was doing some research for the Symbian port, I've discovered that creating sound always used very large audio buffers, no matter the method, producing about one second of latency or more. This is unacceptable for a musical instrument emulation, so network MIDI was the only available option. On the other hand, the Nokia N9xx uses Linux, including ALSA and PulseAudio among other usual infrastructure, so the latency is not a problem and FluidSynth is a perfectly sound complementary addition to VMPK.

Second lesson: an user interface that fits well in the desktop version of the program is barely usable on the mobile phone form factor. The solution is to create a new user interface using QML, the new declarative language for Qt user interfaces. The piano keyboard widget was already built around the Qt Graphics View Framework, so it only required to be wrapped as a QDeclarativeItem subclass and it was readily available as a QML object, to be combined with the Qt Quick Components for Meego library to build the new user interface. Here are some screenshots.

Main page, common controls are shown.

Main menu, note names option activated.

Preferences page.

About page.

Receta para PulseAudio en openSUSE

Falta menos de un mes para la publicación de openSUSE 11.4, y probablemente esta nueva versión seguirá arrastrando los mismos problemas de sonido que las versiones anteriores. Mucha gente prefiere eliminar o desactivar PulseAudio a intentar arreglarlo, pero lo cierto es que otras distribuciones (Ubuntu, Fedora, Mandriva, ...) incluyen PulseAudio activado por defecto, y los usuarios no padecen los mismos problemas. Aparte de que al desactivar PulseAudio también se pierden algunas funcionalidades interesantes. Pero en primer lugar: ¿de que problemas estamos hablando? El sonido básicamente funciona, pero cuando estás reproduciendo sonido en segundo plano y al mismo tiempo ejecutas alguna tarea larga que consume CPU, aparecen ruidos molestos que arruinan el sonido. Esto se debe a un fallo llamado "underrun", que aparece cuando el proceso que quiere reproducir sonido no es capaz de suministrar al driver ALSA del sistema los datos (buffers, fragmentos) necesarios a tiempo.

El tamaño de los fragmentos influye en la probabilidad de que aparezcan los ruidos no deseados. Cuanto más pequeño sea el tamaño de los fragmentos, más veces será necesario que el sistema interrumpa otras tareas para escribir datos en el driver de sonido, y esto también incrementa el consumo de batería en un portátil o un terminal móvil. La ventaja de los fragmentos pequeños es que se reduce la latencia de audio, es decir, el tiempo de respuesta entre la producción y la audición del sonido, lo cual puede ser importante para los juegos o los programas musicales que usan MIDI. El tamaño por defecto de los fragmentos en PulseAudio es bastante pequeño, de tan solo 25 milisegundos, y se puede configurar editando  /etc/pulse/daemon.conf

Para mantener reducido el tamaño de los fragmentos y asegurar la puntualidad y cadencia de alimentación de datos al driver de audio, una posible solución es incrementar ligeramente la prioridad del proceso que gestiona el sonido, en este caso PulseAudio. Para ello hay dos paquetes de software implicados: RealtimeKit y PolicyKit. El problema es que RealtimeKit en openSUSE está de adorno, se instala por defecto y se activa automáticamente, pero es incapaz de hacer nada por culpa de la configuración de PolicyKit, así que no solo no cumple su función, sino que malgasta recursos. El primer paso será actualizar la versión instalada del paquete "rtkit". En openSUSE 11.3 se proporcionaba la versión 0.5, y en openSUSE 11.4, la versión disponible es la... 0.5 también. Esto es inexplicable, porque la última versión publicada es la 0.9, que incorpora algunas funcionalidades necesarias no solamente para PulseAudio, sino también para otros programas como FluidSynth, Drumstick, KMid, etc...

Hay RPM de rtkit-0.9 en los repositorios de OBS
http://software.opensuse.org/search?q=rtkit&baseproject=openSUSE%3AFactory

Para comprobar que RealtimeKit funciona como debe, es conveniente   poder inspeccionar la prioridad de los hilos (threads) individuales de cada programa. Esto se puede hacer con top(1) o con ps(1). Para mayor comodidad, te propongo usar un sencillo script, que puedes guardar en ~/bin/threads:

#!/bin/bash
P=$(pidof $1)
if [ $? == 0 ]; then
    ps -O policy,rtprio,lwp -m -p $P
fi

Usando este script, es posible ver los hilos de cualquier programa en funcionamiento mediante un comando sencillo:

$ threads rtkit-daemon
  PID POL RTPRIO   LWP S TTY          TIME COMMAND
 4116 -        -     - - ?        00:00:00 /usr/lib/rtkit/rtkit-daemon
    - TS       -  4116 S -        00:00:00 -
    - TS       -  4120 S -        00:00:00 -
    - RR      99  4121 S -        00:00:00 -

La columna POL indica la política de planificación, donde TS es "time sharing", y  RR significa "round robin". La columna RTPRIO indica la prioridad del hilo. Como puedes ver, rtkit tiene un hilo con la máxima prioridad. El mismo comando contra PulseAudio cuando está correctamente configurado da el siguiente resultado:

$ threads pulseaudio
  PID POL RTPRIO   LWP S TTY          TIME COMMAND
 4114 -        -     - - ?        00:04:35 /usr/bin/pulseaudio --start
    - TS       -  4114 S -        00:00:51 -
    - RR       5  4149 S -        00:03:43 -
    - RR       5  4152 S -        00:00:00 -

En la configuración por defecto de openSUSE, los dos últimos hilos usan la política TS por defecto, y no tienen prioridad especial. Para resolverlo, hay que hacer un cambio de política sobre RealtimeKit en la configuración de PolicyKit. Para ello, añade un archivo con el siguiente contenido:

[org.freedesktop.RealtimeKit1]
Identity=unix-user:*
Action=org.freedesktop.RealtimeKit1.*
ResultAny=no
ResultInactive=yes
ResultActive=yes

Sitúa el archivo en la siguiente ubicación y nombre:
/etc/polkit-1/localauthority/50-local.d/org.freedesktop.RealtimeKit1.pkla

Para aplicar los cambios anteriores es necesario reiniciar. Nota: esta receta se aplica tanto a openSUSE 11.4rc1 como a la anterior versión publicada 11.3