Wednesday, October 12, 2011

VMPK & FluidSynth 0.1.0 released

VMPK & FluidSynth is a MeeGo Harmattan application for Nokia N9/N950 smartphones. It contains a  QML based VMPK user interface bundled with FluidSynth for sound generation.

You may download it from OVI Store right now.

Several enhancements have been included since the 0.0.1 beta announced in August.
  • Controllers, Bender, and Velocity values can be optionally controlled by the device's accelerometer.
  • Internationalization. This version includes translations to Spanish, Russian (thanks to Serguey Basalaev) and Czech (thanks to Pavel Fric).
  • Inverted color theme. This dark color combination consumes less power, enabling longer battery life.
  • Latest FluidSynth included.
Sources available at SourceForge.net, as usual.







Sunday, September 11, 2011

SoundFonts want to be free

A "SoundFont" file  (suffix .SF2) is a definition of one or several musical instruments, which can be used with synthesizers (hardware or software) to render, or convert musical notes (eg MIDI files, suffix. MID) into digital sound, which can be used by an audio interface and speakers to play music. Another file format for the same purpose is DownLoadableSounds (suffix. DLS). Both include sound samples that can be entirely synthetic or digitized from real instruments.

General MIDI is a very popular specification, that among other things define a palette of instruments. The instrument #1 is a piano, #41 a violin, #57 trumpet, #74 flute... GM SoundFonts offer 128 instruments arranged in that particular order. GS and XG are extensions of this specification.

Linux needs GM soundfonts that could be distributed together with GPL programs, similar to the need of typographic fonts for text rendering applications. Many Linux distributions incorporate the FluidR3 soundfont in their repositories. It is free and produce good quality sound, but is not small: more than 140 megabytes. MuseScore distributes "TimGM6mb" soundfont, which "only" weights 5.8 megabytes.

There are several software synthesizers using SoundFonts. Well known are FluidSynth and TiMidity++, both with free licenses. One lesser known, but no less interesting, is Gervill. It is part of OpenJDK, and therefore GPLv2 licensed. It is implemented in Java, of course. I am not very fond of Java, and I do not usually use it except for commercial projects when the customer requires their use, but this time I will do it only for fun.

Gervill can use SoundFonts, DLS, or WAV files. A very interesting feature is the so-called "Emergency Soundbank", used when no other external SoundFont is available. Definitions of this SoundFont instruments are fully synthetic and follow the GM standard.

This EmergencySoundbank is a Java class, and does not reside on a file, therefore it can not be used on another synthesizer. However, nothing prevents us from creating a Java program that instantiates the class, and stores the instrument definitions on a disk file. How complicated may this program be? Let's see:

$ cat MakeEmergencySoundfont.java
import com.sun.media.sound.*;
public class MakeEmergencySoundfont {
        public static void main(String[] args) throws Exception {
                SF2Soundbank sf2 = EmergencySoundbank.createSoundbank();
                sf2.save("GervillEmergencySoundbank.sf2");
        }
}


7 lines is not much after all. Let's compile it:

$ javac MakeEmergencySoundfont.java
MakeEmergencySoundfont.java:5: warning: com.sun.media.sound.SF2Soundbank is internal proprietary API and may be removed in a future release
                SF2Soundbank sf2 = EmergencySoundbank.createSoundbank();
                ^
MakeEmergencySoundfont.java:5: warning: com.sun.media.sound.EmergencySoundbank is internal proprietary API and may be removed in a future release
                SF2Soundbank sf2 = EmergencySoundbank.createSoundbank();
                                   ^
2 warnings


We have earned two warnings for the naughty boys, because it is ugly to directly use classes in the namespace "com.sun.media.sound.*", and if the Oracle finds out our prank, he could lock the pantry. Here, take another cookie, Neo...

To compile and use this program you only need OpenJDK6 (runtime and compiler). For older Java versions you can get a "gervill.jar" from the project website.

Running the program produces a SoundFont file:


$ java MakeEmergencySoundfont
$ ls GervillEmergencySoundbank.sf2
-rw------- 1 pedro users 1.8M 2011-09-11 12:50 GervillEmergencySoundbank.sf2


The result weights less than 2 megabytes. Of course the quality of the SoundFont is not high, but it is better than installing nothing at all by default, leaving users wondering what do they need to hear something in their programs that depend on software synthesizers.

It is interesting that a similar technique, also very simple, can be used to produce other SoundFonts based on samples of arbitrary sounds. For more details, see the MakeSoundFont example in Gervill's repository.

Finally, a question remains about what license we can use to distribute the file "GervillEmergencySoundbank.sf2" generated by our program. As a general rule, the output of a GPL program has no restrictions. However, in this case the program output does not come from processing input data, but simply dumps the results of running the algorithms included in the EmergencySoundbank class. To play it safe, we should release it as GPL.

Saturday, August 27, 2011

Presenting VMPK for Nokia N950

I've been playing with my new and sexy Nokia N950 developer device, and here is the fruit: a newborn VMPK. I've just released a beta for testing, usable but not yet optimized. Please, try it. Your feedback will be appreciated.

Download VMPK & FluidSynth for N950 from sourceforge.net

I've learned two lessons from the Symbian port of VMPK published at Nokia's OVI Store: people expect that if a program looks like a piano, it should sound like a piano. It doesn't matter if the product description says that it doesn't produce any sound by itself. Dozens of comments in OVI Store page confirm that there is no hope that users read the description before downloading a program.

When I was doing some research for the Symbian port, I've discovered that creating sound always used very large audio buffers, no matter the method, producing about one second of latency or more. This is unacceptable for a musical instrument emulation, so network MIDI was the only available option. On the other hand, the Nokia N9xx uses Linux, including ALSA and PulseAudio among other usual infrastructure, so the latency is not a problem and FluidSynth is a perfectly sound complementary addition to VMPK.

Second lesson: an user interface that fits well in the desktop version of the program is barely usable on the mobile phone form factor. The solution is to create a new user interface using QML, the new declarative language for Qt user interfaces. The piano keyboard widget was already built around the Qt Graphics View Framework, so it only required to be wrapped as a QDeclarativeItem subclass and it was readily available as a QML object, to be combined with the Qt Quick Components for Meego library to build the new user interface. Here are some screenshots.

Main page, common controls are shown.

Main menu, note names option activated.

Preferences page.

About page.

Sunday, June 5, 2011

VMPK 0.4.0 Released

This release adds touch screen support (multi-touch) for all platforms, if it is supported by the OS and hardware. New RtMidi classes have been merged, which includes Jack MIDI support and a new network MIDI driver (UDP multicast) compatible with IpMIDI and QMidiNet. A Symbian^3 port has been made. There are new translations to Dutch and Swedish. A bug crashing the program when changing the octave base has been fixed.

My suggestion is to use the packages provided by your distribution when possible. If you prefer to compile the program yourself, I would like to give you some advices to build VMPK with the new features. First of all, you need CMake 2.8. The touch screen feature is available on all platforms, thanks to Qt >= 4.6 (but is better supported in Qt 4.7). There are two new build options:
  • RTMIDI_DRIVER: selects the RtMidi backend. The default one depends on the OS. Allowed values: ALSA, JACK, COREMIDI, IRIX, WINMM, NET.
  • PROGRAM_NAME: selects the executable name.
In Linux, you can compile VMPK with ALSA, JACK or NET drivers, but only one each time. You may build the program with different names, though, if you want several versions installed at once. For instance:

$ mkdir build-jack
$ cd build-jack
$ cmake .. -DRTMIDI_DRIVER=JACK -DPROGRAM_NAME=vmpk-jack -DCMAKE_BUILD_TYPE=release
$ make

$ mkdir build-net
$ cd build-net
$ cmake .. -DRTMIDI_DRIVER=NET -DPROGRAM_NAME=vmpk-net -DCMAKE_BUILD_TYPE=release
$ make

For the network version, there is a new help page here: http://vmpk.sourceforge.net/m/


Enjoy!

Sunday, May 29, 2011

Akademy-es 2011



I've been attending the Akademy-es 2011 at Barcelona last week-end, instead of protesting and making the Spanish Revolution 2.0. It was a pleasure to meet Antonio Larrosa face to face again, after 5 years. We met the first time at Akademy-es 2006, also in Barcelona, where the idea for KMid2 was born. Hugs, Antonio!

My presentation was rejected. I've offered the same paper ("MIDI en Linux") used at the IX Jornadas de Software Libre. As an alternative, they proposed a lightning talk about KMid, but I wasn't interested in this format. My project is not KMid, but KMid is a piece of my project: providing Linux and KDE the MIDI tools that are needed, and this includes KMid, KMetronome, KMidimon, Drumstick, and some more programs.

About the conferences. Some talks were recycled, year after year, like the Valgrind talk by Albert that anyway deserves to be repeated even more times because it is a key tool to enhance the code quality of our projects. Other topics were brand new and interesting, for instance the half-rolling release and bundle system of the Chakra distribution, presented by Manuel Tortosa.

After the meetings, supper and drinks. This is the so called "social program". Everything is social, of course, when a couple of geeks meets to do things together. Saturday supper, I was shocked by a question of one of the attendees. He said to me something like this: Who wants MIDI at 2011, anyway? The answer for this question was in my paper, but he didn't bothered to read it. To summarize: MIDI is not an alternative to MP3, like drawing is not an alternative to photography. Who thinks that every program related to drawing should be banned from KDE? After all, it is a technology invented in the paleolithic era.

Saturday, May 28, 2011

Mordor attacks

How to explain yesterday's events to all the inhabitants of Middle-earth?

JULI_GARZON_27.05.2011_15
The Squad of Orcs, here called "Orcos de Esquadra", attacked a large meeting of Hobbits at Shire's Square, when they were pacifically claiming real Democracy.

sytsew_27-5-2011_28web
There have been 121 injured, including 37 orcs.

Felip Puig
There was a repulsive Nazgul commanding the Orcs, Felip Puig.

La Caixa
Orcs and his commander defend the interests of the Black Tower, where evil dwells.

Wednesday, May 4, 2011

VMPK: new release, soon

A new VMPK release is coming, with some new features:

* Touch screen interface support.

* RtMidi 1.0.14, comes with Jack MIDI support. Note that RtMidi allows only one driver, so you need to choose at configuring time the desired MIDI backend.

* Network MIDI support (UDP multicast). It may be used in Linux, Windows and Mac, but is more interesting for portable devices. The protocol is compatible with ipMIDI for Windows and Mac (http://nerds.de) and with qmidinet and multimidicast in Linux/Unix (http://qmidinet.sourceforge.net).

* Symbian^3 port.

Sunday, April 3, 2011

Symbian ports

I want to start this post with my biggest kudos to Nokia. I've been accepted in the Qt Ambassadors Program and VMPK is now in the Ambassador Showcase among other great Qt applications.

Not only that, they have sent me a great Nokia C7-00 smartphone as a gift. This device uses Symbian^3 and can be programmed using Qt. It has a touchscreen with two concurrent touch points. Just what I wanted for VMPK! So there is hope that the next VMPK release will enjoy multitouch support, thanks to Qt and Nokia.

There will be also a Symbian port of VMPK, but some tests using the device's MIDI synthesizer and a few other tests show that the audio latency is so big that it is unbearable for real time usage. Another direction to explore is the wireless network, to send MIDI data using for instance a protocol based on UDP multicast like QMidiCtl, a program by our fellow Qt Ambassador Rui Nuno Capela.

QMidiCtl was created for Maemo, and it is available for Nokia N900 devices. It can be compiled also in Linux and Windows, but required a few adaptations for Symbian. Anyway, it works well in my C7 and here is a proof


I've tested it sending MIDI events to Linux, Windows and Mac OSX. The complementary software for Linux is QMidiNet, also made by Rui. It is compatible with IpMIDI by nerds.de that is available for Windows and Mac OSX. This may open interesting possibilities for VMPK not only on smartphones, but also on netbooks and tablets...

Saturday, April 2, 2011

The MIDI Connection


How to Connect MIDI Things with FluidSynth Using Several Operating Systems

We are going to address two scenarios. First, we want to connect FluidSynth with an external USB MIDI keyboard, so we can play the piano keys producing sounds in FluidSynth. Second scenario: connect Virtual MIDI Piano Keyboard (VMPK) to FluidSynth, so we can use the mouse and the computer keyboard to produce sounds. We are going to use Qsynth instead of the command line client because its graphic user interface is much more friendly for beginners, but it could be done with the CLI fluidsynth program as well. The external USB MIDI keyboard is the Edirol PCR-M50, which works well with Linux, Windows and Mac OSX. Also FluidSynth, Qsynth and VMPK can be used in all three operating systems, which gives the opportunity to unveil and understand the differences among them comparing the same scenarios.
  1. External USB MIDI keyboard
The PCR-M50 has one MIDI IN and one MIDI OUT standard DIN-5 sockets, and also an USB socket. We are going to connect the device and the computer using an USB cable.

In Linux, ALSA includes full support for this device in the “snd-usb-audio” driver, which should work out of the box. For the other operating systems, the first step should be installing the PCR-M50 operating system drivers. The keyboard comes with a CD-ROM disk including the drivers for Windows and Mac OSX, which may be also downloaded from the manufacturer's web site: http://www.roland.com/products/en/_support/dld.cfm?PRODUCT=PCR-M50

To verify that the device is working with the operating system...
  • In Linux, open a terminal window and type the command “cat /proc/asound/seq/clients”. There should be a “PCR” client, corresponding to our device.
  • In Windows, open the Device Manager in the Control Panel, and show the “EDIROL PCR” properties under the “Sound, video and games controllers”. It should state that the device is working correctly.
  • In Mac OSX, open the “Utilities/Audio and MIDI Setup” program, and you should see the “EDIROL PCR” icon. There is a “Show device info” option, that enumerates the available ports.
The USB interface provides two output ports and three inputs.
  • PCR MIDI IN/OUT corresponding to the DIN-5 MIDI sockets on the keyboard.

  • PCR1 and PCR2, assignable to MIDI messages. By default PCR1 is used for real time messages like note on/off, pitch bend, modulation controller, etc. so we want to connect this port to the MIDI input of FluidSynth.
You can find more information provided by the manufacturer.

Now that we know that the keyboard works, let's open and configure Qsynth and FluidSynth.
  1. Qsynth configuration
You should install both Qsynth and the libfluidsynth library or DLL. Recommended versions are FluidSynth 1.1.3 and Qsynth 0.3.6 or later (it is going to be released in a few days). The details depend on your operating system. You also need a Sound Font file, like the GeneralUser SF2 by S. Christian Collins

To configure Qsynth, open the program and press the “setup...” button to configure the engine. The name provided in the “Engine Name” box becomes the tab name later in the main window.
  • In the MIDI tab:
    • Check the “Enable MIDI Input” box.
    • Select a MIDI driver: “alsa_seq” in Linux, “winmidi” in Windows and “coremidi” in Mac OSX.
    • In Linux and Mac OSX, fill the MIDI Client name ID, writing something like “GeneralUser” (or any other meaningful name of your choice).
  • In the Audio tab, select the audio driver and other settings that you know work well with your computer hardware and your operating system.
  • In the SoundFont tab, add the “GeneralUser GS FluidSynth.SF2” sound font or any other soundfont of you preference.
  • Restart the engine to apply your settings.

Scenario 1 in Linux
To connect the keyboard to Qsynth there are several ways.
  • Using a terminal, and the ALSA CLI utility “aconnect” with a command line like this:
    $ aconnect PCR:1 "FLUID Synth (GeneralUser):0"
  • Using a graphic MIDI patch bay utility, like the ALSA tab in the connections window of qjackctl, or any similar program. Anyway, you need to connect the port 1 of the PCR device to the port 0 of FluidSynth.
  • Play your keyboard.

In Linux you only need to use the patch bay utility to create the connection. Once it is established, you may close the patch bay program without disturbing the flow of MIDI data.

Scenario 1 in Windows
Plug the keyboard first, then open Qsynth and press the “setup...” button.
  • In the MIDI tab
    • Select the “winmidi” MIDI driver.
    • In the MIDI Device, select the port “1:EDIROL PCR 1”
  • Restart the engine to apply your settings.
  • Play your keyboard.
The alternative to Qsynth is the command line interface:
fluidsynth.exe -o midi.winmidi.device=”1:EDIROL PCR 1”
In Windows you don't need a patch bay utility, because FluidSynth reads directly from the operating system device. This is the only way to do it in Windows, because the operating system doesn't create virtual MIDI ports for software programs, only for system device drivers.

Scenario 1 in Mac OSX
After you finished configuring the Qsynth engine, you need a patch bay program to connect the keyboard to Qsynth, like the one at http://notahat.com/midi_patchbay
You need to create a connection from MIDI IN = “EDIROL PCR 1” to MIDI OUT = “FluidSynth virtual port (xxxx)”. You should keep the patch bay program running while you are playing, because the connection is not persistent. You may save your configuration in a file, though.
FluidSynth creates in Mac OSX a virtual MIDI port, like in Linux/ALSA, and exactly like in this case, you need to create the connection between the keyboard and the soft synth using an external tool.

Scenario 2
We are going to trigger the synthesizer sounds using the Virtual MIDI Piano Keyboard. This is a software emulation of the hardware instrument used in the former scenario. The difference is that VMPK is another software application like Qsynth/FluidSynth. More information here.

Scenario 2 in Linux
First, run Qsynth, then VMPK.
VMPK's menu Edit->Connections opens a dialog box. In “Output MIDI connection”, select the item named “FLUID Synth (xxxx):0”. You don't need any patch bay, because VMPK is already providing a connection function in this dialog. You could use an external connection manager as well, of course.

In Linux/ALSA, all MIDI programs have their own virtual MIDI ports. The virtual ports and the device ports are equivalent.

Scenario 2 in Windows
In Windows, you can't connect the MIDI output of a program to the MIDI input of another program. This mechanism is not provided by the operating system, which only allows a program to read data from a MIDI IN device, or write data to a MIDI OUT device, using of course the installed operating system drivers. To overcome this limitation, the best solution is to install virtual MIDI cables, like the ones provided by MIDI Yoke (http://www.midiox.com/myoke.htm). MIDI Yoke is an operating system driver, without any hardware device associated. You may define how much cables are created at start up using a control panel applet. Each cable has two ends: a MIDI IN and a MIDI OUT. MIDI data flows from OUT to IN, so one program can write data to cable 1, and another program can read the same data from the other end of cable 1.

First, run Qsynth and press the “setup...” button.
  • In the MIDI tab
    • Select the “winmidi” MIDI driver.
    • In the MIDI Device, select the port “In From MIDI Yoke: 1
  • Restart the engine to apply your settings.
Run VMPK, and open the menu Edit->Connections. In “Output MIDI connection:” select “Out To MIDI Yoke: 1”, and press OK. Now you can play.

Scenario 2 in Mac OSX
First, run Qsynth, then VMPK.
VMPK's menu Edit->Connections opens a dialog box. In “Output MIDI connection”, select the item named “FluidSynth virtual port (xxxx)”. You don't need any patch bay, because VMPK is already providing a connection function in this dialog. You could use an external connection manager as well, of course.

In Mac OSX, programs can create virtual MIDI ports or communicate directly to other MIDI ports. Virtual ports and physical ports are equivalent.

Monday, February 14, 2011

Receta para PulseAudio en openSUSE

Falta menos de un mes para la publicación de openSUSE 11.4, y probablemente esta nueva versión seguirá arrastrando los mismos problemas de sonido que las versiones anteriores. Mucha gente prefiere eliminar o desactivar PulseAudio a intentar arreglarlo, pero lo cierto es que otras distribuciones (Ubuntu, Fedora, Mandriva, ...) incluyen PulseAudio activado por defecto, y los usuarios no padecen los mismos problemas. Aparte de que al desactivar PulseAudio también se pierden algunas funcionalidades interesantes. Pero en primer lugar: ¿de que problemas estamos hablando? El sonido básicamente funciona, pero cuando estás reproduciendo sonido en segundo plano y al mismo tiempo ejecutas alguna tarea larga que consume CPU, aparecen ruidos molestos que arruinan el sonido. Esto se debe a un fallo llamado "underrun", que aparece cuando el proceso que quiere reproducir sonido no es capaz de suministrar al driver ALSA del sistema los datos (buffers, fragmentos) necesarios a tiempo.

El tamaño de los fragmentos influye en la probabilidad de que aparezcan los ruidos no deseados. Cuanto más pequeño sea el tamaño de los fragmentos, más veces será necesario que el sistema interrumpa otras tareas para escribir datos en el driver de sonido, y esto también incrementa el consumo de batería en un portátil o un terminal móvil. La ventaja de los fragmentos pequeños es que se reduce la latencia de audio, es decir, el tiempo de respuesta entre la producción y la audición del sonido, lo cual puede ser importante para los juegos o los programas musicales que usan MIDI. El tamaño por defecto de los fragmentos en PulseAudio es bastante pequeño, de tan solo 25 milisegundos, y se puede configurar editando  /etc/pulse/daemon.conf

Para mantener reducido el tamaño de los fragmentos y asegurar la puntualidad y cadencia de alimentación de datos al driver de audio, una posible solución es incrementar ligeramente la prioridad del proceso que gestiona el sonido, en este caso PulseAudio. Para ello hay dos paquetes de software implicados: RealtimeKit y PolicyKit. El problema es que RealtimeKit en openSUSE está de adorno, se instala por defecto y se activa automáticamente, pero es incapaz de hacer nada por culpa de la configuración de PolicyKit, así que no solo no cumple su función, sino que malgasta recursos. El primer paso será actualizar la versión instalada del paquete "rtkit". En openSUSE 11.3 se proporcionaba la versión 0.5, y en openSUSE 11.4, la versión disponible es la... 0.5 también. Esto es inexplicable, porque la última versión publicada es la 0.9, que incorpora algunas funcionalidades necesarias no solamente para PulseAudio, sino también para otros programas como FluidSynth, Drumstick, KMid, etc...

Hay RPM de rtkit-0.9 en los repositorios de OBS
http://software.opensuse.org/search?q=rtkit&baseproject=openSUSE%3AFactory

Para comprobar que RealtimeKit funciona como debe, es conveniente   poder inspeccionar la prioridad de los hilos (threads) individuales de cada programa. Esto se puede hacer con top(1) o con ps(1). Para mayor comodidad, te propongo usar un sencillo script, que puedes guardar en ~/bin/threads:

#!/bin/bash
P=$(pidof $1)
if [ $? == 0 ]; then
    ps -O policy,rtprio,lwp -m -p $P
fi

Usando este script, es posible ver los hilos de cualquier programa en funcionamiento mediante un comando sencillo:

$ threads rtkit-daemon
  PID POL RTPRIO   LWP S TTY          TIME COMMAND
 4116 -        -     - - ?        00:00:00 /usr/lib/rtkit/rtkit-daemon
    - TS       -  4116 S -        00:00:00 -
    - TS       -  4120 S -        00:00:00 -
    - RR      99  4121 S -        00:00:00 -

La columna POL indica la política de planificación, donde TS es "time sharing", y  RR significa "round robin". La columna RTPRIO indica la prioridad del hilo. Como puedes ver, rtkit tiene un hilo con la máxima prioridad. El mismo comando contra PulseAudio cuando está correctamente configurado da el siguiente resultado:

$ threads pulseaudio
  PID POL RTPRIO   LWP S TTY          TIME COMMAND
 4114 -        -     - - ?        00:04:35 /usr/bin/pulseaudio --start
    - TS       -  4114 S -        00:00:51 -
    - RR       5  4149 S -        00:03:43 -
    - RR       5  4152 S -        00:00:00 -

En la configuración por defecto de openSUSE, los dos últimos hilos usan la política TS por defecto, y no tienen prioridad especial. Para resolverlo, hay que hacer un cambio de política sobre RealtimeKit en la configuración de PolicyKit. Para ello, añade un archivo con el siguiente contenido:

[org.freedesktop.RealtimeKit1]
Identity=unix-user:*
Action=org.freedesktop.RealtimeKit1.*
ResultAny=no
ResultInactive=yes
ResultActive=yes

Sitúa el archivo en la siguiente ubicación y nombre:
/etc/polkit-1/localauthority/50-local.d/org.freedesktop.RealtimeKit1.pkla

Para aplicar los cambios anteriores es necesario reiniciar. Nota: esta receta se aplica tanto a openSUSE 11.4rc1 como a la anterior versión publicada 11.3