Sunday, April 3, 2011

Symbian ports

I want to start this post with my biggest kudos to Nokia. I've been accepted in the Qt Ambassadors Program and VMPK is now in the Ambassador Showcase among other great Qt applications.

Not only that, they have sent me a great Nokia C7-00 smartphone as a gift. This device uses Symbian^3 and can be programmed using Qt. It has a touchscreen with two concurrent touch points. Just what I wanted for VMPK! So there is hope that the next VMPK release will enjoy multitouch support, thanks to Qt and Nokia.

There will be also a Symbian port of VMPK, but some tests using the device's MIDI synthesizer and a few other tests show that the audio latency is so big that it is unbearable for real time usage. Another direction to explore is the wireless network, to send MIDI data using for instance a protocol based on UDP multicast like QMidiCtl, a program by our fellow Qt Ambassador Rui Nuno Capela.

QMidiCtl was created for Maemo, and it is available for Nokia N900 devices. It can be compiled also in Linux and Windows, but required a few adaptations for Symbian. Anyway, it works well in my C7 and here is a proof


I've tested it sending MIDI events to Linux, Windows and Mac OSX. The complementary software for Linux is QMidiNet, also made by Rui. It is compatible with IpMIDI by nerds.de that is available for Windows and Mac OSX. This may open interesting possibilities for VMPK not only on smartphones, but also on netbooks and tablets...

Saturday, April 2, 2011

The MIDI Connection


How to Connect MIDI Things with FluidSynth Using Several Operating Systems

We are going to address two scenarios. First, we want to connect FluidSynth with an external USB MIDI keyboard, so we can play the piano keys producing sounds in FluidSynth. Second scenario: connect Virtual MIDI Piano Keyboard (VMPK) to FluidSynth, so we can use the mouse and the computer keyboard to produce sounds. We are going to use Qsynth instead of the command line client because its graphic user interface is much more friendly for beginners, but it could be done with the CLI fluidsynth program as well. The external USB MIDI keyboard is the Edirol PCR-M50, which works well with Linux, Windows and Mac OSX. Also FluidSynth, Qsynth and VMPK can be used in all three operating systems, which gives the opportunity to unveil and understand the differences among them comparing the same scenarios.
  1. External USB MIDI keyboard
The PCR-M50 has one MIDI IN and one MIDI OUT standard DIN-5 sockets, and also an USB socket. We are going to connect the device and the computer using an USB cable.

In Linux, ALSA includes full support for this device in the “snd-usb-audio” driver, which should work out of the box. For the other operating systems, the first step should be installing the PCR-M50 operating system drivers. The keyboard comes with a CD-ROM disk including the drivers for Windows and Mac OSX, which may be also downloaded from the manufacturer's web site: http://www.roland.com/products/en/_support/dld.cfm?PRODUCT=PCR-M50

To verify that the device is working with the operating system...
  • In Linux, open a terminal window and type the command “cat /proc/asound/seq/clients”. There should be a “PCR” client, corresponding to our device.
  • In Windows, open the Device Manager in the Control Panel, and show the “EDIROL PCR” properties under the “Sound, video and games controllers”. It should state that the device is working correctly.
  • In Mac OSX, open the “Utilities/Audio and MIDI Setup” program, and you should see the “EDIROL PCR” icon. There is a “Show device info” option, that enumerates the available ports.
The USB interface provides two output ports and three inputs.
  • PCR MIDI IN/OUT corresponding to the DIN-5 MIDI sockets on the keyboard.

  • PCR1 and PCR2, assignable to MIDI messages. By default PCR1 is used for real time messages like note on/off, pitch bend, modulation controller, etc. so we want to connect this port to the MIDI input of FluidSynth.
You can find more information provided by the manufacturer.

Now that we know that the keyboard works, let's open and configure Qsynth and FluidSynth.
  1. Qsynth configuration
You should install both Qsynth and the libfluidsynth library or DLL. Recommended versions are FluidSynth 1.1.3 and Qsynth 0.3.6 or later (it is going to be released in a few days). The details depend on your operating system. You also need a Sound Font file, like the GeneralUser SF2 by S. Christian Collins

To configure Qsynth, open the program and press the “setup...” button to configure the engine. The name provided in the “Engine Name” box becomes the tab name later in the main window.
  • In the MIDI tab:
    • Check the “Enable MIDI Input” box.
    • Select a MIDI driver: “alsa_seq” in Linux, “winmidi” in Windows and “coremidi” in Mac OSX.
    • In Linux and Mac OSX, fill the MIDI Client name ID, writing something like “GeneralUser” (or any other meaningful name of your choice).
  • In the Audio tab, select the audio driver and other settings that you know work well with your computer hardware and your operating system.
  • In the SoundFont tab, add the “GeneralUser GS FluidSynth.SF2” sound font or any other soundfont of you preference.
  • Restart the engine to apply your settings.

Scenario 1 in Linux
To connect the keyboard to Qsynth there are several ways.
  • Using a terminal, and the ALSA CLI utility “aconnect” with a command line like this:
    $ aconnect PCR:1 "FLUID Synth (GeneralUser):0"
  • Using a graphic MIDI patch bay utility, like the ALSA tab in the connections window of qjackctl, or any similar program. Anyway, you need to connect the port 1 of the PCR device to the port 0 of FluidSynth.
  • Play your keyboard.

In Linux you only need to use the patch bay utility to create the connection. Once it is established, you may close the patch bay program without disturbing the flow of MIDI data.

Scenario 1 in Windows
Plug the keyboard first, then open Qsynth and press the “setup...” button.
  • In the MIDI tab
    • Select the “winmidi” MIDI driver.
    • In the MIDI Device, select the port “1:EDIROL PCR 1”
  • Restart the engine to apply your settings.
  • Play your keyboard.
The alternative to Qsynth is the command line interface:
fluidsynth.exe -o midi.winmidi.device=”1:EDIROL PCR 1”
In Windows you don't need a patch bay utility, because FluidSynth reads directly from the operating system device. This is the only way to do it in Windows, because the operating system doesn't create virtual MIDI ports for software programs, only for system device drivers.

Scenario 1 in Mac OSX
After you finished configuring the Qsynth engine, you need a patch bay program to connect the keyboard to Qsynth, like the one at http://notahat.com/midi_patchbay
You need to create a connection from MIDI IN = “EDIROL PCR 1” to MIDI OUT = “FluidSynth virtual port (xxxx)”. You should keep the patch bay program running while you are playing, because the connection is not persistent. You may save your configuration in a file, though.
FluidSynth creates in Mac OSX a virtual MIDI port, like in Linux/ALSA, and exactly like in this case, you need to create the connection between the keyboard and the soft synth using an external tool.

Scenario 2
We are going to trigger the synthesizer sounds using the Virtual MIDI Piano Keyboard. This is a software emulation of the hardware instrument used in the former scenario. The difference is that VMPK is another software application like Qsynth/FluidSynth. More information here.

Scenario 2 in Linux
First, run Qsynth, then VMPK.
VMPK's menu Edit->Connections opens a dialog box. In “Output MIDI connection”, select the item named “FLUID Synth (xxxx):0”. You don't need any patch bay, because VMPK is already providing a connection function in this dialog. You could use an external connection manager as well, of course.

In Linux/ALSA, all MIDI programs have their own virtual MIDI ports. The virtual ports and the device ports are equivalent.

Scenario 2 in Windows
In Windows, you can't connect the MIDI output of a program to the MIDI input of another program. This mechanism is not provided by the operating system, which only allows a program to read data from a MIDI IN device, or write data to a MIDI OUT device, using of course the installed operating system drivers. To overcome this limitation, the best solution is to install virtual MIDI cables, like the ones provided by MIDI Yoke (http://www.midiox.com/myoke.htm). MIDI Yoke is an operating system driver, without any hardware device associated. You may define how much cables are created at start up using a control panel applet. Each cable has two ends: a MIDI IN and a MIDI OUT. MIDI data flows from OUT to IN, so one program can write data to cable 1, and another program can read the same data from the other end of cable 1.

First, run Qsynth and press the “setup...” button.
  • In the MIDI tab
    • Select the “winmidi” MIDI driver.
    • In the MIDI Device, select the port “In From MIDI Yoke: 1
  • Restart the engine to apply your settings.
Run VMPK, and open the menu Edit->Connections. In “Output MIDI connection:” select “Out To MIDI Yoke: 1”, and press OK. Now you can play.

Scenario 2 in Mac OSX
First, run Qsynth, then VMPK.
VMPK's menu Edit->Connections opens a dialog box. In “Output MIDI connection”, select the item named “FluidSynth virtual port (xxxx)”. You don't need any patch bay, because VMPK is already providing a connection function in this dialog. You could use an external connection manager as well, of course.

In Mac OSX, programs can create virtual MIDI ports or communicate directly to other MIDI ports. Virtual ports and physical ports are equivalent.

Monday, February 14, 2011

Receta para PulseAudio en openSUSE

Falta menos de un mes para la publicación de openSUSE 11.4, y probablemente esta nueva versión seguirá arrastrando los mismos problemas de sonido que las versiones anteriores. Mucha gente prefiere eliminar o desactivar PulseAudio a intentar arreglarlo, pero lo cierto es que otras distribuciones (Ubuntu, Fedora, Mandriva, ...) incluyen PulseAudio activado por defecto, y los usuarios no padecen los mismos problemas. Aparte de que al desactivar PulseAudio también se pierden algunas funcionalidades interesantes. Pero en primer lugar: ¿de que problemas estamos hablando? El sonido básicamente funciona, pero cuando estás reproduciendo sonido en segundo plano y al mismo tiempo ejecutas alguna tarea larga que consume CPU, aparecen ruidos molestos que arruinan el sonido. Esto se debe a un fallo llamado "underrun", que aparece cuando el proceso que quiere reproducir sonido no es capaz de suministrar al driver ALSA del sistema los datos (buffers, fragmentos) necesarios a tiempo.

El tamaño de los fragmentos influye en la probabilidad de que aparezcan los ruidos no deseados. Cuanto más pequeño sea el tamaño de los fragmentos, más veces será necesario que el sistema interrumpa otras tareas para escribir datos en el driver de sonido, y esto también incrementa el consumo de batería en un portátil o un terminal móvil. La ventaja de los fragmentos pequeños es que se reduce la latencia de audio, es decir, el tiempo de respuesta entre la producción y la audición del sonido, lo cual puede ser importante para los juegos o los programas musicales que usan MIDI. El tamaño por defecto de los fragmentos en PulseAudio es bastante pequeño, de tan solo 25 milisegundos, y se puede configurar editando  /etc/pulse/daemon.conf

Para mantener reducido el tamaño de los fragmentos y asegurar la puntualidad y cadencia de alimentación de datos al driver de audio, una posible solución es incrementar ligeramente la prioridad del proceso que gestiona el sonido, en este caso PulseAudio. Para ello hay dos paquetes de software implicados: RealtimeKit y PolicyKit. El problema es que RealtimeKit en openSUSE está de adorno, se instala por defecto y se activa automáticamente, pero es incapaz de hacer nada por culpa de la configuración de PolicyKit, así que no solo no cumple su función, sino que malgasta recursos. El primer paso será actualizar la versión instalada del paquete "rtkit". En openSUSE 11.3 se proporcionaba la versión 0.5, y en openSUSE 11.4, la versión disponible es la... 0.5 también. Esto es inexplicable, porque la última versión publicada es la 0.9, que incorpora algunas funcionalidades necesarias no solamente para PulseAudio, sino también para otros programas como FluidSynth, Drumstick, KMid, etc...

Hay RPM de rtkit-0.9 en los repositorios de OBS
http://software.opensuse.org/search?q=rtkit&baseproject=openSUSE%3AFactory

Para comprobar que RealtimeKit funciona como debe, es conveniente   poder inspeccionar la prioridad de los hilos (threads) individuales de cada programa. Esto se puede hacer con top(1) o con ps(1). Para mayor comodidad, te propongo usar un sencillo script, que puedes guardar en ~/bin/threads:

#!/bin/bash
P=$(pidof $1)
if [ $? == 0 ]; then
    ps -O policy,rtprio,lwp -m -p $P
fi

Usando este script, es posible ver los hilos de cualquier programa en funcionamiento mediante un comando sencillo:

$ threads rtkit-daemon
  PID POL RTPRIO   LWP S TTY          TIME COMMAND
 4116 -        -     - - ?        00:00:00 /usr/lib/rtkit/rtkit-daemon
    - TS       -  4116 S -        00:00:00 -
    - TS       -  4120 S -        00:00:00 -
    - RR      99  4121 S -        00:00:00 -

La columna POL indica la política de planificación, donde TS es "time sharing", y  RR significa "round robin". La columna RTPRIO indica la prioridad del hilo. Como puedes ver, rtkit tiene un hilo con la máxima prioridad. El mismo comando contra PulseAudio cuando está correctamente configurado da el siguiente resultado:

$ threads pulseaudio
  PID POL RTPRIO   LWP S TTY          TIME COMMAND
 4114 -        -     - - ?        00:04:35 /usr/bin/pulseaudio --start
    - TS       -  4114 S -        00:00:51 -
    - RR       5  4149 S -        00:03:43 -
    - RR       5  4152 S -        00:00:00 -

En la configuración por defecto de openSUSE, los dos últimos hilos usan la política TS por defecto, y no tienen prioridad especial. Para resolverlo, hay que hacer un cambio de política sobre RealtimeKit en la configuración de PolicyKit. Para ello, añade un archivo con el siguiente contenido:

[org.freedesktop.RealtimeKit1]
Identity=unix-user:*
Action=org.freedesktop.RealtimeKit1.*
ResultAny=no
ResultInactive=yes
ResultActive=yes

Sitúa el archivo en la siguiente ubicación y nombre:
/etc/polkit-1/localauthority/50-local.d/org.freedesktop.RealtimeKit1.pkla

Para aplicar los cambios anteriores es necesario reiniciar. Nota: esta receta se aplica tanto a openSUSE 11.4rc1 como a la anterior versión publicada 11.3

Thursday, November 4, 2010

FluidSynth and QSynth: Mac OSX status.

Here is a summary of the latest changes I've made to FluidSynth and QSynth in Mac OSX.
  • CoreAudio driver now uses the HAL output AudioUnit. A new function has been added to enumerate existing hardware devices, and an option named "audio.coreaudio.device" with the same functionality as other FluidSynth hardware drivers. It can be selected in QSynth using the audio device combo box.
  • CoreMIDI driver option "midi.coremidi.id" now works much like the equivalent option of the Linux ALSA driver. Added support in QSynth for this setting.
  • FluidSynth has a new CMake boolean option: "enable-framework", default=true. It creates a Mac OSX style "FluidSynth.framework", a type of bundle that includes headers and libraries together (and other resources if required). This framework bundle is installed by default in "/Library/Frameworks". If this option is disabled at configuration time, the build system creates an Unix style dynamic library as before.
  • QSynth has also a CMake build system. It can use either the new framework or the old Unix style dylib. After compilation, you can copy/embed all the used frameworks (including FluidSynth) into the app bundle, running the Qt utility "macdeployqt". To build universal binaries, use a CMake argument like CMAKE_OSX_ARCHITECTURES=i386;ppc (all the dependencies need to be compiled using the same set of architectures).
Pending tasks: testing, packaging, documentation. Opinions?

Saturday, October 30, 2010

Good times are gone

My sabbatical is over, and all this month I have been busy on a full time job, programming Windows applications in C# for a living. For this reason I have not had much time to devote to open source projects, and the forecast is that this situation will not change in the near future.

In the past few months I have contributed as much as I could to the free software community with MIDI applications, an area I know well and where  there is not too much material, particularly on Linux. Result of this has been KMid2 and Drumstick. I have also maintained my previous projects: KMetronome and KMidimon. About VMPK, this month has beaten their previous records on SourceForge with more than 10000 downloads in October. Thank you very much to all Mac and Windows users who trusted this software, and of course to all participants, contributors and translators who made it  possible. In Debian, the situation is also "fantastic", with 137 installations to date, and only one release late. I have no data on other Linux.