Midichlorians in the blood

Choosing MIDI or Digital Audio by Analogy

2012-01-07T19:41:00.002+01:00

Whenever I talk to someone about the relationship between MIDI and digital audio, one of my favorite analogies is that of computer images.

A digital raster image like a JPG file contains a bitmap. It is equivalent to an MP3 file containing digital audio. Both JPG and MP3 files contain quality loss compressed data, although other formats such as BMP and WAV files can contain pictures and digital sound without compression, respectively. In both cases the files store a set of digitized values. In the case of images, the data are individual pixels or dots that represent colors of the cells in a matrix of rows and columns that divide the digitized image. In the case of sound, individual data are samples that represent moments of time which divides the digitized sound. The digitization consists in dividing alike the image or sound into small fragments, the number of which depends on the resolution we want to get and the size of the scanned original.

Another type of images is called vector graphics. They are not suitable to represent photographs, but drawings. SVG files that are used in many illustrations of Wikipedia are of this type. Instead of image fragments, they contain symbolic descriptions using coordinates of points, distances, lines, and colors... They have the advantage of scalability without loss of quality, and ease of arbitrary modification of some of its components and properties without affecting the rest. The equivalent of this technology in the world of sound is MIDI. A MIDI sequence contains timestamped messages such as notes, instrument changes, controls, etc.. Not a proper format for storing sounds recorded by a microphone, but a symbolic representation of music similar to a score.

Images are two dimensional objects, so the digitized images consist of rows and columns of elements (pixels), and the position of the elements of a drawing is characterized by a pair of numbers that represent its Cartesian coordinates. On the other hand sound recordings are one-dimensional, sound samples are taken at constant time intervals and also MIDI messages are labeled by their position in the time line.

The above similarities have implications that reflect additional parallelism. An uncompressed digitized image consisting of any single solid color takes the same amount of memory than an image of the same size representing a photograph or a complex composition of multiple colors. Similarly, a recording of silence (for example John Cage's 4'33'') takes the same amount of memory than any symphonic piece of the same duration. On the other hand, a simple vector image takes much less memory than a complex picture of the same dimensions. And a few notes MIDI sequence occupies much less memory than a complex sequence of the same duration made up of many notes or other messages.

The problems posed by digital images and sounds on stretch and reduction of dimensions are also similar. In both cases artifacts are generated, an effect known as 'aliasing', which can be offset to some extent by using 'antialiasing' filters. On the other hand, in the case of vector graphics as MIDI sequences, you can easily perform stretching and shrinking of dimensions and duration without risking artifacts or quality loss whatsoever.

Starting from a vector image, it is necessary a rendering engine to get a digital image that can be displayed on the screen or a printer. In the case of MIDI, a sequencer and a MIDI synthesizer are required to produce digital audio that can be used by an audio interface.

The programs Inkscape and Gimp, used in Linux for creating and editing vector graphics and digital images respectively, are comparable to the Adobe programs Illustrator and Photoshop. They cover different needs and audience, thriving on different niches. An example of this type of niche is the architects, who use vector graphics to design and represent buildings with Autocad or similar programs. These are not watertight compartments. Gimp can import vector graphic files, rendering them as bitmaps. Inkscape can also import a bitmap image as a drawing object. In each case, the users may choose the best tool for each task.

While it has been easy to list some essential image processing programs for Linux and other systems, to do the same exercise in the field of audio and MIDI is much more risky. The problem is that the way musicians work with computers is not homogeneous, with each musician working in a different way. For old school types the ideal work-flow is to note down musical ideas, develop drafts and refine compositions using tools that work with symbolic elements, producing as a final result a paper copy of the score. Rosegarden could be appropriate at this stage. On the other extreme, there are those who never in his life read or write a score, and whose only tools of creation (other than musical instruments) are the mixer and multi-track recorder. In this case, Ardour could be right.

The two applications mentioned above allow the use of digital audio and MIDI at the same time. In the same way as in the world of images, some applications are focused on the symbolic representation (MIDI) and others in a final product (digital audio). In each case, the use of the other technology will be subordinate. For instance, Ardour MIDI messages are aligned to the audio samples. It has even developed an API (Jack MIDI) to ensure synchronization of MIDI events to digital audio samples, subordinating MIDI to the rules of digital audio. Obviously this strategy does not fit adequately on all scenarios where MIDI is useful.

As in the imaging world, symbolic representation (MIDI) is probably better suited for design, drafting and composition. By contrast, digital audio is the dominant technology in the studio, at mixing stage and production, to obtain a finished product.

VMPK & FluidSynth 0.1.0 released

2011-10-12T00:10:00.000+02:00

VMPK & FluidSynth is a MeeGo Harmattan application for Nokia N9/N950 smartphones. It contains a QML based VMPK user interface bundled with FluidSynth for sound generation.

You may download it from OVI Store right now.

Several enhancements have been included since the 0.0.1 beta announced in August.

Controllers, Bender, and Velocity values can be optionally controlled by the device's accelerometer.

Internationalization. This version includes translations to Spanish, Russian (thanks to Serguey Basalaev) and Czech (thanks to Pavel Fric).

Inverted color theme. This dark color combination consumes less power, enabling longer battery life.

Latest FluidSynth included.

Sources available at SourceForge.net, as usual.

SoundFonts want to be free

2011-09-11T16:11:00.002+02:00

A "SoundFont" file (suffix .SF2) is a definition of one or several musical instruments, which can be used with synthesizers (hardware or software) to render, or convert musical notes (eg MIDI files, suffix. MID) into digital sound, which can be used by an audio interface and speakers to play music. Another file format for the same purpose is DownLoadableSounds (suffix. DLS). Both include sound samples that can be entirely synthetic or digitized from real instruments.

General MIDI is a very popular specification, that among other things define a palette of instruments. The instrument #1 is a piano, #41 a violin, #57 trumpet, #74 flute... GM SoundFonts offer 128 instruments arranged in that particular order. GS and XG are extensions of this specification.

Linux needs GM soundfonts that could be distributed together with GPL programs, similar to the need of typographic fonts for text rendering applications. Many Linux distributions incorporate the FluidR3 soundfont in their repositories. It is free and produce good quality sound, but is not small: more than 140 megabytes. MuseScore distributes "TimGM6mb" soundfont, which "only" weights 5.8 megabytes.

There are several software synthesizers using SoundFonts. Well known are FluidSynth and TiMidity++, both with free licenses. One lesser known, but no less interesting, is Gervill. It is part of OpenJDK, and therefore GPLv2 licensed. It is implemented in Java, of course. I am not very fond of Java, and I do not usually use it except for commercial projects when the customer requires their use, but this time I will do it only for fun.

Gervill can use SoundFonts, DLS, or WAV files. A very interesting feature is the so-called "Emergency Soundbank", used when no other external SoundFont is available. Definitions of this SoundFont instruments are fully synthetic and follow the GM standard.

This EmergencySoundbank is a Java class, and does not reside on a file, therefore it can not be used on another synthesizer. However, nothing prevents us from creating a Java program that instantiates the class, and stores the instrument definitions on a disk file. How complicated may this program be? Let's see:

$ cat MakeEmergencySoundfont.java 
import com.sun.media.sound.*;
public class MakeEmergencySoundfont {
        public static void main(String[] args) throws Exception {
                SF2Soundbank sf2 = EmergencySoundbank.createSoundbank();
                sf2.save("GervillEmergencySoundbank.sf2");
        }
}

7 lines is not much after all. Let's compile it:

$ javac MakeEmergencySoundfont.java 
MakeEmergencySoundfont.java:5: warning: com.sun.media.sound.SF2Soundbank is internal proprietary API and may be removed in a future release
                SF2Soundbank sf2 = EmergencySoundbank.createSoundbank();
                ^
MakeEmergencySoundfont.java:5: warning: com.sun.media.sound.EmergencySoundbank is internal proprietary API and may be removed in a future release
                SF2Soundbank sf2 = EmergencySoundbank.createSoundbank();
                                   ^
2 warnings

We have earned two warnings for the naughty boys, because it is ugly to directly use classes in the namespace "com.sun.media.sound.*", and if the Oracle finds out our prank, he could lock the pantry. Here, take another cookie, Neo...

To compile and use this program you only need OpenJDK6 (runtime and compiler). For older Java versions you can get a "gervill.jar" from the project website.

Running the program produces a SoundFont file:


$ java MakeEmergencySoundfont
$ ls GervillEmergencySoundbank.sf2 
-rw------- 1 pedro users 1.8M 2011-09-11 12:50 GervillEmergencySoundbank.sf2

The result weights less than 2 megabytes. Of course the quality of the SoundFont is not high, but it is better than installing nothing at all by default, leaving users wondering what do they need to hear something in their programs that depend on software synthesizers.

It is interesting that a similar technique, also very simple, can be used to produce other SoundFonts based on samples of arbitrary sounds. For more details, see the MakeSoundFont example in Gervill's repository.

Finally, a question remains about what license we can use to distribute the file "GervillEmergencySoundbank.sf2" generated by our program. As a general rule, the output of a GPL program has no restrictions. However, in this case the program output does not come from processing input data, but simply dumps the results of running the algorithms included in the EmergencySoundbank class. To play it safe, we should release it as GPL.

Presenting VMPK for Nokia N950

2011-08-27T16:01:00.000+02:00

I've been playing with my new and sexy Nokia N950 developer device, and here is the fruit: a newborn VMPK. I've just released a beta for testing, usable but not yet optimized. Please, try it. Your feedback will be appreciated.

Download VMPK & FluidSynth for N950 from sourceforge.net

I've learned two lessons from the Symbian port of VMPK published at Nokia's OVI Store: people expect that if a program looks like a piano, it should sound like a piano. It doesn't matter if the product description says that it doesn't produce any sound by itself. Dozens of comments in OVI Store page confirm that there is no hope that users read the description before downloading a program.

When I was doing some research for the Symbian port, I've discovered that creating sound always used very large audio buffers, no matter the method, producing about one second of latency or more. This is unacceptable for a musical instrument emulation, so network MIDI was the only available option. On the other hand, the Nokia N9xx uses Linux, including ALSA and PulseAudio among other usual infrastructure, so the latency is not a problem and FluidSynth is a perfectly sound complementary addition to VMPK.

Second lesson: an user interface that fits well in the desktop version of the program is barely usable on the mobile phone form factor. The solution is to create a new user interface using QML, the new declarative language for Qt user interfaces. The piano keyboard widget was already built around the Qt Graphics View Framework, so it only required to be wrapped as a QDeclarativeItem subclass and it was readily available as a QML object, to be combined with the Qt Quick Components for Meego library to build the new user interface. Here are some screenshots.

Main page, common controls are shown.

Main menu, note names option activated.

Preferences page.


About page.

VMPK 0.4.0 Released

2011-06-05T17:51:00.001+02:00

This release adds touch screen support (multi-touch) for all platforms, if it is supported by the OS and hardware. New RtMidi classes have been merged, which includes Jack MIDI support and a new network MIDI driver (UDP multicast) compatible with IpMIDI and QMidiNet. A Symbian^3 port has been made. There are new translations to Dutch and Swedish. A bug crashing the program when changing the octave base has been fixed.

My suggestion is to use the packages provided by your distribution when possible. If you prefer to compile the program yourself, I would like to give you some advices to build VMPK with the new features. First of all, you need CMake 2.8. The touch screen feature is available on all platforms, thanks to Qt >= 4.6 (but is better supported in Qt 4.7). There are two new build options:

RTMIDI_DRIVER: selects the RtMidi backend. The default one depends on the OS. Allowed values: ALSA, JACK, COREMIDI, IRIX, WINMM, NET.

PROGRAM_NAME: selects the executable name.

In Linux, you can compile VMPK with ALSA, JACK or NET drivers, but only one each time. You may build the program with different names, though, if you want several versions installed at once. For instance:

$ mkdir build-jack
$ cd build-jack
$ cmake .. -DRTMIDI_DRIVER=JACK -DPROGRAM_NAME=vmpk-jack -DCMAKE_BUILD_TYPE=release
$ make

$ mkdir build-net
$ cd build-net
$ cmake .. -DRTMIDI_DRIVER=NET -DPROGRAM_NAME=vmpk-net -DCMAKE_BUILD_TYPE=release
$ make

For the network version, there is a new help page here: http://vmpk.sourceforge.net/m/

Enjoy!

Akademy-es 2011

2011-05-29T23:07:00.000+02:00

I've been attending the Akademy-es 2011 at Barcelona last week-end, instead of protesting and making the Spanish Revolution 2.0. It was a pleasure to meet Antonio Larrosa face to face again, after 5 years. We met the first time at Akademy-es 2006, also in Barcelona, where the idea for KMid2 was born. Hugs, Antonio!

My presentation was rejected. I've offered the same paper ("MIDI en Linux") used at the IX Jornadas de Software Libre. As an alternative, they proposed a lightning talk about KMid, but I wasn't interested in this format. My project is not KMid, but KMid is a piece of my project: providing Linux and KDE the MIDI tools that are needed, and this includes KMid, KMetronome, KMidimon, Drumstick, and some more programs.

About the conferences. Some talks were recycled, year after year, like the Valgrind talk by Albert that anyway deserves to be repeated even more times because it is a key tool to enhance the code quality of our projects. Other topics were brand new and interesting, for instance the half-rolling release and bundle system of the Chakra distribution, presented by Manuel Tortosa.

After the meetings, supper and drinks. This is the so called "social program". Everything is social, of course, when a couple of geeks meets to do things together. Saturday supper, I was shocked by a question of one of the attendees. He said to me something like this: Who wants MIDI at 2011, anyway? The answer for this question was in my paper, but he didn't bothered to read it. To summarize: MIDI is not an alternative to MP3, like drawing is not an alternative to photography. Who thinks that every program related to drawing should be banned from KDE? After all, it is a technology invented in the paleolithic era.

Mordor attacks

2011-05-28T15:42:00.001+02:00

How to explain yesterday's events to all the inhabitants of Middle-earth?

The Squad of Orcs, here called "Orcos de Esquadra", attacked a large meeting of Hobbits at Shire's Square, when they were pacifically claiming real Democracy.

There have been 121 injured, including 37 orcs.

There was a repulsive Nazgul commanding the Orcs, Felip Puig.

Orcs and his commander defend the interests of the Black Tower, where evil dwells.

VMPK: new release, soon

2011-05-04T23:32:00.001+02:00

A new VMPK release is coming, with some new features:

* Touch screen interface support.

* RtMidi 1.0.14, comes with Jack MIDI support. Note that RtMidi allows only one driver, so you need to choose at configuring time the desired MIDI backend.

* Network MIDI support (UDP multicast). It may be used in Linux, Windows and Mac, but is more interesting for portable devices. The protocol is compatible with ipMIDI for Windows and Mac (http://nerds.de) and with qmidinet and multimidicast in Linux/Unix (http://qmidinet.sourceforge.net).

* Symbian^3 port.

Symbian ports

2011-04-03T00:37:00.000+02:00

I want to start this post with my biggest kudos to Nokia. I've been accepted in the Qt Ambassadors Program and VMPK is now in the Ambassador Showcase among other great Qt applications.

Not only that, they have sent me a great Nokia C7-00 smartphone as a gift. This device uses Symbian^3 and can be programmed using Qt. It has a touchscreen with two concurrent touch points. Just what I wanted for VMPK! So there is hope that the next VMPK release will enjoy multitouch support, thanks to Qt and Nokia.

There will be also a Symbian port of VMPK, but some tests using the device's MIDI synthesizer and a few other tests show that the audio latency is so big that it is unbearable for real time usage. Another direction to explore is the wireless network, to send MIDI data using for instance a protocol based on UDP multicast like QMidiCtl, a program by our fellow Qt Ambassador Rui Nuno Capela.

QMidiCtl was created for Maemo, and it is available for Nokia N900 devices. It can be compiled also in Linux and Windows, but required a few adaptations for Symbian. Anyway, it works well in my C7 and here is a proof

I've tested it sending MIDI events to Linux, Windows and Mac OSX. The complementary software for Linux is QMidiNet, also made by Rui. It is compatible with IpMIDI by nerds.de that is available for Windows and Mac OSX. This may open interesting possibilities for VMPK not only on smartphones, but also on netbooks and tablets...

The MIDI Connection

2011-04-02T01:15:00.001+02:00

How to Connect MIDI Things with FluidSynth Using Several Operating Systems

We are going to address two scenarios. First, we want to connect FluidSynth with an external USB MIDI keyboard, so we can play the piano keys producing sounds in FluidSynth. Second scenario: connect Virtual MIDI Piano Keyboard (VMPK) to FluidSynth, so we can use the mouse and the computer keyboard to produce sounds. We are going to use Qsynth instead of the command line client because its graphic user interface is much more friendly for beginners, but it could be done with the CLI fluidsynth program as well. The external USB MIDI keyboard is the Edirol PCR-M50, which works well with Linux, Windows and Mac OSX. Also FluidSynth, Qsynth and VMPK can be used in all three operating systems, which gives the opportunity to unveil and understand the differences among them comparing the same scenarios.

External USB MIDI keyboard

The PCR-M50 has one MIDI IN and one MIDI OUT standard DIN-5 sockets, and also an USB socket. We are going to connect the device and the computer using an USB cable.

In Linux, ALSA includes full support for this device in the “snd-usb-audio” driver, which should work out of the box. For the other operating systems, the first step should be installing the PCR-M50 operating system drivers. The keyboard comes with a CD-ROM disk including the drivers for Windows and Mac OSX, which may be also downloaded from the manufacturer's web site: http://www.roland.com/products/en/_support/dld.cfm?PRODUCT=PCR-M50

To verify that the device is working with the operating system...

In Linux, open a terminal window and type the command “cat /proc/asound/seq/clients”. There should be a “PCR” client, corresponding to our device.

In Windows, open the Device Manager in the Control Panel, and show the “EDIROL PCR” properties under the “Sound, video and games controllers”. It should state that the device is working correctly.
In Mac OSX, open the “Utilities/Audio and MIDI Setup” program, and you should see the “EDIROL PCR” icon. There is a “Show device info” option, that enumerates the available ports.

The USB interface provides two output ports and three inputs.

PCR MIDI IN/OUT corresponding to the DIN-5 MIDI sockets on the keyboard.
PCR1 and PCR2, assignable to MIDI messages. By default PCR1 is used for real time messages like note on/off, pitch bend, modulation controller, etc. so we want to connect this port to the MIDI input of FluidSynth.

You can find more information provided by the manufacturer.

http://www.rolandus.com/products/productdetails.php?ProductId=706

Now that we know that the keyboard works, let's open and configure Qsynth and FluidSynth.

Qsynth configuration

You should install both Qsynth and the libfluidsynth library or DLL. Recommended versions are FluidSynth 1.1.3 and Qsynth 0.3.6 or later (it is going to be released in a few days). The details depend on your operating system. You also need a Sound Font file, like the GeneralUser SF2 by S. Christian Collins

To configure Qsynth, open the program and press the “setup...” button to configure the engine. The name provided in the “Engine Name” box becomes the tab name later in the main window.

In the MIDI tab:

Check the “Enable MIDI Input” box.
Select a MIDI driver: “alsa_seq” in Linux, “winmidi” in Windows and “coremidi” in Mac OSX.
In Linux and Mac OSX, fill the MIDI Client name ID, writing something like “GeneralUser” (or any other meaningful name of your choice).

In the Audio tab, select the audio driver and other settings that you know work well with your computer hardware and your operating system.
In the SoundFont tab, add the “GeneralUser GS FluidSynth.SF2” sound font or any other soundfont of you preference.
Restart the engine to apply your settings.

Scenario 1 in Linux

To connect the keyboard to Qsynth there are several ways.

Using a terminal, and the ALSA CLI utility “aconnect” with a command line like this:
$ aconnect PCR:1 "FLUID Synth (GeneralUser):0"
Using a graphic MIDI patch bay utility, like the ALSA tab in the connections window of qjackctl, or any similar program. Anyway, you need to connect the port 1 of the PCR device to the port 0 of FluidSynth.
Play your keyboard.

In Linux you only need to use the patch bay utility to create the connection. Once it is established, you may close the patch bay program without disturbing the flow of MIDI data.

Scenario 1 in Windows

Plug the keyboard first, then open Qsynth and press the “setup...” button.

In the MIDI tab
- Select the “winmidi” MIDI driver.
- In the MIDI Device, select the port “1:EDIROL PCR 1”
Restart the engine to apply your settings.
Play your keyboard.

The alternative to Qsynth is the command line interface:

fluidsynth.exe -o midi.winmidi.device=”1:EDIROL PCR 1”

In Windows you don't need a patch bay utility, because FluidSynth reads directly from the operating system device. This is the only way to do it in Windows, because the operating system doesn't create virtual MIDI ports for software programs, only for system device drivers.

Scenario 1 in Mac OSX

After you finished configuring the Qsynth engine, you need a patch bay program to connect the keyboard to Qsynth, like the one at http://notahat.com/midi_patchbay

You need to create a connection from MIDI IN = “EDIROL PCR 1” to MIDI OUT = “FluidSynth virtual port (xxxx)”. You should keep the patch bay program running while you are playing, because the connection is not persistent. You may save your configuration in a file, though.

FluidSynth creates in Mac OSX a virtual MIDI port, like in Linux/ALSA, and exactly like in this case, you need to create the connection between the keyboard and the soft synth using an external tool.

Scenario 2

We are going to trigger the synthesizer sounds using the Virtual MIDI Piano Keyboard. This is a software emulation of the hardware instrument used in the former scenario. The difference is that VMPK is another software application like Qsynth/FluidSynth. More information here.

Scenario 2 in Linux

First, run Qsynth, then VMPK.

VMPK's menu Edit->Connections opens a dialog box. In “Output MIDI connection”, select the item named “FLUID Synth (xxxx):0”. You don't need any patch bay, because VMPK is already providing a connection function in this dialog. You could use an external connection manager as well, of course.

In Linux/ALSA, all MIDI programs have their own virtual MIDI ports. The virtual ports and the device ports are equivalent.

Scenario 2 in Windows

In Windows, you can't connect the MIDI output of a program to the MIDI input of another program. This mechanism is not provided by the operating system, which only allows a program to read data from a MIDI IN device, or write data to a MIDI OUT device, using of course the installed operating system drivers. To overcome this limitation, the best solution is to install virtual MIDI cables, like the ones provided by MIDI Yoke (http://www.midiox.com/myoke.htm). MIDI Yoke is an operating system driver, without any hardware device associated. You may define how much cables are created at start up using a control panel applet. Each cable has two ends: a MIDI IN and a MIDI OUT. MIDI data flows from OUT to IN, so one program can write data to cable 1, and another program can read the same data from the other end of cable 1.

First, run Qsynth and press the “setup...” button.

In the MIDI tab

Select the “winmidi” MIDI driver.
In the MIDI Device, select the port “In From MIDI Yoke: 1

Restart the engine to apply your settings.

Run VMPK, and open the menu Edit->Connections. In “Output MIDI connection:” select “Out To MIDI Yoke: 1”, and press OK. Now you can play.

Scenario 2 in Mac OSX

First, run Qsynth, then VMPK.

VMPK's menu Edit->Connections opens a dialog box. In “Output MIDI connection”, select the item named “FluidSynth virtual port (xxxx)”. You don't need any patch bay, because VMPK is already providing a connection function in this dialog. You could use an external connection manager as well, of course.

In Mac OSX, programs can create virtual MIDI ports or communicate directly to other MIDI ports. Virtual ports and physical ports are equivalent.

Receta para PulseAudio en openSUSE

2011-02-14T00:17:00.000+01:00

Falta menos de un mes para la publicación de openSUSE 11.4, y probablemente esta nueva versión seguirá arrastrando los mismos problemas de sonido que las versiones anteriores. Mucha gente prefiere eliminar o desactivar PulseAudio a intentar arreglarlo, pero lo cierto es que otras distribuciones (Ubuntu, Fedora, Mandriva, ...) incluyen PulseAudio activado por defecto, y los usuarios no padecen los mismos problemas. Aparte de que al desactivar PulseAudio también se pierden algunas funcionalidades interesantes. Pero en primer lugar: ¿de que problemas estamos hablando? El sonido básicamente funciona, pero cuando estás reproduciendo sonido en segundo plano y al mismo tiempo ejecutas alguna tarea larga que consume CPU, aparecen ruidos molestos que arruinan el sonido. Esto se debe a un fallo llamado "underrun", que aparece cuando el proceso que quiere reproducir sonido no es capaz de suministrar al driver ALSA del sistema los datos (buffers, fragmentos) necesarios a tiempo.

El tamaño de los fragmentos influye en la probabilidad de que aparezcan los ruidos no deseados. Cuanto más pequeño sea el tamaño de los fragmentos, más veces será necesario que el sistema interrumpa otras tareas para escribir datos en el driver de sonido, y esto también incrementa el consumo de batería en un portátil o un terminal móvil. La ventaja de los fragmentos pequeños es que se reduce la latencia de audio, es decir, el tiempo de respuesta entre la producción y la audición del sonido, lo cual puede ser importante para los juegos o los programas musicales que usan MIDI. El tamaño por defecto de los fragmentos en PulseAudio es bastante pequeño, de tan solo 25 milisegundos, y se puede configurar editando /etc/pulse/daemon.conf

Para mantener reducido el tamaño de los fragmentos y asegurar la puntualidad y cadencia de alimentación de datos al driver de audio, una posible solución es incrementar ligeramente la prioridad del proceso que gestiona el sonido, en este caso PulseAudio. Para ello hay dos paquetes de software implicados: RealtimeKit y PolicyKit. El problema es que RealtimeKit en openSUSE está de adorno, se instala por defecto y se activa automáticamente, pero es incapaz de hacer nada por culpa de la configuración de PolicyKit, así que no solo no cumple su función, sino que malgasta recursos. El primer paso será actualizar la versión instalada del paquete "rtkit". En openSUSE 11.3 se proporcionaba la versión 0.5, y en openSUSE 11.4, la versión disponible es la... 0.5 también. Esto es inexplicable, porque la última versión publicada es la 0.9, que incorpora algunas funcionalidades necesarias no solamente para PulseAudio, sino también para otros programas como FluidSynth, Drumstick, KMid, etc...

Hay RPM de rtkit-0.9 en los repositorios de OBS
http://software.opensuse.org/search?q=rtkit&baseproject=openSUSE%3AFactory

Para comprobar que RealtimeKit funciona como debe, es conveniente poder inspeccionar la prioridad de los hilos (threads) individuales de cada programa. Esto se puede hacer con top(1) o con ps(1). Para mayor comodidad, te propongo usar un sencillo script, que puedes guardar en ~/bin/threads:

#!/bin/bash
P=$(pidof $1)
if [ $? == 0 ]; then
    ps -O policy,rtprio,lwp -m -p $P
fi

Usando este script, es posible ver los hilos de cualquier programa en funcionamiento mediante un comando sencillo:

$ threads rtkit-daemon
PID POL RTPRIO   LWP S TTY          TIME COMMAND
4116 -        -     - - ?        00:00:00 /usr/lib/rtkit/rtkit-daemon
    - TS       - 4116 S -        00:00:00 -
    - TS       - 4120 S -        00:00:00 -
    - RR      99 4121 S -        00:00:00 -

La columna POL indica la política de planificación, donde TS es "time sharing", y RR significa "round robin". La columna RTPRIO indica la prioridad del hilo. Como puedes ver, rtkit tiene un hilo con la máxima prioridad. El mismo comando contra PulseAudio cuando está correctamente configurado da el siguiente resultado:

$ threads pulseaudio
PID POL RTPRIO   LWP S TTY          TIME COMMAND
4114 -        -     - - ?        00:04:35 /usr/bin/pulseaudio --start
    - TS       - 4114 S -        00:00:51 -
    - RR       5 4149 S -        00:03:43 -
    - RR       5 4152 S -        00:00:00 -

En la configuración por defecto de openSUSE, los dos últimos hilos usan la política TS por defecto, y no tienen prioridad especial. Para resolverlo, hay que hacer un cambio de política sobre RealtimeKit en la configuración de PolicyKit. Para ello, añade un archivo con el siguiente contenido:

[org.freedesktop.RealtimeKit1]
Identity=unix-user:*
Action=org.freedesktop.RealtimeKit1.*
ResultAny=no
ResultInactive=yes
ResultActive=yes

Sitúa el archivo en la siguiente ubicación y nombre:
/etc/polkit-1/localauthority/50-local.d/org.freedesktop.RealtimeKit1.pkla

Para aplicar los cambios anteriores es necesario reiniciar. Nota: esta receta se aplica tanto a openSUSE 11.4rc1 como a la anterior versión publicada 11.3

FluidSynth and QSynth: Mac OSX status.

2010-11-04T23:25:00.000+01:00

Here is a summary of the latest changes I've made to FluidSynth and QSynth in Mac OSX.

CoreAudio driver now uses the HAL output AudioUnit. A new function has been added to enumerate existing hardware devices, and an option named "audio.coreaudio.device" with the same functionality as other FluidSynth hardware drivers. It can be selected in QSynth using the audio device combo box.

CoreMIDI driver option "midi.coremidi.id" now works much like the equivalent option of the Linux ALSA driver. Added support in QSynth for this setting.

FluidSynth has a new CMake boolean option: "enable-framework", default=true. It creates a Mac OSX style "FluidSynth.framework", a type of bundle that includes headers and libraries together (and other resources if required). This framework bundle is installed by default in "/Library/Frameworks". If this option is disabled at configuration time, the build system creates an Unix style dynamic library as before.

QSynth has also a CMake build system. It can use either the new framework or the old Unix style dylib. After compilation, you can copy/embed all the used frameworks (including FluidSynth) into the app bundle, running the Qt utility "macdeployqt". To build universal binaries, use a CMake argument like CMAKE_OSX_ARCHITECTURES=i386;ppc (all the dependencies need to be compiled using the same set of architectures).

Pending tasks: testing, packaging, documentation. Opinions?

Good times are gone

2010-10-30T23:19:00.000+02:00

My sabbatical is over, and all this month I have been busy on a full time job, programming Windows applications in C# for a living. For this reason I have not had much time to devote to open source projects, and the forecast is that this situation will not change in the near future.

In the past few months I have contributed as much as I could to the free software community with MIDI applications, an area I know well and where there is not too much material, particularly on Linux. Result of this has been KMid2 and Drumstick. I have also maintained my previous projects: KMetronome and KMidimon. About VMPK, this month has beaten their previous records on SourceForge with more than 10000 downloads in October. Thank you very much to all Mac and Windows users who trusted this software, and of course to all participants, contributors and translators who made it possible. In Debian, the situation is also "fantastic", with 137 installations to date, and only one release late. I have no data on other Linux.

Drumstick Time

2010-09-03T20:39:00.002+02:00

A new Drumstick version is in the oven, with two tasty novelties:

1. OVE file parsing support (files created by the Overture program).
The code for this feature has been contributed by Rui Fan. This file format can be played by the included program drumstick-guiplayer, in addition to the SMF and WRK formats. There is also a new drumstick-dumpove CLI utility to inspect the file contents as text.

2. Acquisition of real-time priority, using RealtimeKit.
Until now, the thread processing MIDI input from ALSA tried to acquire real-time priority using the RLIMIT_RTPRIO mechanism, which requires some configuration and administration work. If the above mechanism is not available, now Drumstick tries to acquire real-time priority from RealtimeKit.

This release is scheduled for sometime during next week.

What's next?

At this moment there are two libraries: drumstick-alsa (Linux) and drumstick-file. There will be two more: drumstick-win and drumstick-mac. The foundations of these two new libraries are already implemented in KMid2. On the other hand, it is desirable to unify all the backends for all platforms and file formats under a single set of multiplatform interfaces: drumstick-simple. VMPK will benefit from this, because replacing RtMIDI by Drumstick will be easier to implement features like MIDI file recording, playback and arpeggiator.

Electronic Music Stands

2010-08-06T23:41:00.000+02:00

We are in full swing of electronic books, bubbles that are likely to explode into a new phase of the war between the old and new generations of publishers, content providers and consumers. It is likely that sheet music enter the battlefield.

Before proceeding I must warn you about two facts. One: despite the name of the blog, I am a Trekkie. Two: I am not a neutral observer. In my opinion traditional books publishing, as well as media labels, are going to lose. Those not able to adapt to changing times will disappear. Resistance is futile.

The musicians are going to adopt electronic lecterns sooner rather than later. The electronic scores begin to replace the editions printed on dead trees, as is already happening with books. There are some solutions on the market, more or less primitive.

MusicPad: Hardware and software.
How could you describe it? It is like a Kindle with legs. There is a "Master" version with twice the surface. Among its outstanding features: it is completely silent. Great virtue, because silence is also music.

MusicReader: Software only.
"The facto MusicPad Killer", they say. It is a program with Windows and Mac versions, which has more than just a page viewer. There is an optional accessory for turning pages with the foot. It can display half pages enlarged, as an aid to accessibility. Integrated additional functions: recorder, player, metronome and tuner.

eStand: Software for Windows, page viewer.

All of them are readers for electronic formats such as PDF or similar, showing pages, sections or page sets. MusicReader seems the most complete, yet is far from perfect. Some desirable features for an electronic music stand:

Pages no, thanks. The pages have no musical function and are mainly a nuisance. No pedal to turn pages, automatic scrolling is much better. Bookmarks, notes, sections, movements, labels, individually numbered bars. Everything indexed and easily searchable.

Size does matter. The musicians are supposed to have good ear, but nothing is said about sight. So it is better if the size of the staves is adjustable. For this reason, PDF is not very well suited for scores, it may be necessary to adjust the layout depending on the available area of representation. In this sense, the graphical representation of music resembles the visual organization of the widgets in graphical user interfaces.

Content indexing, cross referenced, not only metadata but all the data resulting from semantic content analysis. For example: tessitura of the parts (lowest and highest notes used by each voice and instrument in the composition) for each one of the scores in the library. With this data available, it may be possible to filter the appropriate pieces for different levels of students. Publishers must provide rich metadata, MIDI sequences, critical texts, and musicological analysis. It is not just changing the medium of the product, it is about changing the cellulose based mindset, into another way of thinking based on information.

Metronome. Probably many of the users will be students, so it will be convenient to have a metronome at the same place, taking the rhythm meter and the speed automatically from the values in the score. And of course, in addition to the optional audio reference, each bar should be highlighted while playing, keeping the current bar clearly visible in a central position, taking control of the scroll function, even when the metronome is quiet. Something like this.

Pitch tuner, recorder, playback. Computer: add an accompaniment. Computer: change the clarinet by an oboe. Computer: pause recording ...

Star Trek TNG 6/8: "A Fistful of Datas"

High connectivity: USB, Bluetooth, WiFi, Ethernet ...

If there is a small or large group of musicians, they may want to synchronize the electronic stands, loading and unloading documents and synchronizing the timers/metronomes.

We already have almost all the required technology as free software:

Music Typesetting, engraving PDF or PostScript scores: Lilypond.
Programs, libraries and components to display these documents: Okular.
Sequencers, synthesizers and MIDI players: KMid.
Free content: www.mutopiaproject.org

We have the opportunity to occupy a niche in musical education, and from here future musicians shall develop the habit of using free software. Perhaps with time, even to generate free content.

Star Trek Voyager 5/22: "Someone to Watch Over Me"

Diagramas de Venn de las tribus de software

2010-08-05T17:25:00.000+02:00

A los calores propios de la temporada estival, se han unido hace poco los de la última polémica sobre los males del tribalismo. El artículo de Aaron se abre con una frase hecha, que significa que cuando tomas una decisión debes aceptar sus consecuencias. Algo así como nuestro "cosechar lo sembrado". Supongo que se refiere a las decisiones de Canonical, y que el chorreo que le está cayendo es su consecuencia. También se puede traducir literalmente al castellano e interpretar como una metáfora: hemos construido nuestras camas, y ahora dormimos en ellas. Eso es lo que hacemos las comunidades humanas. Los del software libre también, pero en general los humanos vivimos en tribus de forma cómoda y natural, y lo difícil es superar ese tribalismo atávico e innato colaborando en metas comunes a toda la especie. Y el problema es que la rivalidad entre tribus degenera a menudo en sectarismo de la peor especie, y de ahí la amarga queja de Shuttleworth.

El hecho de que existan dos tribus independientes, Gnome y KDE, en el mundo de los escritorios de software libre a mi me preocupa bastante poco. Cuando necesito un programa para solucionar un problema concreto suelo buscar entre las ofertas de opendesktop.org, que me presentan distintas opciones a elegir entre kde-aps.org, qt-apps.org, gtk-apps.org y otras. Hago aquí un inciso para decir que no me gusta demasiado esa organización, que sugiere alguna especie de guetos. Al final me quedo con la aplicación que me guste más, y a veces con más de una por el mismo precio, y ya elegiré con el tiempo. Otros no comparten la misma filosofía, y van de compras con el requisito irrenunciable de que los programas han de estar construidos con una única materia prima, pura y sin contaminación. Casi como una cuestión religiosa. Sus juguetes del Sect'R'Us, si se me permite el juego de palabras.

En este diagrama de Venn que es el Mundo, comparto una pequeña intersección con los habitantes de la tribu de los linux-audios. En algunos artículos recientes del planet.linuxaudio.org, alguien buscaba un reemplazo a su querido y viejo editor de textos NEdit de toda la vida para escribir programas. Gedit sería su alternativa natural, salvo que no satisface algunas características:

Buscar y reemplazar usando expresiones regulares o texto simple.
Buscar y reemplazar en todo el documento o solamente el área seleccionada.
Resaltado de parejas de llaves, paréntesis y corchetes.
Modos de edición específicos para lenguajes de programación, con auto-indentación.
Macros, como mecanismo de extensibilidad que permita una mayor personalización.
Atajos de teclado, por ejemplo para saltar a una línea concreta del documento.

Como el blog en cuestión no permite publicar comentarios, me he quedado con las ganas de recomendar Kate, que cumple de sobra con todos los puntos anteriores y no es muy diferente del reemplazo elegido: Geany. De hecho, Kate es el editor preferido por buena parte de mi tribu en esta otra intersección del diagrama de Venn. Pero a pesar de todas sus ventajas, algunos ni se plantearían su uso simplemente porque está basado en KDE.

Future of KMid

2010-07-24T18:46:00.003+02:00

The next version of KMid is in the kitchen right now, and it will have 2.4.0 as identifier. Tentatively, August 15th shall be the release date.

The main novelty is KMidPart, a component that implements the interface KMediaPlayer, playing MIDI/Karaoke files and using KMid infrastructure. The component shares the same configuration as the main program, so by default the MIDI port selection and Soft Synth will be common. This component by default shows only a play/stop combined button and a slider for the time position, in addition to the actions Play/Pause/Stop. Hidden, but selectable by means of KMediaPlayer::View interface methods, there are several additional controls: pause button, volume, pitch transpose and tempo (speed). The following pictures show the minimum and complete user interface respectively.

Once installed the new version of KMid, the KMidPart component is immediately available in Konqueror, either as file manager or web browser, since it is a simple standard KParts/ReadOnlyPart service. Examples of web sites that can benefit from this component can be anaigeon.free.fr and greekmidi.com.

Of course, this component is available to all fellow KDE developers so they can include MIDI playback in other programs. The programming API is quite simple, as demonstrated by two examples in C++ and Python included in the examples directory. If the KMediaPlayer interface is not enough, there are other methods and additional signals available through DBus.

From here, KMid will continue to evolve. I am considering further integration with KDE desktop technologies. For example, standard MIDI files are not scanned or indexed currently, but much of their metadata could be exploited with Nepomuk: tonality, rhythm, tempo, number of tracks and instruments. The settings of the songs, now stored in individual text files, may also be candidates for better integration. Another candidate for integration would be Solid, publishing system MIDI ports. Integrating KMid backends in Phonon can not be addressed at this time, but maybe in the future Phonon may evolve and such integration become possible.

Further integration into KDE unfortunately means less chance for KMid users of using it on Windows and Mac OSX. Although there are ports of KDE for those operating systems, both communities are not very active. I have received a suggestion that KMid does not need to depend on KDE, but only Qt4. This would make possible to distribute KMid binaries for Windows and Mac easily. I'm not especially interested in this option, followed for instance by the Rosegarden project, sacrificing both functionality and community by this way.

MIDI en las Jornadas

2010-07-07T21:12:00.000+02:00

El pasado viernes 2 de Julio se celebraron en Barcelona las IX Jornades de Programari Lliure (jornadas de software libre). Este año han tenido un formato abreviado de un solo día, y han sido organizadas por la Politécnica de Catalunya. Los contenidos han girado en torno a dos ejes: el software libre y los contenidos libres en la educación, y el software libre en la empresa.

Al tratar de educación y software libre, lógicamente se ha hablado del programa eduCAT1x1 de la Generalitat, que básicamente consiste en entregar un ultraportátil a cada alumno e implantar el libro de texto digital; así como de Linkat, la distribución de Linux educativa del Departament d'Educació.

Mi intervención se ha titulado "MIDI en Linux". He presentado brevemente los programas KMid, KMetronome y VMPK con especial énfasis en las posibilidades didácticas en el ámbito musical de dichos programas y de la tecnología MIDI en general. Muchísimas gracias al público asistente y a la organización por su cálida acogida.

A pesar de su brevedad, el contenido de las mesas redondas y las conferencias ha sido interesante, ameno y en algunos casos no exento de polémica, lo cual también es enriquecedor. Espero con impaciencia las jornadas del próximo año.

D-Bus in VMPK

2010-06-19T20:38:00.000+02:00

Yesterday, I've released VMPK 0.3.2 with a new D-Bus interface. VMPK is not a KDE application, it uses only Qt4, so a D-Bus interface wasn't included by default. This VMPK feature is far from perfect, and it will require some improvements in future releases, but it is a start.

What can be done with this D-Bus thing? D-Bus provides a mechanism to send and receive messages between processes (it is an IPC, inter process communication mechanism). This allows one application to be controlled from another one, or from a script. This is the classic application automation scenario solved in Windows with COM/DCOM, and in Mac OSX with AppleScript. D-Bus is a cross platform solution, with nice Qt4 bindings: the module QtDBus. The Qt4 SDK for Windows doesn't include this module, and anyway the bus daemon and libraries are external dependencies. D-Bus has been adopted in Linux by the KDE and Gnome desktops replacing other former solutions.

As use case demos, in a directory named "dbus" of the VMPK source tarball, there are a Bash script playing a simple tune and two Python scripts showing how to remotely execute actions and receive signals from VMPK. Another interesting use case for this D-Bus interface is to define new hot keys to send MIDI messages using VMPK. For instance, let's define Alt+S to send a sustain=100 control change, and Alt+Z to send sustain=0. Here is a brief how-to for the KDE desktop.

To configure hot keys in KDE, run the program "Personal Settings (Configure Desktop)", and select "Computer Administration" -> "Input Actions", or launch the command 'kcmshell4 khotkeys' from a terminal. Here you can create a new group with the name "VMPK", and two new actions (new -> global shortcut -> D-Bus command) for your controllers. Name your actions for instance "sustain on" and "sustain off", and assign the key strokes in the tab labeled "Trigger". In the tab "Action", the data for "sustain on" is:

Remote application: net.sourceforge.vmpk
Remote object: /
Function: controlchange
Arguments: 64 100

The arguments are a pair of numbers, first the controller number, and then the control value. 64 is the sustain (hold1) controller, and 100 is the value. The arguments for the "Sustain Off" are "64 0".

You must activate the check boxes in the actions tree, to enable the new hot keys. You can also set conditions for the group, for instance checking that the window title or class matches the VMPK main window.

Note that you need to compile VMPK with ENABLE_DBUS activated. The binary packages for Windows and Mac OSX don't have this option enabled.

Binary packages? didn't say I something about stopping to ship binary packages of my programs? Yes, there are no binary Linux packages for this VMPK release. The question is that I can provide a single VMPK package for Windows that includes all dependencies (a few Qt4 and runtime DLL), and this package runs in XP, Server, Vista and Windows7. Same for Mac OSX, where I can provide a single application bundle compiled in Universal mode (x86 and PPC) including all the dependencies as well. The burden for Linux is the existence of RPM and DEB package types, with differences among several distros, and several versions, and processors, and whatever. VMPK is provided by several major distros already. If it is not yet included in your distro, open a request in the distribution tracker or compile it from sources.

Weird KMetronome

2010-05-21T11:56:00.001+02:00

After a long development cycle focused almost exclusively on KMid, it's time to give some love to other projects. I've been working lately on KMetronome, a MIDI metronome for KDE and ALSA. It is the oldest project of my little family.

The only issue in KMetronome's tracker is this one year old request, which has suggested the title of this post. The "Weird Metronome" is a popular simple MIDI metronome for Windows, which is the origin of "Open Metronome".

The major advantage of Weird / Open Metronome is the definition of rhythm patterns. The drawback is that it only works on Windows. It may be a good idea to make a Linux port. His definition of patterns is very simple and elegant, if somewhat limited. It has two branches: one producing MIDI events, and the other producing digital audio.

Although I can't rule out porting Open Metronome to Linux in the future, I've decided to implement the pattern editor differently. Here are some screenshots of the upcoming version 0.10 of KMetronome, dressed in Skulpture style.

Settings

Like KMidimon and VMPK, now KMetronome also uses an instrument definition file in .INS format, the same format as Qtractor, TSE3, Cakewalk and Sonar. I've added an "Output instrument" drop-down list in order to choose one among the standard General MIDI, Roland GS and Yamaha XG drum maps. You can add more definitions creating a file named "drums.ins" at $HOME/.kde4/share/apps/kmetronome. The options "Bank", "Program", "Weak note" and "Strong note" are now drop-down lists as well.

Main Window

There is a new button "Patterns" opening the rhythm pattern editor, and a drop-down list to choose the pattern. The default "Automatic" value means that the program generates the pattern as usual, using the notes set in the configuration dialog (Strong / Weak) and the rhythm definition provided by "Beats / Bar" and "Rhythm figure". The same drop-down list contains also the names of the user-defined patterns.

Pattern Editor

This is a new dialog box. You may edit, test and select patterns with it. To create new patterns, you simply save the current definition under a new name. Patterns are represented by a table. The rows in the table correspond to the percussion sounds. You can remove and add rows from a list of sounds defined by the instrument settings in the configuration dialog. The number of columns in the table determine the length of the pattern, between 1 and 99 elements of any rhythm figure.

Each table cell accepts values between N = 1 and 9, corresponding to the MIDI velocity (N * 127 / 9) of the notes, or 0 to cancel the sound. Valid values are also "f" (forte) and "p" (piano) corresponding to variable velocities defined by the rotary knobs (Strong / Weak) in the main window. The cell values can be selected and modified using either the keyboard or the mouse. No need to press the stop button before modifying the cells ...

This version is not ready yet, but will be probably published by the middle of next month. For now, you can get the sources from the Subversion repository. You need to install Drumstick before compiling it.

By the way, I'm planning to stop distributing binaries for all programs. If you like them and your favorite Linux distribution does not include Drumstick or KMetronome, it would be a good idea to start opening bug reports in its enhancement request system, or sending messages to the distribution's mailing lists or web forum.

Apples and Oranges

2010-04-26T12:33:00.000+02:00

KMid is now a multi-platform application for Linux, Windows, and Mac OSX. It may be the right time to make a comparison between the different operating systems with regarding to the development of KMid backends.

First, the functional components needed by a KMid backend

Read and parsing of SMF: MIDI and Karaoke files. This mechanism must offer not only timestamped MIDI events, but also the metadata (for instance, song lyrics) embedded into the SMF data.

Facility for sequencing MIDI events. Events read from a file are labeled with timestamps, to be delivered to MIDI synthesizers at the right times, handling also common player actions like play, pause and stop.

Internal and external, hardware and software MIDI synthesizers. The main goal of KMid is to support external musical instruments, but as many potential users do not have one, it is interesting to be able to use software synthesizers, in a transparent way without complicating the program design.

Organization of the backends

KMid::Backend is modelled after several Phonon interfaces. There is not a dependency on Phonon, only inspiration and Copy&Paste. There is a KMid::MIDIObject abstract class that resembles more or less a Phonon::MediaObject, encapsulating the sequencer functionality, and a KMid::MIDIOutput class representing a MIDI output port, similar to Phonon::AudioOutput. Main differences are:

KMid::MIDIObject time is measured in musical time (ticks) instead of milliseconds. Some additional properties: timeSkew, textEncoding, lyrics. Several signals, one for Each MIDI event type and metadata, feeding the program's graphic interface animations: lyrics highlighting, visual metronome, channel meters, and piano player keys.

KMid::MIDIOutput has pitchShift and midiMap properties. Volume and mute properties take a MIDI channel argument. There are also some real-time MIDI event slots, one for each MIDI event type, used by widgets like the channel instrument selectors or the piano player keys when they are triggered by the mouse or the computer keyboard.

Linux

ALSA provides a very advanced MIDI sequencer. The API is big and cumbersome to use in a KDE program, so I've developed a library layer, named "drumstick-alsa", providing a C++/Qt4 wrapper around it. ALSA does not provide a mechanism for reading and parsing MIDI files, so there is another library named "drumstick-file" taking care of it. In ALSA, MIDI hardware and software clients are all equivalent and there is a routing mechanism between MIDI OUT and MIDI IN ports, fully transparent and very powerful, where either of the two ends of the connection can be a program or a MIDI device driver. Of course, the data transmitted by the communications connection uses the standard MIDI protocol. KMid creates a MIDI output port externally visible, to be connected to a MIDI synthesizer, and also private loopback ports (input and output) which are used for visual feedback in the graphic interface (highlighted lyrics, metronome, piano keys...) The MidiPlayer class schedules MIDI events to a loopback port, which are received in time and propagated activating the signals emitted by the MIDIObject class and at the same time are sent through the MIDIOutput class to their final destination, which is any kind of MIDI synthesizer.

Any software synthesizer that is also an ALSA sequencer client can be connected to the MIDI output of KMid. Although not strictly necessary, for the users convenience it is possible to enable and launch FluidSynth or TiMidity++ on KMid's initialization. Unfortunately, a software synthesizer receives MIDI events and converts them into digital audio, and here is where some users may find problems that go beyond the scope of KMid (ALSA/Jack/OSS/PulseAudio..., you know). In addition, a software synthesizer may require soundfonts. Enough to say that Linux distributions should package and integrate all the components that the users may need. The owners of external MIDI musical instruments are lucky, because they are quite well supported by ALSA.

Windows

The operating system provides the same MIDI support since the days of Windows 3.1 practically without any changes. Windows cannot connect the MIDI output from a program to the input of another program, only to a system driver, so the synthesizer must be external, or having a device driver, or using MidiYoke or any other similar solution. However, Windows provides a software synthesizer unconditionally installed and activated, so the Windows backend does not have or need a soft-synth configuration page.

The MMSystem API does not provide virtual MIDI ports, nor transparent MIDI routing, so the strategy of playing to a loopback port as in Linux is simply not possible. What is possible is to use "custom" midi events and define a callback function that is invoked whenever the time arrives to play a scheduled MIDI event. This is how all the visual feedback is implemented. The system libraries do not provide reading and parsing of MIDI files, so drumstick-file is used.

Mac OSX

The operating system provides all kind of MIDI support services, starting with reading and processing of MIDI files. If we compare the AudioToolkit SMF processing API with drumstick-file, the latter provides a model similar to XML-SAX, while AudioToolkit would be similar to XML-DOM. After reading the MIDI files, the results are explored to extract the required metadata, and an additional track is added containing the "custom" MIDI events for visual feedback.

Virtual MIDI ports are fully supported, with routing and full transparency between applications and devices, so the loopback port strategy works exactly the same as in Linux/ALSA. The Apple software synthesizer is an object of type "AudioUnit", and requires activation before a program can use it, so the Mac OSX backend has a simple soft-synth configuration page. The DLS soundfont provided by Apple is made by Roland, like the Windows one.

Conclusion

The LOC numbers needed to implement each backend are very similar: mac = 2358, windows = 2435, alsa = 2715 (not counting drumstick), and its complexity is also similar. Despite the different features offered by each platform, it is possible to implement KMid backends presenting the same programming interface to the application. We can create new backends in the future using the existing ones as models, or duplicating the "dummy" template that exists in a directory with the same name in the source repository.

KMid in Windows

2010-04-17T00:46:00.000+02:00

KMid is aimed at a wider audience in this next version. This song was generated with MMA (Musical MIDI Accompaniment), and it is included as a new example.

History of KMid

2010-04-15T14:18:00.000+02:00

KMid was originally created by Antonio Larrosa. According to his website, the exact birth date was the 11th of September of 1997.

KMid initially ran in Linux and FreeBSD using the OSSv3 MIDI sequencer. The release 1.0 coincided with the KDE1 publication in 1998, and version 2.0 with KDE2 in 2000. It was ported later to KDE3. Version 2.0 and was functionally very similar to KMid2, having already ALSA support.

I've started using KDE by 2002, along with the initial KDE3 releases. At that time I also became involved in the development of Rosegarden, a KDE program for music edition using MIDI technology, and used KMid frequently. The main drawback was the inability of teaming KMid with other ALSA sequencer based programs, because the play/pause/stop actions created and destroyed KMid's MIDI ports, preventing reliable external connections made by other programs like aconnect or qjackctl.

In Akademy-es 2006, that was held in Barcelona, I met Antonio and explained him my concerns about KMid, and subsequently exchanged some patches by email trying to solve the implementation of libkmid regarding the ALSA sequencer. However, any solution explored involved binary incompatibilities, so the issue was out of the question within the KDE 3.5 development cycle.

On the other hand, KDE4 development started and OSSv4 was published, so there was no need to support OSSv3 anymore. The transfer of KMid in the SVN repository from kde-multimedia to extragear provided more time and freedom to rethink the software architecture and develop a new implementation.

KMid2 was developed as a revamping of the original KMid, with the following main features and goals:

Playback to external MIDI devices.
Allow to use software synths as well, like TiMidity++ and FluidSynth.
Change tempo and volume controls. Add a key (transpose) control.
Independent MIDI channel mute control.
Support for many character encodings for lyrics, and configurable fonts.

Additional short-term goals:

Remove the deprecated OSSv3 /dev/sequencer interface support, dropped from OSSv4 anyway.
A fair ALSA sequencer backend implementation: do not create/destroy the client and port instances on each play/pause/stop action. This enables the usage of a MIDI Patch Bay application like aconnect or qjackctl.
More native backends for other platforms.

Version 2.2.2 was released on March 15th, 2010, closing the initial implementation cycle. The next release 2.3.0 will be published within a few weeks, including new native backends for Windows and Mac OSX.

KMid in Mac OSX

2010-04-01T16:37:00.000+02:00

This is not an April Fool's joke. KMid is running in Mac OSX, native:

I've developed KMid's Mac native support using Fink, that includes KDE SC 4.4.1

By the way, there are fresh news at Fink web site: "Effective 1 May 2010, we are going to cease work on our Mac efforts, and switch to development for the iPad, iPhone...". Oh, wait! today is April 1st...

Now seriously. You need: Xcode 3.1.4, that provides GCC 4.2
And some dependencies from Fink: kdelibs4-mac-dev, kde4-buildenv, cmake. These packages require some other dependencies.
Recommended: bundle-kde4-mac. This set is very large, and includes everything for a complete KDE environment.

KMid doesn't use Phonon. The native Mac OSX backend uses CoreMIDI and CoreAudio, and the Apple DLS soft synth enabled by default. Everything is included in OSX. Receipt:

1. Get the sources from the repository:
$ svn co svn://anonsvn.kde.org/home/kde/trunk/extragear/multimedia/kmid/

2. Prepare the build environment:
$ export KDE4_TYPE=mac
$ source /sw/sbin/kde4-buildenv.sh

3. Configure, compile, install:
$ cd kmid
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=/sw/opt/kde4/mac -DBUNDLE_INSTALL_DIR=/sw/opt/kde4/mac/bin
$ make
$ sudo make install

Enjoy!

Loving Music

2010-01-28T23:08:00.000+01:00

I love music even more than I love computers. And here is a new KMid2 release, for all the music and KDE lovers. It has several new features, but that is something that you can read in the announcements or the ChangeLog. There are also four new songs replacing the old examples coming from the original KMid. It was not clear the license of those files, so I've created new ones. Like the two first examples (Ay Linda Amiga and Mozart's Ave Verum) the songs were edited in Rosegarden, more or less tweaked, and finally exported in SMF format. The sources of all the songs are the scores published by the Mutopia project. The LilyPond process that creates the beautiful PDF scores also produces MIDI files that have been imported into Rosegarden as the foundation for the final products. Here are the addresses and references, and one rendering as audio.

Schubert (1797-1828): Ave Maria (Ellens dritter Gesang)
Mutopia score
Wikipedia article

The following rendering has been produced in FluidSynth using the General User soundfont.

Mozart (1756-1791):
Der Hölle Rache kocht in meinem Herzen (Die Zauberflöte / The Magic Flute - No. 14 Arie)
Mutopia score
Wikipedia article

Dies Irae (Requiem)
Mutopia score

Handel (1685-1759): Hallelujah (Messiah)
Mutopia score

MP3 killed MIDI?

2010-01-21T18:34:00.000+01:00

Of course not. At the turn of the 20th century everybody was saying that photography killed painting. And later cinema killed theatre, and television killed cinema. The last technological revolution, the digital one, killed the business of the record companies. And this time, it may be finally true...

MIDI is about making music, and making music is about musical instruments. After all, a business that remains still alive is the musical instruments industry. Not all the musical instruments sold around the world are bought by professional musicians. Most of the market are aficionados and hobbyists. Everybody would like, in one or another point of the life, to become a musician, or learn to play a musical instrument. Well, this is the place of the MIDI software in the desktop ecosystem. It is not about passively listening music, but using tools to inspect, interact, learn, modify, understand music. That may explain how VMPK had more than 5000 downloads from SourceForge last month. Even when it is not able to play songs by itself!

There was once a day something called "Pianola", a piano player product manufactured by the Aeolian Company. It was so popular that the trade mark has been adopted in Spanish (and in other languages as well) as a generic name, and entered the official dictionary. The company ended in bankruptcy about only 25 years ago, but the idea behind it still exists. The Disklavier models by Yamaha are offering today a similar functionality, but using MIDI files instead of punched paper rolls.

Well. All the above small talk was, of course, an excuse and introduction for another KMid post. The next week there will be a new release (0.2) of the program, after some changes triggered by an usability review. Big kudos to the KDE usability team and Diego Moya for his report! One of the new included features is the ability to launch soft-synths (TiMidity++ or FluidSynth) from inside KMid2, as child processes. The communication between KMid and the soft synths is handled by the ALSA sequencer, using the standard MIDI protocol. This has two consequences: the soft synths are still loosely tied optional dependencies, and the program is still able to communicate with external musical instruments, hardware synths, or many other soft synths, which is a very important feature for me. This also means that KMid is still for Linux only, because ALSA is only available for Linux (the Salsa library has only very basic ALSA features, and it is not enough for KMid).

There will be in the future other backends for KMid. I'm planning native backends for Mac OSX and Windows, using the native MIDI subsystems of each operating system, and also a backend based on libfluidsynth, which may be very comfortable for casual users, and available for more platforms.

MIDI on Linux is not easy

2009-12-22T21:14:00.002+01:00

I've put a little tutorial in the KDE userbase wiki about MIDI on Linux because Linux users have very little help setting up their systems for MIDI usage.

First, I would like to apologize for a joke, that was not my original idea. I've found it at the renaissance music web site of Alain Naigeon. He says that MP3 is like fast food, WAV is tasting the meal, and MIDI is cooking the recipe. He also says that playing a MIDI file is not reproducing a performance, it's a new performance. I agree, but on the other hand each MIDI file is a recording of a musical performance. Even more than a MP3 or a CD, because it contains no sound, only the musician actions. For instance, the folks at the Minnesota International Piano e-Competition distribute MIDI files of the competitors' performances, recorded in Yamaha Disklavier concert grand pianos, as standard MIDI files. And you can play these files in KMid2. Try this one: Igor Stravinsky's, Petrouchka played by Alessandro Taverna, first prize winner of this year's contest.

Returning to the subject. I would like to show you a screenshot of the about dialog from the software synth included (and ready to be used out of the box) in all MS Windows operating systems:

The Roland Sound Canvas samples are also included in Mac OSX. In both cases it is a small sound font file in DLS format. It is not the full and great Roland Virtual Sound Canvas, but a light version. Anyway, the point is that both software manufacturers provide resources to the users wanting MIDI support out of the box. If you download and install VMPK in any Windows version, you can start playing and getting sound at once without having to configure anything at all, and without reading a single word of system documentation.

What is the situation on Linux? There are MIDI applications. There are many operating system MIDI drivers (ALSA). Some software synths and free SoundFonts are also available. What is missing? Distributions gathering all the required pieces, putting them together and easing the task of setting up Linux for MIDI usage out of the box. It is their job, after all.

Precompiled Headers

2009-12-07T16:35:00.001+01:00

Hello, KDE Planet!

Let me insist with another post about time. But in this case about the time spent compiling programs; always too much, because vita brevis est.

I've added to KMid2's build system a CMake script enabling precompiled headers (PCH) for the GCC compiler. It is optionally enabled using a CMake option "-DWANT_PCH=yes". I've written this script initially for Rosegarden, and later it has been refined and included in several other projects with the goal of reducing the build time.

Enabling PCH for a project is not automatic. You need to write a header file including enough library headers, and give this big header to the ADD_PRECOMPILED_HEADER macro. It is also a good idea to use ADD_DEPENDENCIES in the targets, to ensure that a PCH file is generated before the target components. The macro automatically adds an "-include" argument to each compilation step in the directory, so PCH can be enabled or disabled for a project without needing to modify the sources.

The trick is creating the big header file with enough #includes to be worth, but not too much. Compiling the PCH file takes not only time, but also many megabytes of disk space. For KMid2 this header is named "qt_kde.h" and includes mostly Qt and KDE headers, and also several Standard Library and ALSA includes.

From my development machine, here are some measures comparing compile times, with and without PCH.

Without PCH
real    1m44.137s (104137 ms)
user    1m25.417s
sys     0m8.149s

With PCH
real    1m21.081s (81081 ms)
user    1m10.732s
sys     0m8.641s

KMid2 is a very small application. For comparison, Rosegarden 1.7 needs between 11 and 15 minutes to build in the same machine. The savings in KMid2 using PCH are 23 seconds, 22.14%. Not too bad! But KDE's build system has another way to save compiling time, and it is ready out of the box for any KDE project using the CMake option   "-DKDE4_ENABLE_FINAL=yes". This setting creates a Single Compilation Unit for each target. Times measured for KMid2:

With ENABLE_FINAL, and without PCH:
real    1m19.808s (79808 ms)
user    1m14.025s
sys     0m4.932s

Savings: 24.329 seconds, 23.36%

Conclusion: ENABLE_FINAL is slightly better than PCH, and it can be automatically used by all KDE4-based applications. So why bother with PCH? I think that PCH may win if you can use the same PCH file when compiling a whole set of KDE programs at once, but I've not tested this hypothesis yet...

Tempus Fugit

2009-12-04T18:18:00.001+01:00

Time is the dimension of music, likewise width and height are the dimensions of pictures. When transmitting MIDI data on a wire, events don't need to be marked with time labels, because the time simply flows. MIDI is a real-time protocol. When the events are stored on data structures, like SMF (MIDI files), the events must be timestamped. In SMF, each event has a delta-time, the elapsed time since the previous event, measured in ticks. The time formats on SMF closely resembles the conventions used by the written music tradition, the musical score.

Let's start with the metronome. A metronome is a device that signals musical tempo. It is used by musicians to keep the rhythm and speed while practicing. Metronome units are beats per time unit, for instance crotchets per minute. MM=60 means 60 crotchets in one minute, or one crotchet equal to one second. The crotchet, also known as quarter note, is the double in length of a half, and the half is the double of a whole. Conversely, the half of a crotchet is an eighth, and this is the double of a sixteenth. These figures are the time lengths of the musical notes, and they are relative measures. Music is written in terms of relative times.

In SMF, the time is measured in ticks. The division value, which is declared into the file header, is the number of ticks in a quarter note. Common division values are 96, 120, 240, 384, 960. The numbers are usually divisible by 3 and 4, and big enough to avoid decimal numbers when measuring the length of very short notes.

Tempo changes are SMF meta-events. They may happen at any point in the time line and are represented in microseconds per quarter note. This magnitude is inverted with regarding to the usual metronome units (quarters per minute), because it can be represented by integer instead of decimal numbers. for instance, mm=60 would be represented as SMF tempo=10000000, and mm=120 as tempo=500000.

As note start and length properties are represented by relative magnitudes instead of absolute times, and tempo changes are flexible enough to be placed at arbitrary points, it is easy to modify them without collateral effects. If you insert a tempo change in a SMF using a MIDI editor software, you don't need to change the length or the starting time of the following or previous notes. Of course, when the SMF is finally rendered to be listened, the sequencer engine needs to do all the time calculations. But not so fast, wait a minute!

When you play MIDI files, it is a common requirement to perform slower or faster than the nominal tempo changes encoded into the file. It is like zooming in picture viewers. For instance, a student may need to render a piece half or slightly slower than it normally plays, or a dancer may need it a bit faster. The ALSA sequencer engine provides a very handy mechanism for programs to do this: the time skew property. You can enjoy this vary-speed functionality in KMid2 and KMidimon, when it is used to play MIDI files.

Parallel Lives

2009-12-03T14:07:00.000+01:00

Plutarch was a Roman-Greek historian and biographer, well known because of his work "Parallel Lives", a series of biographies of famous Greeks and Romans, arranged in pairs to illuminate their virtues and vices. I wanted to try a similar approach pairing some digital audio and digital image technology citizens. But the vices and virtues only apply to the computer users and developers, because the technology, like the science, is neutral.

Let me start before the computers epoch. At some point in the past, painting was the only way to fix an image in a 2D surface. The sound, on the other hand, was impossible to fix in a literal way. Only using a symbolic music language, the same used today (musical scores). In the 19^th century appeared the photography and the phonograph. At this point there were two artistic approaches for taking images: painting and photography. Music has a similar duality: it is still possible to “draw” scores, and also to take snapshots of musical performances to be preserved and later reproduced literally. Of course there are non-artistic usages of photographic and phonographic recordings, as well.

Let's stop here a moment. After the photography birth, nobody really has any doubt about the future of the plastic arts. Today, we all enjoy with paintings, children learn to draw and paint, and architects and engineers use technical drawings in their daily jobs. And there is also photography, with artistic and non-artistic branches. We have specific tools in our computers to fulfill all the tasks related to the images. There is Krita and Karbon14, GIMP and Inkscape, Photoshop and Illustrator. For each digital image program there is a vectorial drawing one. There is also CAD software, of course, and ray tracing. More about this later.

Bitmap (raster) graphics are digital representations of 2D images. A digital bitmap is a matrix of pixels (dots), each pixel representing the color (and sometimes transparency) for a point. Different resolutions, or pixel density, are measured in pixels by length unit. Common resolutions are 100 dots per inch, for display devices, and 600 dpi for printers. We can calculate the size in bits of a picture in bitmap format if we know the pixel size and the with and length in pixels of the picture. For instance, an image of 50x20 pixels in true color, 24 bits per pixel, weights exactly 50*20*24 = 24000 bits = 3000 bytes. It is not possible to make a similar calculation for a vectorial image, as the size depends on the complexity and details of the drawing, and not the dimensions.

Digital audio is represented in PCM (pulse code modulation) format as a stream of samples. Each sample is a measure taken by a microphone in equal instants of time. Resolution, or sampling frequency, is measured in samples per time unit. For instance, in Hz (samples per second). Common resolutions are 44100 Hz for CD quality recordings, or 96000 for higher quality. We can calculate the size of an audio recording knowing the number of channels, the sample size, sampling frequency and time length. For instance, a recording of 1 second of monaural (1 channel) sound, using 24 bit samples at 44100 Hz takes 24*44100*1*1= 1058400 bits = 132300 bytes.

Both bitmap graphics and PCM digital audio recordings are usually compressed to reduce it's weight. Some of the compression methods also discard the less relevant information. The JPEG and the MP3 formats are examples of lossy compression. Uncompressed formats are for instance BMP and WAV. Lossless compression examples are PNG and FLAC.

Vector graphics are stored in several file formats. A modern standard is SVG, based on XML. There is also a XML-based music representation, called MusicXML. A very common music format, part of the MIDI standard is SMF (Standard MIDI File) using the .MID filename extension. Vector graphics need to be rendered into bitmap graphics before being displayed, using a rendering graphics library as Cairo. Same happens with the MIDI music, rendered into audio with a synthesizer.

Vector graphics are usually schematic, not very photo realistic. But there are some ray tracing programs allowing photo realistic rendering of a symbolic textual vector graphics source. Something similar happens with MIDI rendering using samplers, producing orchestral music with high level of realism. But many MIDI synthesizers usually leave a very characteristic electronic timbre.

When you render an empty vector graphic (say, a simple white surface) into a bitmap, you can realize that the size of the resulting rendering doesn't depend on the image contents, while the size of the original vector graphics image does. Same with MIDI. If you create a MIDI file of John Cage's composition 4'33'' (which is absolute silence), it will take only a few bytes. The rendering into digital audio weights several megabytes uncompressed. The same size taken by 4 minutes and 33 seconds of a jazz song.

About the transformations. The name Scalable Vector graphics already hints one of its strengths: scalability. You can resize a vector graphic without losing quality, in contrast to a bitmap graphic resize operation. In digital audio, the only dimension is time. You can change the speed, the tempo of a musical composition in MIDI format without losing quality at all. Or you can transpose a song, lowering or rising the notes' pitches. In digital audio you can do those transformations (time stretch and pitch shift) using a FFT (fast Fourier transform) algorithm, but it usually creates artifacts. You can do more transformations in MIDI: mute one instrument, or change it for another one. Adjust the volume of an instrument, or even individual notes.

There are also deep differences in computer support for these technologies. Digital audio is very well supported, like bitmap graphics. Vector graphics are also very well supported, and since the adoption of the SVG standard, it has been very beneficial for Graphic User Interfaces. MIDI is, by contrast, really bad supported, both for hardware and software. Modern graphics cards include 2D and 3D acceleration, and graphics libraries like Cairo and Qt4 obtain a real benefit of the hardware enhancements, and the graphics acceleration is being increasingly adopted by most computer manufacturers and software vendors. On the other hand, audio hardware interfaces have been dropping MIDI support, which was never very good (remember the old cheesy sound of MIDI files?) And Linux distros don't include an easy to use software synthesizer. Both Windows and Mac OSX include a software synthesizer ready to be used out of the box, including the Roland SoundCanvas (lite) SoundFont, which is not a beast but has reasonable quality. Meanwhile, the Linux distros don't care about that. And what about MIDI software? OSSv4 dropped MIDI support. The ALSA sequencer is very good, but needs more good applications.

VMPK supported platforms

2009-12-02T15:19:00.007+01:00

I've read a debian bug complaining because VMPK fails to compile in FreeBSD (by the way, it will fail also in other Unix flavors, except maybe SGI Irix).

When I was planning VMPK, it was going to be a Linux only program, ALSA sequencer based. Indeed, there is still a test program in my aseqmm library using the same piano widget. Because I only needed limited functionality, I decided to try RtMIDI instead, which offers Windows and Mac OSX support. As Qt4 is also supported in these operating systems, the program was born multi-platform.

Maybe the number of supported platforms is going to grow in the future. That depends on RtMIDI. Any volunteer out there? The developer of RtMIDI, Gary P. Scavone, is very collaborative and kindly accepted the patches I've sent him in the past. About OSS, there is a note in the RtMIDI documentation saying that a decision was made to not include support for the OSS API. And there is no MIDI support in OSSv4. But there are more MIDI APIs, like Jack MIDI...

Inauguration

2009-12-01T21:55:00.000+01:00

So, Why I am writting this blog?

I am a software developer to make a living and also as a passion, vocational activity, developing free software pro bono. Almost always for Linux, using Qt and KDE platforms, and MIDI technology. That is: music software. Some of my programs: KMid2, a MIDI/karaoke player, VMPK: a virtual piano, KMetronome: a MIDI metronome, KMidimon: a MIDI monitor. I will talk here about these, and other computer programs.

Sometimes people is surprised by my interest on MIDI. Isn't it a dead, ancient technology? Why should anybody be interested on MIDI, having better alternatives like MP3? What is really MIDI about? I will try to answer these questions in future posts.

Meanwhile, I've released a few days ago a preview of KMid2, a new incarnation of the classic KMid software. Here is a little demo. "Ay, linda amiga" is a renaissance anonymous Spanish song from the "Cancionero de Palacio".