End of 2020

This website is written in English. Select your prefered language (right or below).

Covid-19

This year has been rather strange, not only for you as readers of this blog, but also for us here on El Hierro! In February we made a trip to La Palma, our neighbour island at a visual distance of 80km. But returning home, we entered the Spanish State of Alarm because of Covid-19. This continued until July, and for me going outside making new recordings became impossible. In July the State of Alarm was over and we could go outside again. But then the island was overwhelmed by tourists, more than ever, only from the other Canarian Islands to enjoy their new freedom and security (we still have only few cases here) to see friends and relatives. So what about “El Hierro, tranquilidad…” as many visitors say here, even without knowing the island…

El Hierro, tranquilidad…

It is clear that Covid even has its effect on a more or less stand-alone project like this. I hoped at the beginning of 2020 to make a lot of new recordings and movies, but things went differently. Read on …

Musical findings

First some musical aspects before describing more technical details. During the time in confinement, I listened and experimented with the tools that I have now. As I wrote earlier, I was impressed by the orchestral works of Ligeti and that inspired me to use natural sounds of my recordings combined with microtonal effects. The (Ableton) samplers that I use have options for very small tonal changes and combined with multiple voicings, “portamento”1) or “glide”2) effects give very interesting results. This is especially true for long sounds with a duration of several seconds. I hope to add some examples here in the near future. At this moment I am still in the experimental phase.

A new Mac

In the meantime I was worrying about what would happen in the future if my old iMac (from 2007) became rather outdated or even worse. Just after I did some investigation on my possibilities, the bad things happened: first the hard disk nearly stopped working. After a replacement also the graphics processor stopped working and now the old iMac is blind and only usable as backup disk. Now I have a MacBook Pro with MacOS 10.14. (Mojave) installed. For development this meant installing and configuring new versions of Xcode, Qt, etc., hoping that my sources would compile on this machine in this new ‘environment’. Fortunately, with some small adjustments this was ready in about a week. It could have been worse…

Because of this change I went on streamlining and refactoring my code and of course I repaired some bugs and irregularities. I also spend a lot of time on small refinements to cope with a more natural and intuitive object detection. Due to Covid I had plenty of time.

Because I use Qt for development, it cannot be very difficult to build a Windows version too, but I better leave this to someone with more Windows expertise. If you think to be the right person for this, you can contact me.

New detectors, based on Artificial Intelligence

Reading some posts on the blog of LearnOpenCV.com inspired me to dug further into artificial intelligent (AI) solutions to achieve better and more stable detection methods. Using AI, in whatever field or scope, always involves a training phase for software to generate a large dataset (think of ‘Big Data’) to be used for discrimination between positive and negative (detection) results. With these datasets software programs can answer questions like “Is this a cat?”, or “Is this a US president?”, or even “Is this a criminal?” (be aware that these datasets might have some bias…). A training phase is always necessary to answer questions with a high degree of probability. With this in mind, I built a separate program to build and train a new object detector (a so called HOG detector, from Histogram of Oriented Gradients) with a large set of pictures of my webcam and use that detector in my original software. I can train the detector on any object that I have at hand.

The results are good albeit that this detector is rather vulnerable for small deviations from the learned dataset: small rotations of the object can lead to false negative or positive detections. My older detectors (trackers) work a bit “smoother” but their detection view always grows after some time or moves away from the original object. That is not the case with this new detector. Even losing the object out of sight is not a problem, it catches the object once again in view. Multiple similar objects are also detected. This type of detector is very reliable and stable over time. Achieving a frame rate of 15 frames per second is not bad for this application.

These days a lot of people are using Zoom and Skype to communicate. Both applications have a feature for replacing the background from your webcam with a picture, to avoid other people to have a close look in your home or office. This feature inspired me to explore more of the functionality that OpenCV has to offer and I think there are usable effects to try. That’s for the next blog post.

More outside interest

During the year I met some people who might be interested in an adapted form of my software. It can be used in schools or individually by children or young people who need other forms of nonverbal expression as an addition to ordinary communication. It also can serve people with a motor disability.

I hope to meet more people because I’m curious to see my program being used in all kind of circumstances. In mutual collaboration we must find a way to make this work. If you are interested, please feel free to contact me via the contact page.

Latest results

After all this hustle and bustle with Covid, facemasks and computers I am now again at the point to combine new sounds.

Here is my latest result with the new HOG detector. I used four different sounds to create a soundscape for the pastors (herds) of this island. There still are a few of them here… This video also demonstrates that the detector can detect multiple similar objects and generate more sounds simultaneously.

Sonidos de pastores, used with coasters of the Spanish “Paradores”

So now I keep my arms crossed for next year 2021. Again I hope to post some additional results. I wish you al the best, and as they say here: ¡Cuidate! (Take care!)

Now I can go on, I have to go on …


1) see portamento

2)see glissando 

Start of 2020

In this post you can read about the progress of my project in 2019, after april. You also can read my previous posts because now and then I added some new examples…
I hope to post more this year.

Musical findings

In 2019 first I experimented a lot with my recorded sounds in Ableton, and later with some sounds from Spitfire Audio plugins. The sounds of these plugins are based on recordings from real instruments, orchestras, the London metro, etc. They sometimes have funny and unpredictable effects, especially when using the ‘Dynamics’ effect in combination with long sounds. I was experimenting a lot with these sounds in my program, and I wanted to find out if they would combine very well with e.g. my wind and sea sounds.
Independent of this, but at the same time, I discovered the works of the composer György Sándor Ligeti. I saw the very impressive orchestral performance of the piece Atmosphères by the Berliner Philharmoniker, conducted by Sir Simon Rattle. By using a new technique called micropolyphony, Ligeti in fact created very dense ‘soundscapes’ that are built of sounds that move at different speeds and have different tonal spacings. Here I quote how Ligeti described it himself:

Technically speaking I have always approached musical texture through part-writing. Both Atmosphères and Lontano have a dense canonic structure. But you cannot actually hear the polyphony, the canon. You hear a kind of impenetrable texture, something like a very densely woven cobweb. I have retained melodic lines in the process of composition […]. The polyphonic structure does not come through, you cannot hear it; it remains hidden in a microscopic, underwater world, to us inaudible. I call it micropolyphony (such a beautiful word!)

I was struck by the similarity between the sound effects of the Spitfire plugins in my project and the orchestral sounds of Ligeti. This discovery widened my view on a new musical area. As for me, Ligeti bridged the gap between traditional orchestral compositions and soundscapes.
Although I prefer to use my own recordings, using the plugin sounds and listening to Ligeti give me a better understanding of when sounds are musical or produce music in my opinion. Based on these findings, I begin to see great opportunities to create new music with my recordings but still more research is needed to grab the ideal sounds in nature that can be useful.
I also will concentrate on microtonal effects, by not using the discrete whole steps as in western chromatic scales, but using more subtle pitch changes.

Audio recordings

In summer, apart from the motorbikes and boom cars of a few tourists, El Hierro is very quiet: birds are silent, the sea is calm, everything is at rest. Often you only hear your own heart beat. Now, at the turn of the year the singing of the birds starts again, you hear the bees and the sea and the wind gets rougher.
I already made a number of new recordings based on my new experiences and this expedition continues in this year.

piedra_el_rey
at Piedra el Rey, on the footpath to the Fuente de Mencáfete

Programming

While experimenting with my sounds, making movies, using my notes, searching through folders on my computer I was getting tired of the way my process was hindered by the implied structure of my computer. So I started searching for some tools to make life easier. The result is that I created a rather complete set of software tools that I use simultaneously to create new instruments with my recordings, make notes, describe findings etc. Here an overview.

For maintenance of al my recordings, pictures etc., I used a journal, a word-document with all data and pictures. But for experimenting with the sounds this was not very useful, it required a lot of searching, switching applications and so on. So I started to look for a better tool and I came upon Swinsian, comparable with iTunes. This software is ok, but I made a new user interface for faster and more adequate searching, playing, copying and pasting, and annotating, based on the Swinsian database. It now works very smooth and I use it side by side with Ableton to build new instruments.

maintainance_app
screenshot of the maintenance application

For synchronising my recorded videos from my detection application and audio from Ableton, I used some basic ffmpeg commands, build in an Applescript. This was not as flexible as I needed, especially when some shifting of the audio was necessary to adjust audio and video to fractions of a second. Synchronising audio and video is a time consuming process, so a good and responsive tool is not a luxury. Standard commercial applications are very clumsy to perform these tasks or are very expensive. My detection application generates videos with a frame rate nearly twice as much as necessary for the final result. I did this because the detection mechanism can cause some timing irregularities in the video recording, due to the limited processor capacity. By scaling back the video frame rate afterwards by adjusting it to the audio, these irregular effects smooth out and disappear. So I ‘wrapped’ the ffmpeg commands in a C++ program with functionality to adjust audio and video with a precision of 0.01 sec, and with a simple efficient user interface.

ffmpeg_app
screenshot of the synchronize application

The third programming project was the complete rebuild of the user interface of my original detection program. During my work on the previous programs, I learned a lot more about the Qt/C++ programming environment. This rebuild made the program structurally less complicated and easier to maintain. My previous remarks, that it was finished were a bit premature, but now I think there is little to change or add.

detection_appmidi_channel_app
screenshots of the detection application

One thing that I will mention is that currently I am working on a simple client-server mechanism to use the webcam of another Mac in the same network. This is based on a robust and fast mechanism (using a UDP connection and jpg-compression). In my ‘freetime’ I will try to make this more perfect… It might open the posibility to perform on a more independent spot, apart from the main computer, running Ableton and my detection application.

2019 April

Recordings

I recorded a swarm of bees near their beehive. Analyzing the recordings at home I found their sound is centered around 241 Hz. This corresponds with values I found in literature. So this sound is musical in its own way and can be combined with the sounds of waves and the wind, as I expected.

Demo’s

Here you can watch the demo of the waves, the bees and the wind. There is also a pastor in the background, calling for his animals. Detection method is color detection.

I also made a demo to show the possibilies of my software for controlling volume and speed of 4 channels with drums.

This goes ‘beyond El Hierro’ but might be interesting for people who love to use the instrument for making (techno) music. This shows that with gestures you can control Ableton’s effects and plug-ins: in fact it is a sophisticated MIDI controller.

Events

Last week I presented my project to the audience of my Spanish class in Valverde (El Hierro, Canary Islands), in Spanish. I concentrated on reaching a state of ‘clairaudience’, as described by R. Murray Schafer. I challenged the people in the audience to describe the sounds they hear for about 2 minutes in the small town. Then I asked them to make a drawing from this experience. Finally I asked them to describe the sounds they heard at the time they were about 6 years old. In this way they became aware of the concept of ‘soundscape’. The results on paper varied, but for many it was all very new and abstract and at the same time also a revelation. These exercises were followed by a simple technical explanation and some demos. A few enjoyed playing with the instrument.

presentacion-eva
(Photo by Eva, 2019)

Technical

I experimented with and implemented the ‘Midi remote scripts’ in Ableton. These can be made on the Remotify website (you have to create an account).
On this site you can create a controller script and load it in the Preferences of Ableton. The script fixes connections between the controls of Ableton and received midi cc messages. They can easily be applied in other Ableton projects too, so the connections do not have to be made again and again. Examples of connections are:
– pan per channel
– start and stop recording