End of summer 2021

This website is written in English. Select your prefered language (right or below).

Welcome everyone and also some new readers of this blog!

What happened on El Hierro since my previous post in December 2020?

We had a fresh spring 2021 here on El Hierro, plenty of rain after the long draught, with winds from the ocean and not much sun. All very welcome for the vegetation and agriculture, and especially of course in our garden. Our red banana tree started to flower for the first time and produced two hands of small red bananas. After some months its flower dropped and I cut the fruits. I let them ripen for some days in a plastic bag and then they were really, really delicious, better than any banana I ate before. The tree died but had produced its offspring: two new trees. Maybe they produce more next year.

But gradually the weather changed during the summer, it started to warm up, and with a calima now and then we had a nice time. Further in July and august after the ‘opening’ of the Canary Islands a lot of tourists, even more than last year, were disturbing the silence they are searching here…, what a controversy. Impossible for making recordings of weak sounds. But let’s leave the weather and the tourist (for now).

What progress did I make this year?

I have done a lot of technical activities and improvements but also some new musical experiments. The technical improvements are written below in some paragraphs, the other steps follow here.

Experimenting with the new natural and artificial sounds

Because I am also a local collaborator and observer of landslides in the new international AGEO project for geohazard risk management, I want to make sound recordings of landslides. They happen now and then near my house (on a save distance…) and when I am fast enough I can grab their interesting sounds. I use them to report the events to the AGEO website (with this recordings, some pictures and geological data) but I can also use them for my project here. They also are sounds of El Hierro…

I did some experiments with the landslide sounds and they look very promising (look at the video below the last paragraph). I have to capture more and different sounds. Of course I looked on the web for these kind of sounds but they seem rather artificial to me. I have to look further… or record them myself.

Also more artificial sounds become available to use in my program with Ableton Live. Especially the Spitfire Audio plugins (LABS) are promising but I have to find out how to combine them with the natural sounds.

Contact with a Belgian sound producer

A Belgian sound producer asked me if my program could be used in a nice theatre production, to be performed in Belgium and France. He showed me some videos and I was convinced I could contribute to his work by giving him a (limited) version of my program. I took some time to prepare my program to work for a short period of time and only on one dedicated machine. Unfortunately without any reason or notice the contact was stopped between him and me …

Here people say “Es lo que hay”, it is what it is…

The La Palma volcano

And since a month on La Palma, our neighbour island, a very dangerous eruption of a new volcano is happening. Many people lost their homes, their work, their fincas, their schools… Of course this is very interesting for scientists who are able to fly their drones over the fissures with boiling lava, do measurements and distribute their pictures often in real time on the local and international news pages like Eldia or ElPais. I am interested in the sounds that this phenomenon is producing, but I can only record the sounds here at home from internet, at a distance of some 80km. And brushing away the very fine ashes that fall from time to time around and even inside the house.

Currently I am experiencing with these sounds, although they are not from El Hierro.

Internal technical improvements

Experimenting with new sounds brought to light new shortcomings in my software that needed a solution. I made a lot of internal improvements, not visible from ‘outside’ but that are only emerging in special cases. It’s a long list that does not need to be published here. Sometimes they take a lot of time and investigation, but of course a lot of satisfaction if there is a positive result. Many improvements are audible under certain circumstances and that is very positive. Also the internal streamlining of the user interface kept me busy for a great deal of time. And there was a severe problem that caused the program to crash while quitting.

The contact with external users made me think about how to distribute this software without all the hassle of property rights and Apple stores, Github etc. I think a time limited version for a dedicated computer might be an optimal solution. That is now implemented. I hope this will work on Macs with higher versions than OSX Mojave, be it that I dont have these machines here… I have to look for further appropriate solutions once there is an interesting partner to collaborate with. Writing a manual is also something to keep in mind…

Distributing this software made me also puzzling how to make it run even on an ’empty’ machine. As I use several external libraries, these and their dependencies have to be incorporated in the final application, this is a tedious process. Deploying software like this often looks like building a pyramid in historical Egyptian times, many dependencies are handled by deployment tools, but other are all hand work and require a lot of tedious testing. Fortunately in these times for every problem, big or small, you can find solutions in blogs on the web.

But here some important steps I made.

Using an iPad as camera

Free software (as Reincubate’s Camo Studio) is available to use an iPad, connected via USB with the Macbook, as external camera. This is really nice because the quality of the iPad camera is much better and faster than the Facetime camera of the Macbook. I hope someday they have, as they promise, a version that works via WiFi, like Epocam in the past. This makes the performers more independent from the more fixed desktop apparatus.

Implemented the use of movies

I added the possibility in the software to use videos in addition to the webcam. This also gives a good idea how and when this program is functioning at its best. I discovered that the detection algorithms often can work much faster than I previously thought. The camera of my Macbook is often the limiting factor in the whole chain, especially in poor lighting conditions. I can speed up movies to much more than 30 fps (frames per second) without disturbing the detection, while the webcam’s maximum is often about 15 fps. See also the next paragraph on streaming.

Implemented the use of RTMP streams

Although I had implemented a mechanism to use the webcam of another mac in a local network as input, this is not a generic solution. So I investigated time to explore rtmp streaming. With this another computer with streaming software (as OBS) and a special rtmp server (as Local RTMP Server) can function as input for my program. This is true for local and external sources. With this method I have to take into account some latency but that is not always a big problem. It even opens the possibility of using more computers with my software connected to the same camara stream.

New stereo effect on samples

In Ableton Live the use of samples is very flexible and still I am experimenting with its possibilities. There were some restrictions in generating stereo effects using them in ‘the simple mode’: only the ‘pan’ of whole sample can be manipulated this way. This gives rather unwanted switches of the pan because the noisy nature of object and motion detection. Now I use within one track 51 copies of the same sound sample, each copy is connected to one note and one pan position. This effect causes a generated sound (note) to be heard on its own pan position, while other sounds from the same sample can be heard at other pan positions. This gives a much more real ‘picture’. I am experimenting with the sounds of the landslides, and the result sounds promising. Here is a new example where you can hear the stereo effect together wth the sound of landslides.

sound of landslides on El Hierro, 2021

This was the news for now, written on my 70th birthday. I hope to give more new examples soon.

By the way, if you meet people who are interested in this project, give them the webaddress of this site!

End of 2020

This website is written in English. Select your prefered language (right or below).

Covid-19

This year has been rather strange, not only for you as readers of this blog, but also for us here on El Hierro! In February we made a trip to La Palma, our neighbour island at a visual distance of 80km. But returning home, we entered the Spanish State of Alarm because of Covid-19. This continued until July, and for me going outside making new recordings became impossible. In July the State of Alarm was over and we could go outside again. But then the island was overwhelmed by tourists, more than ever, only from the other Canarian Islands to enjoy their new freedom and security (we still have only few cases here) to see friends and relatives. So what about “El Hierro, tranquilidad…” as many visitors say here, even without knowing the island…

El Hierro, tranquilidad…

It is clear that Covid even has its effect on a more or less stand-alone project like this. I hoped at the beginning of 2020 to make a lot of new recordings and movies, but things went differently. Read on …

Musical findings

First some musical aspects before describing more technical details. During the time in confinement, I listened and experimented with the tools that I have now. As I wrote earlier, I was impressed by the orchestral works of Ligeti and that inspired me to use natural sounds of my recordings combined with microtonal effects. The (Ableton) samplers that I use have options for very small tonal changes and combined with multiple voicings, “portamento”1) or “glide”2) effects give very interesting results. This is especially true for long sounds with a duration of several seconds. I hope to add some examples here in the near future. At this moment I am still in the experimental phase.

A new Mac

In the meantime I was worrying about what would happen in the future if my old iMac (from 2007) became rather outdated or even worse. Just after I did some investigation on my possibilities, the bad things happened: first the hard disk nearly stopped working. After a replacement also the graphics processor stopped working and now the old iMac is blind and only usable as backup disk. Now I have a MacBook Pro with MacOS 10.14. (Mojave) installed. For development this meant installing and configuring new versions of Xcode, Qt, etc., hoping that my sources would compile on this machine in this new ‘environment’. Fortunately, with some small adjustments this was ready in about a week. It could have been worse…

Because of this change I went on streamlining and refactoring my code and of course I repaired some bugs and irregularities. I also spend a lot of time on small refinements to cope with a more natural and intuitive object detection. Due to Covid I had plenty of time.

Because I use Qt for development, it cannot be very difficult to build a Windows version too, but I better leave this to someone with more Windows expertise. If you think to be the right person for this, you can contact me.

New detectors, based on Artificial Intelligence

Reading some posts on the blog of LearnOpenCV.com inspired me to dug further into artificial intelligent (AI) solutions to achieve better and more stable detection methods. Using AI, in whatever field or scope, always involves a training phase for software to generate a large dataset (think of ‘Big Data’) to be used for discrimination between positive and negative (detection) results. With these datasets software programs can answer questions like “Is this a cat?”, or “Is this a US president?”, or even “Is this a criminal?” (be aware that these datasets might have some bias…). A training phase is always necessary to answer questions with a high degree of probability. With this in mind, I built a separate program to build and train a new object detector (a so called HOG detector, from Histogram of Oriented Gradients) with a large set of pictures of my webcam and use that detector in my original software. I can train the detector on any object that I have at hand.

The results are good albeit that this detector is rather vulnerable for small deviations from the learned dataset: small rotations of the object can lead to false negative or positive detections. My older detectors (trackers) work a bit “smoother” but their detection view always grows after some time or moves away from the original object. That is not the case with this new detector. Even losing the object out of sight is not a problem, it catches the object once again in view. Multiple similar objects are also detected. This type of detector is very reliable and stable over time. Achieving a frame rate of 15 frames per second is not bad for this application.

These days a lot of people are using Zoom and Skype to communicate. Both applications have a feature for replacing the background from your webcam with a picture, to avoid other people to have a close look in your home or office. This feature inspired me to explore more of the functionality that OpenCV has to offer and I think there are usable effects to try. That’s for the next blog post.

More outside interest

During the year I met some people who might be interested in an adapted form of my software. It can be used in schools or individually by children or young people who need other forms of nonverbal expression as an addition to ordinary communication. It also can serve people with a motor disability.

I hope to meet more people because I’m curious to see my program being used in all kind of circumstances. In mutual collaboration we must find a way to make this work. If you are interested, please feel free to contact me via the contact page.

Latest results

After all this hustle and bustle with Covid, facemasks and computers I am now again at the point to combine new sounds.

Here is my latest result with the new HOG detector. I used four different sounds to create a soundscape for the pastors (herds) of this island. There still are a few of them here… This video also demonstrates that the detector can detect multiple similar objects and generate more sounds simultaneously.

Sonidos de pastores, used with coasters of the Spanish “Paradores”

So now I keep my arms crossed for next year 2021. Again I hope to post some additional results. I wish you al the best, and as they say here: ¡Cuidate! (Take care!)

Now I can go on, I have to go on …


1) see portamento

2)see glissando 

Start of 2020

In this post you can read about the progress of my project in 2019, after april. You also can read my previous posts because now and then I added some new examples…
I hope to post more this year.

Musical findings

In 2019 first I experimented a lot with my recorded sounds in Ableton, and later with some sounds from Spitfire Audio plugins. The sounds of these plugins are based on recordings from real instruments, orchestras, the London metro, etc. They sometimes have funny and unpredictable effects, especially when using the ‘Dynamics’ effect in combination with long sounds. I was experimenting a lot with these sounds in my program, and I wanted to find out if they would combine very well with e.g. my wind and sea sounds.
Independent of this, but at the same time, I discovered the works of the composer György Sándor Ligeti. I saw the very impressive orchestral performance of the piece Atmosphères by the Berliner Philharmoniker, conducted by Sir Simon Rattle. By using a new technique called micropolyphony, Ligeti in fact created very dense ‘soundscapes’ that are built of sounds that move at different speeds and have different tonal spacings. Here I quote how Ligeti described it himself:

Technically speaking I have always approached musical texture through part-writing. Both Atmosphères and Lontano have a dense canonic structure. But you cannot actually hear the polyphony, the canon. You hear a kind of impenetrable texture, something like a very densely woven cobweb. I have retained melodic lines in the process of composition […]. The polyphonic structure does not come through, you cannot hear it; it remains hidden in a microscopic, underwater world, to us inaudible. I call it micropolyphony (such a beautiful word!)

I was struck by the similarity between the sound effects of the Spitfire plugins in my project and the orchestral sounds of Ligeti. This discovery widened my view on a new musical area. As for me, Ligeti bridged the gap between traditional orchestral compositions and soundscapes.
Although I prefer to use my own recordings, using the plugin sounds and listening to Ligeti give me a better understanding of when sounds are musical or produce music in my opinion. Based on these findings, I begin to see great opportunities to create new music with my recordings but still more research is needed to grab the ideal sounds in nature that can be useful.
I also will concentrate on microtonal effects, by not using the discrete whole steps as in western chromatic scales, but using more subtle pitch changes.

Audio recordings

In summer, apart from the motorbikes and boom cars of a few tourists, El Hierro is very quiet: birds are silent, the sea is calm, everything is at rest. Often you only hear your own heart beat. Now, at the turn of the year the singing of the birds starts again, you hear the bees and the sea and the wind gets rougher.
I already made a number of new recordings based on my new experiences and this expedition continues in this year.

piedra_el_rey
at Piedra el Rey, on the footpath to the Fuente de Mencáfete

Programming

While experimenting with my sounds, making movies, using my notes, searching through folders on my computer I was getting tired of the way my process was hindered by the implied structure of my computer. So I started searching for some tools to make life easier. The result is that I created a rather complete set of software tools that I use simultaneously to create new instruments with my recordings, make notes, describe findings etc. Here an overview.

For maintenance of al my recordings, pictures etc., I used a journal, a word-document with all data and pictures. But for experimenting with the sounds this was not very useful, it required a lot of searching, switching applications and so on. So I started to look for a better tool and I came upon Swinsian, comparable with iTunes. This software is ok, but I made a new user interface for faster and more adequate searching, playing, copying and pasting, and annotating, based on the Swinsian database. It now works very smooth and I use it side by side with Ableton to build new instruments.

maintainance_app
screenshot of the maintenance application

For synchronising my recorded videos from my detection application and audio from Ableton, I used some basic ffmpeg commands, build in an Applescript. This was not as flexible as I needed, especially when some shifting of the audio was necessary to adjust audio and video to fractions of a second. Synchronising audio and video is a time consuming process, so a good and responsive tool is not a luxury. Standard commercial applications are very clumsy to perform these tasks or are very expensive. My detection application generates videos with a frame rate nearly twice as much as necessary for the final result. I did this because the detection mechanism can cause some timing irregularities in the video recording, due to the limited processor capacity. By scaling back the video frame rate afterwards by adjusting it to the audio, these irregular effects smooth out and disappear. So I ‘wrapped’ the ffmpeg commands in a C++ program with functionality to adjust audio and video with a precision of 0.01 sec, and with a simple efficient user interface.

ffmpeg_app
screenshot of the synchronize application

The third programming project was the complete rebuild of the user interface of my original detection program. During my work on the previous programs, I learned a lot more about the Qt/C++ programming environment. This rebuild made the program structurally less complicated and easier to maintain. My previous remarks, that it was finished were a bit premature, but now I think there is little to change or add.

detection_appmidi_channel_app
screenshots of the detection application

One thing that I will mention is that currently I am working on a simple client-server mechanism to use the webcam of another Mac in the same network. This is based on a robust and fast mechanism (using a UDP connection and jpg-compression). In my ‘freetime’ I will try to make this more perfect… It might open the posibility to perform on a more independent spot, apart from the main computer, running Ableton and my detection application.