This blog is only in English.

Welcome back to my BLOG pages.

This is a long post of some changes and an important addition to my website. I have written a new and rather elaborated explanation of my motivation for this project. So please read the whole post!

Languages

I have added Spanish versions of some pages on this website. The Home page is now bilingual. The Project pages have separate English and Spanish versions. The other pages are (still) only in English.

Because of these changes I had to reshuffle the menu, and did an update here and there.

New audio

I recently created a new soundscape with my software, I called it “The Tuning of an Unstable World”, rather applicable these days! Some people have heard it already and gave me many positive reactions. So now I want to spread it further to the readers and listeners of this site.

The title is a variation on the title of the book “The Soundscape: our sonic environment and the tuning of the world” (1977) by R. Murray Schafer. He wrote, “… the earth forms the body of an instrument across which strings are stretched and are tuned by a divine hand. We must try once again to find the secret of that tuning.”

This soundscape is about the present turbulent world we live in and the almost impossibility to find such a secret.

For this soundscape I used sounds of El Hierro: the tuning of a local orchestra, the sounds of a landslide near my house and of the ever-blowing wind. It is composed by moving my hands in front of the webcam of my laptop.

You can listen to this and more on my Soundcloud page.

New friends, a new explanation

For new friends I had to reformulate my explanation for doing all this work. It is good to do this after some 5 years of experimenting and programming. Am I still on my way…?

inspiration

In the past I have been searching for the ‘musicality’ of all kinds of sound, mostly natural sounds, but also some anthropogenic sounds such as machines, concrete mixers, and special musical instruments. Since a long time those built by Luigi Russolo (1885-1947) or such as the Theremin inspired me. Or watch this beautiful clip: “Frank Zappa Teaches Steve Allen to Play The Bicycle” (1963). Then maybe you have an idea of what was and is driving me…

recorded sounds

A recorded sound might be interesting or ‘musical’ on its own, but often ‘comes to live‘ when manipulated with changes in pitch, timbre, volume, adding special effects like reverb, echo, stereo widening etc. Combining different sounds opens up new dimensions. Simultaneous sounds can form an ‘orchestra’ and can interact, communicate or even fight with each other.

contemporary experiments

I listen to a lot of audio experiments on the internet, some beautiful, others terrible, but for me the definition of what makes a sound ‘musical’ is still rather vague. I can hardly put it into words. It is personal, emotional and a matter of taste.
I was curious to listen to modern compositions and experiments, especially coming from the French Ircam forum that pretends to be the ‘avant-garde’ in this field. Contemporary composers don’t shy away from using algorithms and mathematical structures, like the golden ratio, Fibonacci sequences, even using artificial intelligence, but seldom I am convinced that these techniques enrich the pure musicality of their pieces. I wonder if long streams of monotonous sounds with impressive and intrusive sound effects that make me think of yoga sessions, shopping centres or waiting rooms, represent the final intended results of these very complex techniques and the sincere intentions of the composers.

possible criteria

I have to think of compositions by the Dutch composer Jan Boerman (1923-2020), for example “De Zee” (1966). He composed a lot of electronic music in the 1960’s, and recorded and mastered his pieces only on magnetic tape, by cutting and pasting with scissors and glue. He articulated very meticulously what he was looking for in sounds, especially those generated by the analog equipment (tone generators, noise generators, filters etc.) that he had at his disposal. He wrote about ‘transparency of sounds‘: “A sound must be transparent for intentions and emotions”.
I appreciate the use of this visual metaphor: there is something ‘behind’ the sound that moves you. You have to look ‘through’ a thin curtain at something beautiful that is behind. Without this metaphor, it’s hard to explain how to enjoy sounds and I can’t make it clearer than this.

Comparing “De Zee” with aspects of general modern compositions or soundscapes, perhaps an anachronism given the resources available to Boerman and today’s composers, would be the best way for me to formulate some criteria for what I consider as ‘musicality’. But that would take too many words here, or is not espressible in words at all. In brief this is what I can say now about what sounds should ‘do’ to me:

• seduce me to unknown places and stimulate my fantasy. Then I can listen again and again and every time something new is happening to me. Isn’t that a criterion for a real work of art?
• fitting of sounds. I hear and make pieces but often the sounds don’t fit with each other, or have no relationship with each other. Therefore they cannot build tension, a climax or relax.
• maladjusted or inappropriate spaces. Sound effects sometimes seem to be very beautiful at first ‘sight’ but the sounds then lack a relation with a physical location where the recordings are supposed to be made. The effects move the sounds and me to a maladjusted or inappropriate space. Acoustical effects must seduce me to a place of fantasy, but if not well done or if nothing really ‘happens‘ in that place, I start to think “What am I doing here?”

my experiences

My own experience in the last 5 years is that manipulating sound only in a few cases might result in success. I discovered that the presence of something like pitch in sound is important. I can manipulate pitch easily (digitally) to make a sound more ‘musical’. Boerman also discovered this. But also atonal sounds, such as from my ‘desprendimientos’ can become ‘musical’, but not always. Manipulating the timbre of a sound by using filter techniques can add to the musicality to a great extend.

Recent neuroscientific research shows that the locations of perception of pitch and timbre in human brains are generally overlapping. Probably these causes similar, intertwined and maybe indistinguishable experiences. This is the field of ‘tonotopy’, the study of the mapping of sound (in particular pitch and timbre) to locations in the human brain. I would like to read more about this…

The combination of several different sounds with different pitches, timbres, effects and dynamics is a problem ‘of higher order’ for me. When I combine natural sounds with digital synthesizer sounds, I have to be very careful, it quickly becomes too smooth, too glossy and too ‘beautiful’ or ‘inappropriate’.

I hope to find some more clues what makes combinations musical. It still is a big challenge for me to get a grip on this. There is a lot of research publications on musical perception available and surely I will read some, but first I have to explore and research my own experiences!

human gestures

For me it is very important to work with ‘natural’ or bodily gestures to generate and orchestrate sound to music. A keyboard or guitar is mentally and sonically very far away from the produced sounds. A musician is often very concentrated on combining his manual, vocal, gestural and electronic techniques in order to express intentions and emotions by means of sound. Finally these techniques can ‘merge’ within a person or a group, but there is a long bridge to cross. I try to cross that bridge by using natural gestures with my hands, body, by manipulating objects, walking, jumping, etc. I experienced that these movements can come mentally and sonically much closer to the manipulation and generation of sound. By doing this, supported by my software, there grows in me a direct and natural link between producing and hearing sound. This requires another ‘mindset’ if your favourite instrument is a piano or a bass guitar!

conclusion

So, to come to the point, what is my purpose with all this work? I am convinced that playing with sounds and manipulating sounds in combination with bodily gestures is a congenital way of making expressive music that can generate some form of ‘alienation’ in me as ‘composer’ and can open my mind for unknown viewpoints, capabilities and experiences. Not only for music, but also for other aspects of life.

Before Russolo nobody would consider my music as ‘musical’. Today, after a century, some of us are liberated from old frontiers by listening to products of ‘sound designers’ who abandoned traditional criteria. This had great influence on all kind of audiovisual products, from websites, movies to video games. I can hear this every moment of the day if I want to.

Creating sound by making gestures is becoming an essential aspect of my life that offers me many possibilities to travel to unknown places and tell about these to my public!

In fact, listening is an essential aspect of life that silently seduces us to enter new environments behind the sounds, provoking all kind of new experiences, not because of the perception of sound itself, but by opening one’s mind. In the past decades there is a strong tendency to close our minds, be it by external oppression, repression or individualistic and protective self-imposed restrictions, or simply the fear for strangeness.

The current time needs people with an open mindedness to solve large intertwined problems to generate a peaceful and dignified societies. I hope to provide a sincere contribution to this with this project.

This blog is only in English.

Welcome everyone and also some new readers of this blog!

What happened on El Hierro since my previous post in December 2020?

We had a fresh spring 2021 here on El Hierro, plenty of rain after the long draught, with winds from the ocean and not much sun. All very welcome for the vegetation and agriculture, and especially of course in our garden. Our red banana tree started to flower for the first time and produced two hands of small red bananas. After some months its flower dropped and I cut the fruits. I let them ripen for some days in a plastic bag and then they were really, really delicious, better than any banana I ate before. The tree died but had produced its offspring: two new trees. Maybe they produce more next year.

But gradually the weather changed during the summer, it started to warm up, and with a calima now and then we had a nice time. Further in July and august after the ‘opening’ of the Canary Islands a lot of tourists, even more than last year, were disturbing the silence they are searching here…, what a controversy. Impossible for making recordings of weak sounds. But let’s leave the weather and the tourist (for now).

What progress did I make this year?

I have done a lot of technical activities and improvements but also some new musical experiments. The technical improvements are written below in some paragraphs, the other steps follow here.

Experimenting with the new natural and artificial sounds

Because I am also a local collaborator and observer of landslides in the new international AGEO project for geohazard risk management, I want to make sound recordings of landslides. They happen now and then near my house (on a save distance…) and when I am fast enough I can grab their interesting sounds. I use them to report the events to the AGEO website (with this recordings, some pictures and geological data) but I can also use them for my project here. They also are sounds of El Hierro…

I did some experiments with the landslide sounds and they look very promising (look at the video below the last paragraph). I have to capture more and different sounds. Of course I looked on the web for these kind of sounds but they seem rather artificial to me. I have to look further… or record them myself.

Also more artificial sounds become available to use in my program with Ableton Live. Especially the Spitfire Audio plugins (LABS) are promising but I have to find out how to combine them with the natural sounds.

Contact with a Belgian sound producer

A Belgian sound producer asked me if my program could be used in a nice and serious theatre production, to be performed in Belgium and France. He showed me some videos and I was convinced I could contribute to his work by giving him a (limited) version of my program. I took some time to prepare my program to work for a short period of time and only on one dedicated machine. Unfortunately without any reason or notice the contact was stopped between him and me …

Here people say “Es lo que hay”, it is what it is…

The La Palma volcano

And since a month on La Palma, our neighbour island, a very dangerous eruption of a new volcano is happening. Many people lost their homes, their work, their fincas, their schools… Of course this is very interesting for scientists who are able to fly their drones over the fissures with boiling lava, do measurements and distribute their pictures often in real time on the local and international news pages like Eldia or ElPais. I am interested in the sounds that this phenomenon is producing, but I can only record the sounds here at home from internet, at a distance of some 80km. And brushing away the very fine ashes that fall from time to time around and even inside the house.

Currently I am experiencing with these sounds, although they are not from El Hierro.

Internal technical improvements

Experimenting with new sounds brought to light new shortcomings in my software that needed a solution. I made a lot of internal improvements, not visible from ‘outside’ but that are only emerging in special cases. It’s a long list that does not need to be published here. Sometimes they take a lot of time and investigation, but of course a lot of satisfaction if there is a positive result. Many improvements are audible under certain circumstances and that is very positive. Also the internal streamlining of the user interface kept me busy for a great deal of time. And there was a severe problem that caused the program to crash while quitting.

The contact with external users made me think about how to distribute this software without all the hassle of property rights and Apple stores, Github etc. I think a time limited version for a dedicated computer might be an optimal solution. That is now implemented. I hope this will work on Macs with higher versions than OSX Mojave, be it that I dont have these machines here… I have to look for further appropriate solutions once there is an interesting partner to collaborate with. Writing a manual is also something to keep in mind…

Distributing this software made me also puzzling how to make it run even on an ’empty’ machine. As I use several external libraries, these and their dependencies have to be incorporated in the final application, this is a tedious process. Deploying software like this often looks like building a pyramid in historical Egyptian times, many dependencies are handled by deployment tools, but other are all hand work and require a lot of tedious testing. Fortunately in these times for every problem, big or small, you can find solutions in blogs on the web.

But here some important steps I made.

Using an iPad as camera

Free software (as Reincubate’s Camo Studio) is available to use an iPad, connected via USB with the Macbook, as external camera. This is really nice because the quality of the iPad camera is much better and faster than the Facetime camera of the Macbook. I hope someday they have, as they promise, a version that works via WiFi, like Epocam in the past. This makes the performers more independent from the more fixed desktop apparatus.

Implemented the use of movies

I added the possibility in the software to use videos in addition to the webcam. This also gives a good idea how and when this program is functioning at its best. I discovered that the detection algorithms often can work much faster than I previously thought. The camera of my Macbook is often the limiting factor in the whole chain, especially in poor lighting conditions. I can speed up movies to much more than 30 fps (frames per second) without disturbing the detection, while the webcam’s maximum is often about 15 fps. See also the next paragraph on streaming.

Implemented the use of RTMP streams

Although I had implemented a mechanism to use the webcam of another mac in a local network as input, this is not a generic solution. So I investigated time to explore rtmp streaming. With this another computer with streaming software (as OBS) and a special rtmp server (as Local RTMP Server) can function as input for my program. This is true for local and external sources. With this method I have to take into account some latency but that is not always a big problem. It even opens the possibility of using more computers with my software connected to the same camara stream.

New stereo effect on samples

In Ableton Live the use of samples is very flexible and still I am experimenting with its possibilities. There were some restrictions in generating stereo effects using them in ‘the simple mode’: ‘panning’ the sounds can be done only by panning the whole sample. This gives rather unwanted jumping left or right of the stereo location of the generated sounds because the noisy nature of object and motion detection. Now I use for example within one track 19 copies of the same sound sample, each copy is connected to 3 notes and one pan position. This effect causes the generated sound (the note) to be heard on its own panning position, while other sounds (other notes) from the same sample can be heard at other panning positions. This gives a much more realistic ‘picture’.

Now I am experimenting with the sounds of the landslides, and the result sounds promising. Here is a new example where you can hear the stereo effect together wth the sound of landslides.

sound of landslides on El Hierro, 2021

This was the news for now, written on my 70th birthday. I hope to give more new examples soon.By the way, if you meet people who are interested in this project, give them the webaddress of this site!

This post is written in English.

Covid-19

This year has been rather strange, not only for you as readers of this blog, but also for us here on El Hierro! In February we made a trip to La Palma, our neighbour island at a visual distance of 80km. But returning home, we entered the Spanish State of Alarm because of Covid-19. This continued until July, and for me going outside making new recordings became impossible. In July the State of Alarm was over and we could go outside again. But then the island was overwhelmed by tourists, more than ever, only from the other Canarian Islands to enjoy their new freedom and security (we still have only few cases here) to see friends and relatives. So what about “El Hierro, tranquilidad…” as many visitors say here, even without knowing the island…

It is clear that Covid even has its effect on a more or less stand-alone project like this. I hoped at the beginning of 2020 to make a lot of new recordings and movies, but things went differently. Read on …

Musical findings

First some musical aspects before describing more technical details. During the time in confinement, I listened and experimented with the tools that I have now. As I wrote earlier, I was impressed by the orchestral works of Ligeti and that inspired me to use natural sounds of my recordings combined with microtonal effects. The (Ableton) samplers that I use have options for very small tonal changes and combined with multiple voicings, “portamento”¹⁾ or “glide”²⁾ effects give very interesting results. This is especially true for long sounds with a duration of several seconds. I hope to add some examples here in the near future. At this moment I am still in the experimental phase.

A new Mac

In the meantime I was worrying about what would happen in the future if my old iMac (from 2007) became rather outdated or even worse. Just after I did some investigation on my possibilities, the bad things happened: first the hard disk nearly stopped working. After a replacement also the graphics processor stopped working and now the old iMac is blind and only usable as backup disk. Now I have a MacBook Pro with MacOS 10.14. (Mojave) installed. For development this meant installing and configuring new versions of Xcode, Qt, etc., hoping that my sources would compile on this machine in this new ‘environment’. Fortunately, with some small adjustments this was ready in about a week. It could have been worse…

Because of this change I went on streamlining and refactoring my code and of course I repaired some bugs and irregularities. I also spend a lot of time on small refinements to cope with a more natural and intuitive object detection. Due to Covid I had plenty of time.

Because I use Qt for development, it cannot be very difficult to build a Windows version too, but I better leave this to someone with more Windows expertise. If you think to be the right person for this, you can contact me.

New detectors, based on Artificial Intelligence

Reading some posts on the blog of LearnOpenCV.com inspired me to dug further into artificial intelligent (AI) solutions to achieve better and more stable detection methods. Using AI, in whatever field or scope, always involves a training phase for software to generate a large dataset (think of ‘Big Data’) to be used for discrimination between positive and negative (detection) results. With these datasets software programs can answer questions like “Is this a cat?”, or “Is this a US president?”, or even “Is this a criminal?” (be aware that these datasets might have some bias…). A training phase is always necessary to answer questions with a high degree of probability. With this in mind, I built a separate program to build and train a new object detector (a so called HOG detector, from Histogram of Oriented Gradients) with a large set of pictures of my webcam and use that detector in my original software. I can train the detector on any object that I have at hand.

The results are good albeit that this detector is rather vulnerable for small deviations from the learned dataset: small rotations of the object can lead to false negative or positive detections. My older detectors (trackers) work a bit “smoother” but their detection view always grows after some time or moves away from the original object. That is not the case with this new detector. Even losing the object out of sight is not a problem, it catches the object once again in view. Multiple similar objects are also detected. This type of detector is very reliable and stable over time. Achieving a frame rate of 15 frames per second is not bad for this application.

These days a lot of people are using Zoom and Skype to communicate. Both applications have a feature for replacing the background from your webcam with a picture, to avoid other people to have a close look in your home or office. This feature inspired me to explore more of the functionality that OpenCV has to offer and I think there are usable effects to try. That’s for the next blog post.

More outside interest

During the year I met some people who might be interested in an adapted form of my software. It can be used in schools or individually by children or young people who need other forms of nonverbal expression as an addition to ordinary communication. It also can serve people with a motor disability.

I hope to meet more people because I’m curious to see my program being used in all kind of circumstances. In mutual collaboration we must find a way to make this work. If you are interested, please feel free to contact me via the contact page.

Latest results

After all this hustle and bustle with Covid, facemasks and computers I am now again at the point to combine new sounds.

Here is my latest result with the new HOG detector. I used four different sounds to create a soundscape for the pastors (herds) of this island. There still are a few of them here… This video also demonstrates that the detector can detect multiple similar objects and generate more sounds simultaneously.

So now I keep my arms crossed for next year 2021. Again I hope to post some additional results. I wish you al the best, and as they say here: ¡Cuidate! (Take care!)

Now I can go on, I have to go on …

---

¹_{) see portamento}

²_{)see glissando}