This post is written in English.
Covid-19
This year has been rather strange, not only for you as readers of this blog, but also for us here on El Hierro! In February we made a trip to La Palma, our neighbour island at a visual distance of 80km. But returning home, we entered the Spanish State of Alarm because of Covid-19. This continued until July, and for me going outside making new recordings became impossible. In July the State of Alarm was over and we could go outside again. But then the island was overwhelmed by tourists, more than ever, only from the other Canarian Islands to enjoy their new freedom and security (we still have only few cases here) to see friends and relatives. So what about “El Hierro, tranquilidad…” as many visitors say here, even without knowing the island…
It is clear that Covid even has its effect on a more or less stand-alone project like this. I hoped at the beginning of 2020 to make a lot of new recordings and movies, but things went differently. Read on …
Musical findings
First some musical aspects before describing more technical details. During the time in confinement, I listened and experimented with the tools that I have now. As I wrote earlier, I was impressed by the orchestral works of Ligeti and that inspired me to use natural sounds of my recordings combined with microtonal effects. The (Ableton) samplers that I use have options for very small tonal changes and combined with multiple voicings, “portamento”1) or “glide”2) effects give very interesting results. This is especially true for long sounds with a duration of several seconds. I hope to add some examples here in the near future. At this moment I am still in the experimental phase.
A new Mac
In the meantime I was worrying about what would happen in the future if my old iMac (from 2007) became rather outdated or even worse. Just after I did some investigation on my possibilities, the bad things happened: first the hard disk nearly stopped working. After a replacement also the graphics processor stopped working and now the old iMac is blind and only usable as backup disk. Now I have a MacBook Pro with MacOS 10.14. (Mojave) installed. For development this meant installing and configuring new versions of Xcode, Qt, etc., hoping that my sources would compile on this machine in this new ‘environment’. Fortunately, with some small adjustments this was ready in about a week. It could have been worse…
Because of this change I went on streamlining and refactoring my code and of course I repaired some bugs and irregularities. I also spend a lot of time on small refinements to cope with a more natural and intuitive object detection. Due to Covid I had plenty of time.
Because I use Qt for development, it cannot be very difficult to build a Windows version too, but I better leave this to someone with more Windows expertise. If you think to be the right person for this, you can contact me.
New detectors, based on Artificial Intelligence
Reading some posts on the blog of LearnOpenCV.com inspired me to dug further into artificial intelligent (AI) solutions to achieve better and more stable detection methods. Using AI, in whatever field or scope, always involves a training phase for software to generate a large dataset (think of ‘Big Data’) to be used for discrimination between positive and negative (detection) results. With these datasets software programs can answer questions like “Is this a cat?”, or “Is this a US president?”, or even “Is this a criminal?” (be aware that these datasets might have some bias…). A training phase is always necessary to answer questions with a high degree of probability. With this in mind, I built a separate program to build and train a new object detector (a so called HOG detector, from Histogram of Oriented Gradients) with a large set of pictures of my webcam and use that detector in my original software. I can train the detector on any object that I have at hand.
The results are good albeit that this detector is rather vulnerable for small deviations from the learned dataset: small rotations of the object can lead to false negative or positive detections. My older detectors (trackers) work a bit “smoother” but their detection view always grows after some time or moves away from the original object. That is not the case with this new detector. Even losing the object out of sight is not a problem, it catches the object once again in view. Multiple similar objects are also detected. This type of detector is very reliable and stable over time. Achieving a frame rate of 15 frames per second is not bad for this application.
These days a lot of people are using Zoom and Skype to communicate. Both applications have a feature for replacing the background from your webcam with a picture, to avoid other people to have a close look in your home or office. This feature inspired me to explore more of the functionality that OpenCV has to offer and I think there are usable effects to try. That’s for the next blog post.
More outside interest
During the year I met some people who might be interested in an adapted form of my software. It can be used in schools or individually by children or young people who need other forms of nonverbal expression as an addition to ordinary communication. It also can serve people with a motor disability.
I hope to meet more people because I’m curious to see my program being used in all kind of circumstances. In mutual collaboration we must find a way to make this work. If you are interested, please feel free to contact me via the contact page.
Latest results
After all this hustle and bustle with Covid, facemasks and computers I am now again at the point to combine new sounds.
Here is my latest result with the new HOG detector. I used four different sounds to create a soundscape for the pastors (herds) of this island. There still are a few of them here… This video also demonstrates that the detector can detect multiple similar objects and generate more sounds simultaneously.
So now I keep my arms crossed for next year 2021. Again I hope to post some additional results. I wish you al the best, and as they say here: ¡Cuidate! (Take care!)
Now I can go on, I have to go on …
1) see portamento
2)see glissando
Paul’s Project 