Software & Prototypes

Work Package 1

Developed by researchers in the SMC Group at INESC TEC and CITAR at UCP.
Some of the Work Package 1 prototypes are described as part of a report in Exame Informática.

  1. Loopalooza (INESC TEC): is a software application that can control the syncopation of rhythmic performances in audio format. Control is based in a set of formalized transformations that can effectively remove and create new syncopations in a rhythmical pattern. Demo video here.
  2. Cellular Automata workbench (UCP). This package explores complexity in one-dimensional linear cellular automata (CA). It offers visualisation and a graphic user interface for parametric control. Mapping assumes the creation of a musical interpretation of CA complexity rooted in a given musical aesthetic. The development of non-trivial mapping strategies is a core challenge. For more information see here.
  3. WindChime (UCP) is a large-scale real-time audiovisual installation. A program collects live weather data (intensity and direction of the wind) from thousands of servers spread around the world. This information is interfaced with a particle system, a swarm of small objects that we think of a virtual dust. WindChime displays a map of the word and visualized weather patterns and how they influence the particles over time. In addition, this work features generative sounds reflecting the complexity of the weather patterns. WindChime implements a virtual version of the archetypical wind chime, a tube structure producing tinkling sounds in a light breeze. For more information see here.
  4. Crawlers (UCP) are animal-like creatures, commonly referred to as ‘animats’. The simple two-legged virtual creatures live and interact in a two-dimensional universe. Crawlers’ behavior is conditioned by the structure, size and flexibility of their legs in addition to their energy consumption.  A computer program coordinates all these parameters to generate dynamic simulated animat locomotion. Again, complex behavior emerges from the interaction between different animats , between the joints of the legs and between the legs themselves. Finally, one might also think of them as them as virtual musicians: the animat movement patterns are sent to second program that interprets and outputs the bodily gestures as musical sounds. For more information see here.
  5. earGram (INESC TEC / FEUP): is software for interactive exploration of large databases of audio snippets in Pure Data. earGram extends concatenative sound synthesis to embed generative strategies as selection procedures and aims at exploring creative spaces with reduced user interaction. The software has four methods that automatically re-arrange and explore a database of sound events (corpus) by other rules then their original temporal order into musically coherent outputs. Demo Video here.
  6. Porto Actors with earGram (UCP / INESC TEC): is a two layer interactive audio-visual system composed of the multi-agent “Actor” model which drives the real-time sound synthesis engine “earGram”. For more information see here.
  7. Improvasher (INESC TEC): is a real-time musical accompaniment system which creates an automatic mashup to accompany live musical input. Improvasher is built around two music processing modules, the first, a performance following technique, makes beat-synchronous predictions of chroma features from a live musical input. The second, a music mashup system, determines the compatibility between beat-synchronous chromagrams from different pieces of music. Demo video here.
  8. Spatium 2.0-a1 (INESC TEC / FEUP): is a piece of spatialization software developed for the CARA concerts (December 2014), controlled by dynamic and gestural spatialization interfaces. See more information on the CARA Ano Zero concert here.
  9. RAMA – Relational Artist Maps (INESC TEC): was developed as an app for Spotify which allows users to interactively explore a network of their favourite music artists and make personalised music playlists. The source code for RAMA is available here. Note: RAMA is not supported in the most recent versions of Spotify due to significant changes in the API for app development.
  10. Conchord (INESC TEC): is a system for real-time automatic generation of musical harmony through navigation in the Tonal Interval SpaceIt was implemented for both Max and Pure Data and allows expert and novice users to creatively explore musical harmony by assisting them in the automatic generation of ‘generic’ chord progressions characteristic of Western tonal music. For more information, see here.

Software code from the SMC Group at INESC TEC is available on this GitHub repository.

Work Package 2

Developed by researchers in the CTM at INESC TEC.

  1. People localization in indoor spaces using video. Use of video cameras and standard computers to detect people in real time performing their localization in the space. The system enable s the localization of people in the images and in the 3D space; these functionalities are necessary for higher level processing such as event detection or activity classification.
  2. Video annotation application for physical activity analysis. A graphic user application to assist the annotation of videos. A first version of an application being developed with a particular focus in the characterization of physical activity. The ultimate goal is to annotate, as easily as possible, and process videos capturing different activities to classify the degree of activity of each individual. This information is to be used by researcher to validate algorithms and understand the factors that favor exercised and consequently contribute to fight obesity.

Work Package 3

Development by GIG Group at DEI/FEUP. Work Package General coordination  António Coelho.
Rui Nóbrega , João Jacob and Ricardo Baptista.

Applications

  1. Invicta - Mobile Historical Tourist Learning Assistant (FEUP/INESC TEC). Mobile application for tourist destination exploration (e.g., city of Porto) using location-based games and storytelling.  With a special focus on the city of Porto, the game mechanics features a location-based game to assist and engage the player in the discovery of the main city landmarks. An offline mode allows the tourist to acquire further knowledge about the city and the visited landmarks while resting at the hotel. Additionally the application provides an in-game assessment in order to certify the user’s knowledge about the city.
    [Video]
  2. ARTourMap (FEUP/INESC TEC). Tourist map exploration through Augmented Reality. Mobile application that  features city and location-based information; augmented reality overlay on top of the official city map; and automatic Point of Interest (POI) and building data extracted from freely available web-sources. Proof-of-concept city of Porto and FEUP map using 3D generated map prototype.
    [Video]
  3. DCTools / Driving Simulator (FEUP – INESC TEC / ISEP). Certification tools for driving simulators. These tools integrate with existing driving simulators in order to provide serious games that certify the training. The instructors are provided with the possibility of designing training plans, that are automatically generated into game levels that trainees are challenged to overcome. These tools also make the automatic assessment of the student, in real time, and evaluate if the trainees achieve the level of skills required. From this data, the game levels are adapted to the trainees’ progress and, at the end of the serious game, it is possible to certify that the trainee has achieved the skills specified by the instructor.
    [Video]

Software Products:

  1. Location-based augmented reality solution (Rui NóbregaJoão Jacob).
  2. Unlocking Porto prototype (link).
  3. Invicta: Historical city exploration and assessment. (link) (.apk)
  4. Lost in FEUP: Augmented Reality for university campus exploration [Video].
  5. XML and documentation.
  6. Location-based serious game development Unity package.
  7. Other software.