The HYBRID CITY II: Subtle rEvolutions


The HYBRID CITY II: Subtle rEvolutions is an international biennale event dedicated to exploring the emergent technological character of the city. The event consists of a conference, workshops, and an online exhibition. The following project is part of my research and it is exhibited during May 2013.

In this project, computational media and sensor technologies are used to measure, analyze, and control aspects of the domestic environment. Reading the measurable world from macro to micro, a large number of possibilities may create unexpected, flexible, and personalized spaces that enhance living qualities of inhabitants, providing added layers of information, affectivity, and aesthetics with the use of calm technologies and ubiquitous computing. Fundamental consideration in this case is to construct sensate spaces that may establish the domestication of computational media with prior interest to elevate aspects of the inhabitants’ well-being, such as mood, emotion, experience, and perception.

Environmental conditions, spatial information, circulation, virtual and physical navigation, social media, or biosensors can collectively define quantitative or qualitative information that is used to properly adjust and personalize each environment and closely match taste and preferences. With the development of middleware applications it becomes even more feasible to approach this goal, providing necessary tools to create links between incoming data and outgoing processes, establish important automations, or suggest new creative and imaginative interactions. Therefore, it is possible to instantly create connections between an isolated sensor reading and projected visualizations, or use a number of similar sensors to control the overall interior lighting. Extracting specific keywords from social media messages or using sentiment analysis methods to define mood and emotion, it becomes possible to directly configure properties of a personal space as a multi-layered canvas. The final result of the configured space can provide a single pixel in the larger screen of the Hybrid City so as the overall well-being may be mirrored, provoke self-consciousness, and define a cartography of lifestyles and living conditions.

 



Art-Athina 2013 Installation

Art-Athina is one of the biggest art shows in Greece, with hundreds of participants around the world. This year I collaborated with Katerina Karoussos from the i-Node of Planetary Collegium, to create a telematic art piece with the use of a custom interactive media system. The system uses the video image that is captured in real-time from the camera that is connected to the remote computer, encodes the video file to a suitable format for network transmission, and streams audio and video to the destination in the gallery space. A number of remote locations - from Crete, to Jerusalem, and Paris - are streamed in the final realtime canvas that creates interesting narratives through real and virtual spaces.



Sensorama Workshop

Sensorama Workshop in GUC (German University Cairo), 1st & 2nd April 2013, together with Mike Phillips and Ziad Ewais, iDAT, Plymouth University.

The workshop consisted of the following modules:
- Sensors and the measurable world
- Sensor technology and Interactive Art
- Introduction to Arduino
- Using the Arduino Starter Kit (buttons, switches, LEDs, photo-resistors, potentiometers, etc)
- Arduino communication with Processing and MaxMSP
- Visualizing physical data
- Android SDK and mobile sensors (accelerometer, compass)
- Extracting brainwave frequencies with Mindwave
- Computer vision (blob detection, face recognition)


* Example code used in this workshop will be available to download soon.

The Source: DI-EGY Festival Exhibition, Cairo, 27th - 10th of April, 2013

The Source is an interactive piece co-created with Mike Phillips and Ziad Ewais, iDAT, Plymouth University for the DI-EGY Festival Exhibition, taken place in Gezira Art Centre, Cairo, Egypt, from 27th of March until 10th of April 2013.


This installation becomes a virtual space of a sonic and visual composition that is created in real time based on information captured from various sources around the world. XML and other data feeds from environmental sources are used by the system to define the score according to random events such as solar wind speed, earthquakes, or average temperature. The composition consists of layers/channels that exist from macro to micro structures, and manifest as granular noises, high-pitched frequencies, or low-bass resonances for the audio domain, and as spectrograms, streaming data text, or colorized planes for the visual domain. Moreover, streaming audio is received from various sources around the globe, or 3D objects morphed and animated according to specific events. A computer vision system is implemented within the space to track visitors' position and movement as this is the interaction method to interpolate between the different composition layers. When in stasis, the composition returns to a static and almost inactive position. 

Objects of Affect: The Domestication of Ubiquity

Paper presentation in: CR13 International Research Conference on Digital Arts in the Series of Consciousness Reframed: Art and Consciousness in the Post-Biological Era, DI-EGY Festival, Cairo, Egypt, 2013. The presentation took place in the GUC (German University of Cairo) in the 27th of March 2013, together with Mike Phillips, iDAT, University of Plymouth. Here follows some parts of the publication / presentation.

"This paper contextualizes digital practices within architectural spaces, and explores the opportunities of experiencing and perceiving domestic environments with the use of media and computing technologies. It suggests methods for the design of reflexive and intimate interiors that provide informational, communicational, affective, emotional, and supportive properties according to embedded sensorial interfaces and processing systems. To properly investigate these concepts, a fundamental criterion is magnified and dissected: dwelling, as an important ingredient in this relationship entails the magical power to merge physical environment with the psyche of inhabitants. For this reason, a number of views are presented and discussed, providing necessary conditions to include matters of affectivity, ubiquity, and layering complexity of interior space. Moreover, specific processes of the possibilities of the digital are mentioned, and examples are presented of the infusion and diffusion of ubiquitous computing technologies within domestic spaces. To briefly conclude, this article is an attempt to discuss the relationship of human-architecture-computer symbiosis and the design process of creative and innovative spaces that affect states of memory, perception, experience, as well as mood and emotion."


Pulse Mode: An Interactive Participatory Audiovisual Installation

Pulse Mode is a commissioned artwork developed for the Code Control Festival, exhibited in Phoenix / Leicester Arts Centre during 22-24 March 2013 (Leicester UK). A fundamental requirement for this award was to use MaxMSP/Jitter for the development of an original piece that exhibits unique properties and suggests new possibilities for computational interaction and control.

Pulse Mode allows participants to interact in real time with the audiovisual mix and trigger events that represent social engagement and mirror aesthetic preferences of the collaborative interaction. The installation space consists of a fragmented 3D screen of 7 irregular rectangular shapes, an info screen (tablet), a Mindwave sensor, and a second tablet for the Mindwave settings. Mobile devices of the participants can be used to select preferences and adjust properties of the system in real time using a web interface and a server that communicates with the processing units. An Ableton Live set includes all the audio tracks that are controlled by Max For Live devices according to the incoming information. The MaxMSP/Jitter visual processing unit controls the composition and the projection mapping, and a Processing sketch reads and controls the Mindwave interface.


An important consideration for the system interaction was to allow every participant to control aspects of the performance. For that reason an HTML5 webpage was created so that everyone with a mobile device and a web browser to be able to select according to their personal taste and preferences various audiovisual properties. Node.js was used as the main server to communicate with the system using OpenSoundControl

 

A QR-code sticker on the installation space immediately directs participants to the web address, where they may find an interface that allows them to select music genre, audio and visual effects, as well as write and send text messages. All the genre selections are received and collectively analyzed giving a ranking of preferences. Just before the current track finishes, the system looks for the genre with the most votes and randomly selects a music track (from that genre) for the next selection. When the new track has been selected, the previous votes are erased, and the voting starts again. This feature provides a method of arranging the macro-montage of the music tracks according to the user preferences. Therefore, the users directly set the style of music in real-time, and following that define the visual imagery that is composed to match the audio quality (speed, noise, texture).


For this installation 300 music tracks have been imported, looped, and wrapped/syncopated in Ableton Live. The tracks have been organized and categorized according to their music style. Each channel in Ableton includes all the tracks from a specific genre. Tempo values vary from 70 BPM to 170 BPM (Beats Per Minute), so when a style changes the main Tempo is set to the average genre speed automatically.


The control of the Live set is defined according to the Max For Live devices that were developed in MaxMSP. The first device collects the genre selections and sets the new track based on the voting system. Moreover, it saves the overall votes (from the start of the session) in to a text file that can be further analyzed using data visualization techniques, providing a useful insight to the participants' taste and preferences. The device also sets the On/Off state of the audio effects (selections are instantly triggered by the users from the web interface). Finally, the device collects track information (genre, name, duration, remaining time), and streams it over the network to the main info screen (tablet), so that users have a visual feedback. A second device in M4L includes the audio effects that are triggered from the users (Shuffler, Modulation Delay, Flanger, Filter, Panning, and Reverb). The effects closely resemble those found in professional DJ mixers and DVD/CD players (i.e. Pioneer CDJ series).


Following that, the M4L devices trigger events in the visual composition that uses a MaxMSP/Jitter patch (loosely based on VJ Mode application developed by SoniconLab) to create a complex visual landscape based on various properties of the music mix. One thousand (1,000) visual clips are included in the system, and each one is selected based on various musical properties, creating a unique composition at every instance. The visual effects are also triggered by the users in real time (with the use of the web interface), and the final result is projected and mapped on the 7 planes in the installation space. The projection mapping is configured within the same max patch. Finally the visual mix uses scrolling text (of various size and speed) based on the messages that are sent from the participants, creating an unexpected real time graffiti.


Pulse Mode raises issues of authority and control, providing tools that open-source the control of an audiovisual performance, allowing the participants to define the environment and gain direct access to its functions. However, authority may return with the use of a Mindwave sensor that is able to by-pass user selections according to the brain activity of the participant (and more accurately the attention level). Thus, the brain activity of the participant competes with the other users and if the activity reaches higher levels, it becomes easier to dictate preferences over the rest of the public.


Therefore, Pulse Mode presents a complex audiovisual system that creates multiple automations and interactions with the use of diverse protocols, programs, and platforms. Nevertheless, it is compatible and fully accessible from a large number of participants allowing to properly define and adjust the overall outcome easily, playfully, and efficiently.

Credits:

Hardware Support: iDAT, Plymouth University
Interior Design: Kallia Platirrahou
Web/Server Design: Florian Bruckner
Graphic Design: Andrew Leeke
Special Thanks: Sean CarrollChris Tyrer

Catalyst Award for Code-Control Festival

Commissioned a Catalyst Award to develop original interactive artwork for Code Control Festival exhibition in Leicester Arts Centre (22nd till 24th March 2013). The installation will consist of realtime audiovisual interactions, mobile and web interfaces, projection mapping, as well as Mindwave sensors used from the audience. More information about the system will be posted here soon.


Paper Publication in DI-EGY Conference, Cairo

Accepted paper publication for the DI-EGY Conference in Cairo, Egypt: International conference on digital arts within the international research conference in the series Consciousness Reframed: Art and Consciousness in the Post-Biological Era, in cooperation with Planetary Collegium, Plymouth University. 

Title of paper: "Objects of Affect: The Domestication of Ubiquity".

Abstract: " [...] Recent technological trends anticipate the embedding of small sensorial interfaces with low-intelligence and easy complexity into every space we use, occupy, and dwell. As technology becomes more ubiquitous and ambient, silent manifestations of computational intelligence appear to surround domestic spaces, and slowly they become important aspects of an inhabitant's lifestyle. [...] "


Sensors R&D

Part of my current research in i-DAT is to examine, prototype, implement, and test under normal daily activities a variety of sensor technologies that are able to identify a number of (physical/virtual) properties such as location, personalization, biological information, movement in space, environmental conditions, and also identify mood and emotion of inhabitants in domestic and interior spaces.

The physical sensor interfaces I am using at the moment include: GPS sensor/shield, RFID tags/reader, Neurosky Mindwave, Kinect, and environmental sensors (temperature, humidity, rain, wind, sunlight, sound, radiation, C0, C02, methane, propane). The combination of these sensors are going to provide necessary information for the system to evaluate, analyze, and personalize aspects of the inhabitant's environment.


For example, I am developing a system for wireless control of interactive lighting in domestic/interior environments that is going to make use of Arduino boards, WiFly and DMX shields. Based on the information that is distributed on the different zones of the space (i.e. environmental conditions - temperature, sunlight, etc) the light composition is going to be altered and adjusted to current information. Aesthetics and practicality are also a matter of importance, and are studied as well to make spaces that adapt to inhabitants' preferences. Following, there are some images from the 3D realtime simulation of the actual development in Unity.


The information that is received from the sensors can be spatially distributed in different media, such as lighting (as the above example), sound, music, visuals, photographs, or even smells. In order to create associations through input and output, I developed an Android application that directly adjusts these connections in real-time. The interface also includes a number of functions for immediate personalization of the media in the interior space. Further experiments are going to make use of touchscreen surfaces (as this one), gesture-based interactions with Kinect/Leap, and semantic analysis from social media activity of the user.



Plinthos Pavilion in the 7th Biennale, Athens.

Plithos Pavilion is a project I created with mabarchitects for the Interior Design Show 2010 in Metamorphosis, Athens. For this project I was responsible to develop a reactive space using sonic landscapes and light ambiences. A real-time composition engine was designed to read visitors' actions and create an atmosphere that influences experience and perception.

This year, this project is exhibited in the 7th Biennale of Young Architects in Benaki Museum. 
Dates: 21 November 2012 - 13 January 2013


FullDome UK Festival, 2012

Participation in lectures, talks, presentations, and viewings in the two-day FullDome Festival in National Space Centre in Leicester, UK. Some of the viewings include: Earthquake (Cal Academy, We are Aliens (NSC Creative), Super Volcanoes (Spitz), Robot Explorers (E&S), Escher's Universe (Parque de las Ciencias & El Exillio), Robin Sip (Mirage 3d), Ancient Skies (Sky-skan), Matrix Optimizer, TBC, Life of Trees (Softmachine), United VJs Real-Time Performance, The Search Engine (DJ Food).



Data Ecologies Symposium

For the Data Ecologies Symposium (i-DAT, Plymouth) I developed and presented a system that uses a number of environmental sensors to create various visualization processes that relate to the natural phenomena. Moreover, the system uses various APIs to extract, share, and collaborate data processing. For example, Twitter messages can control the visualization process, or sensor data feeds can be streamed in social networks.


Arch-OS System Development

Arch-OS (a.k.a. Architectural Operating System) is a framework developed in i-DAT and creates a connection between the sensorial attributes of a building with artistic and technological developments. A large number of sensors is installed in the interior and exterior of the Portland Square building in University of Plymouth. These sensor indications are read from Arch-OS and distributed via RSS and XMAL feeds to any computer or device that is connected to the network (local or remote). For this system I developed an application that displays all the sensor information that is captured from the Arch-OS system. Moreover, the application can process the data in a number of different ways (such as statistical analyses, interpolation, filtering, etc) and distribute them for further processing via the OpenSoundConrol protocol. 


Research Funding from Alexander S. Onassis Public Benefit Foundation

In August 2012 I received a full research grant from the Alexander S. Onassis Public Benefit Foundation for my PhD research (now on my second PhD year) in the Institute of Digital Arts Technology, University of Plymouth. My research topic relates to the uses of computational media within architectural spaces, and more specifically in domestic environments. Main goal of this research is to find ways of using interactive technologies and ambient intelligence to influence mood, experience, and perception. 

Shanghai Biennale 2012

In September 2012 I visited China for the Shanghai Biennale 2012, as technical director for Roy Ascott's exhibition. Main duties I had there included installation of tabletop, projectors, LED screens, sound systems, proper connectivity and equipment functionality, maintenance, support, programming and development.

In the exhibition I had the chance to work with Roy Ascott, one of the pioneers in media art, technoetics, and telematics. After two weeks of intense work we had the chance to present to the public one of the best exhibitions shown in China this year. Many many thanks to Juliette Yuan for her help and support. 







J2W - A Planetary Fairytale

Journey to the West - A Planetary Fairytale is a project I developed for Roy Ascott for the reactivation of "La Plissure du Texte" (1984, Paris) in Shanghai Biennale 2012. 

For this initiative, 50 participants from around the world take part in a real-time seminal work of distributed authorship that unfolds with the use of Skype. Each participant is assigned a mythical character from a range of cultures (such as Greek, English and Chinese), and takes part in a collaborative discussion that relate to one of the most classical novels in Chinese literature (known in Western translations as Monkey).

The text that is sent from the characters is visualized in various forms using custom software that I developed (mainly using Processing). The final visualization is projected in Roy Ascott's installation space in Shanghai Biennale 2012-2013. Visitors are able to virtually navigate in the conversation and explore the narrative that is constructed in real time.



Multi-Participatory Audiovisual Performance

Using a custom-developed interactive tabletop, I used a real-time audiovisual system to create multi-participatory performances. Dozens of tangible objects were created with different shapes and sizes that include a specific fiducial marker. Each physical object is able to control the properties of a virtual instrument. Thus, by using multiple objects, complex compositions are created and controlled in real-time from the participants. This opens new possibilities in experiencing audiovisual performances and defines a new era of democratized interactions.








Multitouch Surface

This is a multitouch table that I designed to support events, seminars, performances, presentations, and many more. It is a rear direct illumination construction that uses infrared light to track the touch points on the surface. It currently uses a variety of frameworks to detect and analyze the blobs (CCV, Reactivision), and I have designed a variety of programming utilities to support the various applications that are needed in Processing, MaxMSP, and Flash AS3.