Datasculptures as Virtual Worlds: VR Atelier

student work for individuals - moving data architecture

Student work for iNDiViDUALS – moving data architecture

I proposed to develop a workshop series for the Virtual Reality Atelier, an interdisciplinary minor program that brings together fashion and branding students, and developers and interaction designers to imagine the future of virtual fashion experiences together with industry partners. I coached a project on interactive cinema in VR a while ago and have seen quite a few VR demos in that context. What I often notice is that sometimes there’s little sense of spatial design or architecture in building an interesting virtual environment. The idea arose to use the process of dataphysicalization (the process of turning data into info-experiences) that I’ve been looking at and see if such a design process could spark new concepts for virtual spaces as well.

01_individuals_datasculpture

Student work for iNDiViDUALS

Getting into a datasculpting mindset

I’ve seen many interesting sculptures and objects in the realm of dataphys, and it’s not a huge step to imagine what they could do on a lifesize scale, if you’d blow them up. What is a better way to try that than in the context of computer-generated virtual worlds? The students accepted my proposal for a 3-part workshop called “Datasculptures as Virtual Worlds”. In the three sessions we: 1) analysed what kind of data was relevant to their project and the fashion brands they were working with (iNDiViDUALS, G-star and Alchemist, a sustainable brand). 2) we studied some examples of dataphysicalization to understand how numbers can be turned into tangible designs, and mapped the students’ datasets onto existing examples to gain better understanding of the process of formgiving with data. And 3) I asked them to develop their own ideas and create paper prototypes of a datasculpture and consequently, a mock-up of how the sculpture could work as architectural element or basis for the virtual environment.

mock-up

Infosculpture for Alchemists’ production process of Tencel fibres

Sometimes you win, sometimes you lose

The process was very interesting for me, because I had to stretch myself hard to find ways to get the students to think in a data designerly way. One of the big wins of the workshop was that the students started to think about data at all in these projects. For many of them, the potential of using datasets, let alone live data streams in their VR experiences was completely untapped as they had been focused more on other things such as simulation of the clothing, and ‘fits’ for examples. All of a sudden they started to ideate around the possibility of using user-generated data to let the experience itself change shape. It seems to spark a new way of thinking about the components of the VR experience, and how information could be layered and folded onto and into the experience.

However, the timing of the workshop would have been better had it been earlier in their project. When I got to meet them they’d already progressed quite a bit and weren’t necessarily inclined to add new objectives and designs to their project at that point. Some groups struggled to find the relevance at the stage they were in and didn’t finish the assignment as planned. I let it be for now when they didn’t strictly stick to it, but it makes it hard to assess to what extent the architectural ideas could have worked out. The teams could all see how it would have helped them imagine the space in a lively and more experiential way, had it been planned earlier. It was also clear how a lot of hidden or unused data could enrich the experience itself, adding meaning to the interaction and the space the users enter.

Data.pages

Notes to self

This kind of design exercise is great earlier on, in the beginning of such a project, because addressing data in a sculptural way does open up space for new ideas about spatial design and experience design. We need to spend time with the datasets/datastreams and allow for the possibility to add and invent new ones. Analysing existing examples and re-mapping them with own data is a very powerful way to help participants come to terms with the process of “datasculpting”. Paper prototyping and making mock-ups works well to refine and communicate ideas (as always!), you can start to see how something could really work, how it would feel to walk around in it. The teams that put themselves through the hands-on process of actually realizing a prototype felt a much stronger connection to the potential of their initial idea. I’ve noticed in previous work that a large percentage of design(!) students really struggle to get their hands dirty, there are large amounts of students that are very reluctant to actually build (see previous design coaching projects).

The work is working on an idea, not having one

What a missed opportunity: hands-on building of rough ideas requires the kind of creative problem-solving that really accelerates the creative process. The idea is only the starting point, the work after the idea is the real work, even if you are just doing it in paper form. Having an idea is not the creative work my darlings! Hmm this is an ongoing struggle, I’m not the only one running into this (Jonathan Ives). Stuart Brown relates play and the ability for hands-on building and problem solving as a crucial factor in innovation. It seems clear that this is a skill that is slowly getting lost in current state of (design) education. The groups that flaked out a bit at the end and didn’t get to an actual paper prototype and mock-up (they only made a render or a sketch), had a harder time seeing how it could be something to benefit their project. C’est la vie. I would like to try this workshop again, I think there’s potential but the program needs some refining to really tap into it.

 

 

Dataphysicalization at Politecnico di Milano: some field research

 

frommindtoreality-223

Numbers to physical variables: Mind to Reality workshop week

From May 2nd to May 6th 2016, Citizen Data Lab visited our colleagues at Density Design in Milan, to witness and assist in an experiment in dataphysicalization. 30 Design & Engineering students and 20 students from the MA Communication Design teamed up for a one-week hackathon organized by Monica Bordegoni and Michele Mauri at the makerspace of Politecnico di Milano. The objective was to create a working prototype for a domestic object that allows new tangible interactions with live, realtime data streams.

frommindtoreality-181

What is dataphysicalization?

Dataphysicalization, is data encoded into physical modalities (geometry, material properties or movement) making data tangible, and providing a multisensory, embodied experience of the data represented. Although not a new phenomenon, the physical representation of data has many contemporary appearances, such as pixel sculptures, object augmentation, wearable visualizations, data sculptures and interactive installations. As the structure and organization of our information evolves into more and more complex, linked and layered datasets, interdisciplinary design teams will have to keep finding ways to keep this data intelligible for human faculties. Interesting opportunities are emerging where current developments in digital fabrication, tangible interfaces and shape-changing displays intersect.

frommindtoreality-237 

Beyond screens and LEDs

During this workshop week, students were free in choosing their datastream of choice (the design students all had experience working with digital methods). The makerspace was equipped and staffed to support the physical and electronic prototyping of the objects.

The most challenging aspect of this experiment was to coax the teams into thoroughly exploring how you can express and experience the data in a tangible way, relying as little as possible on visual cues such as displays or lights and instead experimenting with movement, taste, scent or other modalities. This requires a very iterative hands-on ideation and production process and a certain amount of material research. For designers who are used to designing for print or screens this can be quite a leap, but as the week unfolded the workspace got messier, sweatier and more physical indeed.

frommindtoreality-49

Results to build on

The 10 teams demonstrated working prototypes representing changes in live data about bank transactions, wind and tidal information, geo-location, twitter feeds, and activity on Spotify playlists . They experimented with an interesting array of physical variables apart from LED strips and RGB LEDs, such as: changing direction of dials, manipulating the flow of liquids, dropping balls from a pipe, moving origami and tilting surfaces. This video shows a weekly budgeting machine tracking your expenses (one ball for every euro per day) that one team made (see picture at top of post).

In instances where the datastream was rather singular (eg. “if it’s noisy in my garage, send a trigger”) the question arose whether we are dealing with data gathering, or sensing. Should we consider them to be one and the same? Or is it helpful to make a clear distinction? Where does the sensing system of inputs and outputs end and the dataphysicalization begin? Witnessing this process and getting into the literature sparked a lot of other ideas and questions for future work on dataphysicalization in the context of citizen empowerment:

  • How can the practice of dataphysicalization promote data literacy and data exploration?
  • How can we evaluate a dataphysicalization? What criteria do we base such an evaluation on? Persuasiveness, accuracy, readability, memorability, emotional response?
  • Can dataphysicalization foster data engagement in public spaces?
  • Can a pedagogy of ‘constructive visualization’ help democratize the specialist fields of data analysis & visualization?
  • What would a designer’s guide for encoding data into physical variables comprise of?

 

Suggested reading

Huron, Samuel, et al. “Constructive visualization.” Proceedings of the 2014 conference on Designing interactive systems. ACM, 2014.

Jansen, Yvonne, et al. “Opportunities and challenges for data physicalization.” Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2015.

Jansen, Yvonne, and Pierre Dragicevic. Data Physicalization Wiki. 2015. Web. 17 May 2016. <http://dataphys.org/>.

Jansen, Yvonne, and Pierre Dragicevic. “List of Physical Visualizations.” Data Physicalization Wiki. 2015. Web. 17 May 2016. <http://dataphys.org/list>.

Moere, Andrew Vande. “Beyond the tyranny of the pixel: Exploring the physicality of information visualization.” Information Visualisation, 2008. IV’08. 12th International Conference. IEEE, 2008. 

Stusak, Simon. “Exploring the Potential of Physical Visualizations.”Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction. ACM, 2015.

Stusak, Simon, and Ayfer Aslan. “Beyond physical bar charts: an exploration of designing physical visualizations.” CHI’14 Extended Abstracts on Human Factors in Computing Systems. ACM, 2014.

Zhao, Jack, and Andrew Vande Moere. “Embodiment in data sculpture: a model of the physical visualization of information.” Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts. ACM, 2008.

 

dataphysicalization: first thoughts and research

Drip by Tweet - Domestic Data Streamers

Drip by Tweet – Domestic Data Streamers

This blogpost is a summarized (maybe not so elegant) overview of readings and research I’ve been doing on the topic of dataphyiscialization. I will be using elements of this in a variety of activities at the Amsterdam University of Applied Science’s makerspaces where I’m investigating the potential added value of making activities for (design) researchers.

What is dataphysicalisation?

“Data expressed via physical modalities that can be experienced directly, through the eye […] or other human senses” – Bohrman 2015, p.22

“Infoexperiences” – Domestic Data Streamers

Ocularcentrism is dead, long live embodied, full-sensory experience. There are many definitions, but these two highlight the important point of what physical renderings of data do differently for the explorer of that data. By experiencing data, you get a better understanding of data, it can support analytical tasks, and the empathic, emotional impact and engagement with the data can be greater too (Jansen et.al 2015, Jansen, Dragicevic & Fekete 2013, Stusak, Tabard & Butz 2013, Houben et.al. 2016).

For this reason there could be a great potential role for dataphysicalization in debates and citizen empowerment around the themes of data and social change. Data remains rather invisible, ungraspable for non-specialists but has great impact on our lives, our bodies, work, health and education. We can trace back a long history from Mesopotamian clay tokens to shape-changing interactive displays of data: us humans have been doing this for a long time to understand the world, as shown on this historical list.  

Source unknown

Source unknown

To manipulate is to understand

“Not hearing is not as good as hearing, hearing is not as good as seeing, seeing is not as good as knowing, knowing is not as good as acting; true learning continues until it is put into action.”

–   Xunzi, confucian philosopher  –

The embodied cognition thesis argues that we can understand and learn better if we can manipulate the thing we are dealing with in a tangible way. To explore this, dataphysicalization researchers like Jansen, Dragicevic & Fekete (2013) as well as Stusak, Tabard & Butz (2013) and Jansen et.al. (2015) are studying to what extent data physicalization can actually leverage cognitive skills in terms of information processing. The results so far are in favor of 3D, physical data objects when compared to their on-screen or 2D siblings.

Continue Reading →

Dancecoding project

Making code dance/making dance code

During [SZOBA|R|T], a German/Hungarian artist collaboration that took place in an abandoned building in the centre of Budapest, I organised a week of livecoding and dance improvisation jams (see all images here). The idea behind these jams was to bridge the gap in tacit knowledge that is now held by dancers and coders separately. In order to let these knowledges synergize, we should all get on it together and start creating and let new knowledge emerge from the process.


6185787452_93ddb489f2_b

Improvisation: the sweet spot between structure and freedom

How do you start (getting a grip on common conceptual notions between the fields of dance and coding)? What are good design rules for an improvisational framework that both coders and dancers can work with?  How many rules do you need? How much freedom is necessary? How do you negotiate this as you go? The project was part of my residency at Kitchen Budapest in 2010, more info here.

A short article of mine was published on VVVNT, dealing with the expressive and philosophical potential of performance practices combining dance improvisation and live coding.VVVNT is an online journal, forum, & project space for sharing ways of thinking with practical connections across time, scale, system, & discipline: http://vvvnt.com/media/dancing-the-digital

Livecoding as improvisational practice

Live coding is a type of performance where programmers show off their visual coding skills in realtime. Completely live, you see animated graphics come to life in projections, while a group of programmers punches in code to entertain their audience with a (audio)visual spectacle. A group of coders from Kitchen Budapest have been experimenting with this recently, especially with implementing the Kinect in the Fluxus environment. Kinect captures movement data in 3D using infrared cameras, which makes it a perfect combination with dance.

6171748195_c8a2baab50_b

The aim of this project therefore, was to find a way to get these improvisers to work together and experiment freely during 3-hour jams, not entirely dissimilar to the jams that are common practice in a dance style known as contact improvisation. Eight local dancers – many of them with a contact improv or contemporary dance background – worked with us during the week, resulting in a demonstration at the opening of week 3 at [SZOBA|R|T].

6185271343_45535ffcb6_b

From livecoding to dancecoding: where practices meet

This setup put no pressure on choreographing a stage performance, but stayed closer to the traditions of contact improvisation and livecoding: focusing on the live interaction between all the actors: live music, the several coders, a number of projections, and the moving bodies of the dancers. Responses of all collaborators were very positive so we investigated the possibilities of organising more jams in the future. I left Budapest but the group continued to collaborate and did a number of dance coding work and performances together.

6185270375_4ae60783ea_b

C O D E R S
Krisztián Gergely, Gáspár Hajdu, Gábor Papp, Zoltán Csík-Kovacs
D A N C E R S
Roskó Mária, Kornél Biharvári, Szilvia Németh, Maria Paz Ramirez T., Borbála Anna Simányi, Gábor Czap, Dóra Zsámbokréty, Gábor Török.
M U S I C
András Molnar, Andrés Ortega, Tamás Marquetant.
S P E C I A L   T H A N K S   T O
[SZOBA|R|T] organisers and the Goethe Institute, Kitchen Budapest, Emöke Bada, Victor Diaz Barrales.

[insert period joke] …continued

Cross-language Image Analysis on Wikipedia

This research was part of the DMI Summer School project I did in week two: Menstrual Issues Across Language Spacess

What images are shared across language spaces on Wikipedia regarding menstruation?

> what content specific images are/aren’t shared across languages?
> what wikipedia specific images are/aren’t shared across languages?
> what do the images say about the status of the topic ‘menstruation’ on wikipedia across languages?

Collaborators:
Astrid Bigoni | Zuzana Karascakova | Emily Stacey | Sarah McMonagle

with a big thanks to:
Federica Bardelli  (designs)| Giulia de Amicis (designs) | Han-Teng Liao (technical and linguistic advice)

Tools
Wikipedia Cross-Lingual Image Analysis Tool (DMI)

Continue Reading →

[insert period joke]

Google Autocomplete across languages: most queried word combinations with “I’m having my period”.

This research was part of the DMI Summer School project I did in week two: Menstrual Issues Across Language Spacess

Collaborators:

Astrid Bigoni | Zuzana Karascakova | Emily Stacey | Sarah McMonagle

with a big thanks to:

Federica Bardelli  (designs)| Giulia de Amicis (designs) | Han-Teng Liao (technical and linguistic advice)

Research Question

What questions do women ask google regarding menstruation?

  • what are common queries across languages?
  • what topics are unique and only occur in a single language?
  • how are languages linked through queries about menstruation?

Dataset

Google’s database of Google Autocomplete suggestions per language (and country).

Where the predictions come from (source: support.google.com):

As you type, autocomplete predicts and displays queries to choose from. The search queries that you see as part of autocomplete are a reflection of the search activity of all web users and the content of web pages indexed by Google.”

Tools

The DMI’s Google Autocomplete tool https://wiki.digitalmethods.net/Dmi/ToolGoogleAutocomplete

Read more for method, operalization, findings and all the datavisualisations

Infographic: Top Queries Across Languages, By Size

Infographic: Categories With Example Queries

Infographic: All Countries and Categories (excl UK)

Continue Reading →

Issue Distance on Twitter

In June 2013 I attended the Digital Method’s Initiative’s Summer School at the University of Amsterdam. That year’s topic was social media APIs and how their potential for social research. DMI’s goal is to think critically about the affordances and limitations of using the web and social media as object of study as well as an instrument for social research. In week one, I conducted a project on rape discourse on Twitter together with Saskia Kok, Richard Rogers and Carlo de Gaetano & Stefania Guerra from the DensityDesign team (IT). View the result (a datavisualisation) here: scatterplot-1

Issue Distance on Twitter: The 2012 rape case in Delhi, India within the # rape discourse

Research question: How far from the top of Twitter’s #rape hashtag is the Delhi rape case? OR What is the issue distance between the Delhi gang rape of December 16, 2012 and the general  issue of rape on Twitter in the 3 months following the rape itself (time of legislation revisions and the trial of the offenders)?

Dataset:
8.491.020 tweets by 3.589.777 unique users from 2013-01-15 to 2013-06-27 containing the hashtag gangrape, rape

Operationalisation:

Using the TCAT (Twitter Capture and Analysis Toolset)

  • Locate the top of the general rape discourse by finding the top 15 most retweeted tweets with #rape hashtag
  • Locate the specific discourse around the rape case in India by finding the top 15 most retweeted tweets in a subset (only #rape tweets that also contain India AND rape)
  • Show issue distance between the top 15 tweets and the top 15 tweets from the subset in the entire #rape set
  • Visualize issue distance in scatter plot incorporating tweets and key event timeline from the December 2012 gangrape case

Continue reading for the results and findings

Continue Reading →

Fun Facts

koeki

Fun facts is a workshop format for kids where they learn to translate data into imagery. How can you create the most informative, understandable yet fun and compelling image? In stead of using pencils, kids use physical materials and cameras to allow them to work 2D and 3D. I developed this workshop for the Cinekid Mediafestival in Zoetermeer in collaboration with my colleagues Ingrid Rekers and Timothy Kok at CKC Zoetermeer.

Beyond Data: Baltan Labs & Kitchen Budapest

screen-shot-2016-12-02-at-12-03-00

During my residency in Budapest, Baltan Laboratories and Kitchen Budapest brought together their different working methodologies and networks in a series of two workshops in which Dutch and Hungarian artists and designers explore new ways of embodying digital data.

What does it mean to go ‘beyond data’? In the context of this temporary lab we explored mainly how we could go beyond visualizations, and to explore data embodiment. How is touching, feeling or listening to data different from experiencing traditional representations? Part of the methodology was to build on previous projects done by the two labs: to see if wecould use and recylce part of the work to extend and morph it into something else. Some projects from Studio EDHV and Kitchen Budapest were selected to (loosely) build on.

6194786569_e6b238ef73_b

I participated as artist/designer in the first workshop that took place at Kitchen Budapest from 27 September to 1 October 2011. The best concepts from this workshop week were selected to be developed further into physical interfaces and experiences presented in the form of an open lab during Dutch Design Week in Eindhoven from 22-30 October 2011. The results of these workshops will be brought back to Budapest for presentation in November 2011 as part of a series of Dutch-Hungarian cultural events.

6198786793_df730ddf5b_b

Data Granny Crochet Show

The concept I worked on during the week was a data performance called “Data Granny Crochet Show” with Attila (KB), Jeroen and Remco (EDHV). We proposed to turn the exhibition space into a live datamachine (metadata included). We would invite elderly women who enjoy knitting and crocheting to joing for a coffee and some needling. The idea was that we could use data to bring the abanded generation back into the conversation as the craft symbolizes the old ways, the tacit knowledge and the feminine practices of weaving. It can be argued that computing has its roots in these practices as one of the first automated data machines were looms operating with complex patterns and punch cards. The ladies get a task in the beginning: they scan the visitors for certain properties such as age, gender, overheard conversations etc. and modify their stitch accordingly, producing a variation in the fabric that is being created. During the exhibition, the ladies would produce a tangible data representation of the exhibition week.

screen-shot-2016-12-02-at-11-29-23

During the Dutch Design Week exhibition these ideas took shape (in slightly altered form), as ribbon counters, data strings and other tangible forms of live human data processing and visualising. The exhibition slowly came into being throughout the design week and loads of experimental low-tech live dataprocessing techniques were tried out. Unfortunately I couldn’t make it for the entire Dutch Design Week programme.

Experiencing alternative visualisations

One visitor remarked that normally, data visualisations are commissioned with a distinct purpose in mind (to educate, for marketing, urban planning, politics, etc.). In the live lab, we invited people to suggest their own readings and uses of the visualisations and spatial representations we created.

Angela de Weijer remarked:

“We, as participants, have been live data processors, making the invisible visible and tangible in the lab space. The in- ternational crew consisted of artists and designers who do a lot of their work with digital tools. Now, all of a sudden, everyone started producing analogue hands-on embod- iments of data, derived from the visitors, and processed in real-time. As the space filled, people could experience their impact on the lab space (by visiting) and observing the dif- ferent types of data that any one person can leave behind. Did our experiences add any new tools to the toolboxes of both labs? Are there possibilities for deeper research and development?”

OLYMPUS DIGITAL CAMERA

6283052291_e910269746_o

OLYMPUS DIGITAL CAMERA

Early explorations with dataphysicalisation

Attila Bujdosó wrote an interesting reflection in the beyond data book that I would look to years later when thinking again about data ‘physicalisations’ who recognizes that the personal experiences dataphysicalisations enable could be an important step into fostering more engagement around collective issues. “Though physical, tactile or sensual experiences are only viable at an individual level, these kinds of experiences can be shared as stories and memories” (Bujdosó in Beyond Data:113). Also, Trevor Hogan and Eva Hornecker make interesting methodological observations about the assumption that physical data representations can harbour more engagement. They note that because capturing direct human experience is notoriously difficult, we must take a pheonomenological approach to our research, that involves recording, observing and listening to people’s responses when interacting with the data-driven artefacts. Although more clarity is called for,  they found in their research that the language used by participants tends to b more emotive than around traditional visualizations such as bar charts (Beyond Data: 134-135).

References:

Beyond Data (2011) Baltan Laboratories & Kitchen Budapest. Edited by Angela Plohman and Melinda Sipos.

6283049307_f38c2eac7e_o

 

4-day Digital Dance Theater workshop

The last session of Digitaal Danstheater, a research project on using movement tracking technologies and datavisualization in dance productions, was the Projectweek (also called Ludic Week) at Erasmus College, one of the local high schools. Twelve girls ranging from 12-15 years old chose this workshop to fill their week, that aims to introduce kids to more sports oriented or creative fields, to engage them socially in ways that aren’t traditionally offered in school setting and to stimulate developing personal talents. All girls worked incredibly hard these four days, exploring new dance disciplines (classical, hip hop, salsa, modern), developing their own choreographies and experimenting with interactivity: thinking relationships between the body and the computer.

Two groups each made their own interactive dance piece that they performed at the end of the week. It was a fabulous experiment and a good experience that gave a lot of insight into ways to guide group processes, playing attention spans, and scaleability: the most recent Digital Dance Theater workshop was a 4-hour workshop whereas now participants had 4 x 4 = 16 hours in total to get an introduction to virtual theater and produce their own piece. Having more time is a pure luxury for running this workshop and I liked seeing that kids got the opportunity to get thoroughly engaged working towards their own end project over the course of the week.

In collaboration with: Timothy Kok & Nicolet Sudibyo. Zoetermeer (NL), May 2011