nervousdata

Making text-collages with Hydra and p5js

A documentation on different ways to make text-collages with Hydra (live-coding networked visuals) and p5js (JavaScript library for creative coding). It introduces to basic functions of Hydra and shows how to use a p5js canvas to display text in Hydra.

Text in this setting is basically handled as image. The focus is ‘technically’ on the visual qualities of text but the results often provoke semantic and phonetic interpretations.

Overview Text on the ScreenModulate and ScrollMask and ScrollFeedback

Text on the Screen

So far, I tried two ways to get text on screen in Hydra. One is to load an open window (a document or the Internet browser) as an input source. Another is to create a p5 canvas inside the sketch and use it as a source.

When going for the first option with open windows I find myself switching focus between the sketch in Hydra (editing code) and navigating what is displayed in the open window; surfing a website, scrolling, zooming in and out and highlighting elements with the cursor for example.

Screenshot of a desktop with two windows. In both windows text is displayed. On the right “the form” is highlighted. On the left the information from the right window is visually transformed, re-arranged. On top is another layer showing computer code.

Screenshot of a computer desktop with two open windows. Left: Hydra (.js file) in ATOM editor. Right: Browser

Using a p5 canvas allows to define text size, font, line height, colors and position — and to change these parameters while the Hydra sketch is running. The text() function in p5 simply displays text.

I would store the text size in a variable; a value relative to the height of the window. Resizing the window of the editor or working on different devices often messed up my sketches when I set a fixed size like 20 pt.

height = window.innerHeight*0.06

Initialize a p5 canvas to the variable p5.

p5 = new P5();

Draw the canvas with text as a string.

p5.draw = () => {
  p5.background(210,220,230);
  p5.fill(0,0,0);
  p5.textSize(height);
  p5.textLeading(height * 1.5);
  p5.textFont("serif");
  p5.text("He finds upon calculation, that at the rate at which he can dispose …"
    ,p5.windowWidth*0.04, 30, p5.windowWidth, p5.windowHeight);
}

By default, the p5 canvas is rendered on top of the Hydra buffers. I want to hide the canvas but then use it as an input in Hydra.

p5.hide();
s0.init({ src: p5.canvas });

Instead of typing text directly into the code of the Hydra sketch, loading text from a file (.txt for example) is also possible. With the loadStrings() function in p5 the contents of a file become an array of strings. Each line is a string.

Store the array of strings as a variable and call the variable in the text() function.

abc = p5.loadStrings("file:///home/ji/abc.txt");

p5.text(abc, p5.windowWidth*0.04, 30, p5.windowWidth, p5.windowHeight);

Modulate and Scroll

In this example I use an oscillator to transform the text canvas. To be able to see and understand the transformations well, I made it grayscale or black/white. Hydra allows really smooth visual dynamics but I prefer hard crisp cuts for this purpose. That’s why my sketches rely a lot on the pixelate() and posterize() function.

The oscillator pushes the text on the y-axis while it is moving on the x-axis on the screen. The intensity of the transformation is defined by the brightness. Black means no effect and white full effect.

Store the text source p5.canvas in a variable "txt"

txt = () => src(s0)

Store the osc in a variable "o"

o = () => osc(6,0.06,0).posterize(8).pixelate(10,1)

The pixelate() basically means that the movement takes place in a grid, is rendered as a grid. In this example a grid of 10x1.

A gradient from light gray (left) to black, in four steps.

Call the custom function for the p5 text canvas and modulate it with the oscillator stored as "o".

txt().modulateScrollY(o(),0.36).out(o0);

Modulate the modulator

The simple posterized oscillator produces the effect of separate columns. To get a more wavy and spiky pattern, add a modulation to the oscillator, for example on the basis of a rotated version of the previous one.

o().modulateScrollX(o().rotate(Math.PI*0.5),0.1)

A wobbly grayscale gradient with pixelated curves.

txt().modulateScrollY(o().modulateScrollX(o().rotate(Math.PI*0.5),0.1),0.36) .out(o0);

Modulate the modulated

Add another modulation to the current canvas (not the original text) for movement on the x-axis. Again a rotated version of the oscillator previously defined.

txt()
  .modulateScrollY(o().modulateScrollX(o().rotate(Math.PI*0.5),0.1),0.36)
  .modulateScrollX(o().rotate(Math.PI*0.125).scale(0.62),0.26)
.out(o0);

Open sketch in Hydra web editor

Mask and Scroll

Another method I explored is the use of layers to display the canvas two times, each layer with the same text but with slightly different movements or transformations. In this example the oscillator acts as a stencil or frame for the text image. It is altered into black and white stripes for this purpose.

osc(20,0,0).posterize(8).thresh(0.5)

Vertical black and white stripes.

Display the text only in the white sections of the previously defined oscillator with using the osc() in the mask() function. The black stripes stay black. Make txt() move on the x-axis with setting scrollX().

txt().scrollX(0.1,0.005).mask(osc(22,0.01,0).posterize(8).thresh(0.5))

Add another layer of txt() with the colors of the oscillator inverted (black becomes white and vice versa) and set a different speed and direction for scrolling on the x-axis.

txt().scrollX(0.1,-0.011).mask(osc(22,0.01,0).posterize(8).thresh(0.5))
.add(txt().scrollX(0.1,0.01)
  .mask(osc(22,0.01,0).posterize(8).thresh(0.5).invert(1)))
.out(o0);

Feedback

To add some fuzz, distortion and more tiny repetitions, I’d like to have a layer with feedback. That means the current frame with all modulations is displayed. The current output becomes an input. By adding src(o0) as a whole layer to the chain of functions the frames will simply freeze. But by applying a mask (like in the example above) it is possible to define sections for the feedback. When there is movement (of the mask and everything else) an effect of writing (or drawing) occurs.

txt = () => src(s0)
o = () => osc(22,0.02,0).posterize(8).thresh(0.5)

Create a mask on the basis of a previously set up oscillator.

.mask(o().diff(o().rotate(Math.PI*0.25)).scale(0.75)

A pattern of black and white rhombs.

Use this mask for the layer with the output as source.

txt().scrollX(0.1,-0.011).mask(o())
  .add(txt().scrollX(0.1,0.01).mask(o().invert(1)))
.layer(src(o0)
      .mask(o().diff(o().rotate(Math.PI*0.25)).scale(0.75),0.01)
  ).out(o0);

Open sketch in Hydra web editor


09/2022