Introduction

Soundsketcher is an interactive web tool that translates audio into expressive, data-driven visual sketches. It maps perceptual audio features — such as brightness, roughness, and pitch — into abstract graphic forms, helping you "see" the texture, structure, and evolution of sound.

Whether you're a musician, sound designer, researcher, or student, Soundsketcher bridges the acoustic and visual worlds through a perceptually grounded, spectromorphologically inspired approach. Explore sound, generate graphic scores, compare recordings, or experiment with sonification — all in your browser.

Getting Started

Uploading Audio

Drag and drop an audio file onto the upload area, or click the upload button to browse. Supported formats: .wav, .mp3, .ogg, .flac, and other common audio formats. For best quality, uncompressed .wav is recommended.

When you upload a file, Soundsketcher checks if it has been analyzed before:

• Previously analyzed: Choose between cached data (fast) or reanalyze (needed if window or feature settings changed).
• New file: Analysis begins automatically — feature extraction, segmentation, and pitch estimation run on the server.

Generating a Sketch

Once analysis is complete, press the Sketch button (or press A) to generate the visualization. The sketch is drawn on the canvas using the current mapping settings.

Playing the Audio

Press the Play button in the toolbar. The button turns green while playing, and a progress line sweeps across the canvas in sync with the audio. Press Play again, or press Space, to pause. Use the seek bar below the canvas to jump to any point in time.

Library

The Library button (top-left toolbar) opens a panel showing all previously uploaded and analyzed files. Click any entry to reload that file instantly — no re-upload needed.

The Interface

Top Toolbar

• Library — Browse and reload previously analyzed files.
• Mixer — Open the Mixer panel for per-feature volume, pan, solo, and mute controls.
• Sonification — Quick-open the granular synthesis engine controls.
• Help — Open this manual.

Bottom Toolbar

• Canvas — Display options: dark mode, progress line, axis ticks, CLAP contour, export buttons.
• Data — Analysis parameters: window length, overlap, normalization, filtering, recalculate.
• Mapping — Feature-to-visual-dimension mapping and range sliders.
• Sonification — Granular engine parameters, effects chain, pitch grid.
• Sketch mode buttons — Switch between Lines, Polygons, Trajectories, Blobs, and Gestures modes.
• Eraser — Toggle erase mode to mute or unmute individual sketch strokes.

Panels

All panels (Canvas, Data, Mapping, Sonification, Mixer) are draggable — grab the panel header to reposition anywhere on screen. Each panel has a × close button. Reopen from the toolbar at any time.

Guided Tour

New to Soundsketcher? The built-in guided tour steps through main interface elements one by one. Look for the tour trigger in the top toolbar.

Audio Features

Soundsketcher extracts the following perceptual audio features. Each can be assigned to any visual dimension in the Mapping panel.

Friendly Name Toggle

The Mapping panel header includes a toggle to switch feature labels between friendly names (e.g., “Brightness”) and technical identifiers (e.g., “spectral_centroid”). Use whichever suits your workflow.

Mapping & Visualization

Mapping Panel

Open with the Mapping button, or press F. Each row assigns an audio feature to one visual dimension of the current sketch mode:

• Feature selector — Choose which audio feature drives this dimension.
• Invert — Flip the mapping direction (high values map to bottom instead of top).
• Range sliders — Set min and max of the feature range. Drag inward to focus on a sub-range and exclude outliers.

Toolbar shortcuts: Z loads Preset A, X Preset B, C Preset C. Press D to reset all mappings to defaults. Press S to randomize.

Visualization Modes

• Lines — Time-aligned vertical lines; each represents one analysis frame. Height, angle, thickness, and color are all independently mappable.
• Polygons — Like Lines but with filled polygons; stronger shape identity and richer visual mass per frame.
• Trajectories — Plots feature values as continuous paths over time; emphasizes contour and evolution.
• Blobs — Region-level shapes: width = duration, height = bandwidth, Y = spectral centroid, opacity = loudness. Good for macro structure.
• Gestures — Expressive inner sub-region shapes highlighting onsets and timbral shifts within sound objects.

Interpreting the Sketch

The horizontal axis always represents time (left = earlier, right = later). The vertical and other visual properties follow your feature mappings. Noisy, inharmonic passages tend to look irregular; tonal, pitched passages appear smooth and centered.

CLAP Contour

Enable CLAP Contour in Canvas settings to overlay a semantic contour derived from CLAP audio embeddings. This adds a higher-level shape layer alongside the frame-by-frame sketch.

Objectifier Mode

A specialized mode that renders sound objects as symbolic, object-like graphic forms. Enable in Canvas settings.

Sonification & Mixer

Granular Synthesis Engine

Soundsketcher includes a real-time granular synthesis engine that re-sonifies the analyzed audio following the visual mappings. Open the Sonification panel to configure it.

• ARM — Activates the granular engine. Must be armed before playback generates granular sound.
• Grain Size — Duration of each audio grain (ms). Smaller = more textural; larger = more recognizable source material.
• Grain Density — How many grains fire per second. Higher = denser, more continuous texture.
• Position Scatter — Randomness in which part of the source each grain reads from.
• Pitch Spread — Random pitch deviation per grain for organic variation.

Effects Chain

The engine has a built-in effects chain — reverb, delay, and other real-time processors. Adjust wet/dry levels in the Sonification panel to blend effects with the dry granular output.

A limiter at the very end of the chain keeps loud passages from clipping and leaves quieter sound untouched. It is always last; switch it off in the effects list only for level measurements.

Pitch Grid

Enable pitch quantization to snap granular playback to a musical scale. Configure root note and mode in the Sonification panel for harmonic, scale-aware re-sonification.

Scan Direction

Control how the granular engine reads the source material: forward, backward, random, or locked to the canvas playhead position.

Mixer

Open with the Mixer button (top toolbar). The Mixer provides a channel strip for each active feature layer:

• VU Meter — Real-time signal level for each channel.
• Volume fader — Per-channel output level.
• Pan — Stereo position in the mix.
• Solo (S) — Isolate this channel; all others are silenced.
• Mute (M) — Silence this channel without removing its visual layer.
• Visibility eye — Show or hide the corresponding layer on the canvas.

The Mixer panel is draggable — grab its header to reposition it anywhere on screen.

Drawing & Sound Shapes

Besides the analysis, the sketch can hold your own tracks: brush drawings, sound shapes and imported images. Each has its own row in the mixer (visibility, mute, solo, colour, delete), and the sonificators play it like the analysis. Your edits are kept in the browser for each set of audio files and come back when you open them again.

Brush

Press the brush button (B) and draw on the sketch. The small arrow opens its settings: size, lightness and texture. They use the ranges of the Mapping panel, so drawn strokes look and sound like analysed ones. Pick from sketch copies the settings of an existing stroke; with a pen, pressure changes the size. Save drawing and Load drawing keep all your tracks as a file.

Sound Shapes

The sound shapes tool (S) places shapes built on Denis Smalley's spectromorphology. Open the palette with the small arrow and choose:

• Spectrum — note (a clear pitch: a thin line), node (a band of frequencies), noise (a wide, rough band of leaning lines).
• Morphology — how it starts and ends: impulses (short hits), attack-decay, swell, continuant.
• Motion — plane, ascent, descent, parabola, oscillation, dilation, contraction, divergence, convergence.
• Presets (Penderecki cluster, Xenakis glissandi, pizzicato, noise wedge) and parameters: density, intensity, irregularity, oscillation rate.

Placing and Editing Shapes

Drag a box on the sketch: its width is the time span, its height the pitch region; a click places a two-second shape. Edges snap to the time grid (or the tempo grid) and to notes (or the Pitch Grid scale); hold Alt to place freely.

A placed shape stays selected: drag it to move it, drag its handles to resize it, or change any choice in the palette. Delete removes it, Ctrl/⌘+D duplicates it, the arrow keys nudge it. Every change can be undone.

Each placed or changed shape is played on its own; P plays the selected one. In the palette, Listen with shows which engine plays it: the armed one, or one you fix.

Importing an Image

The image button imports any picture. It is placed at the cursor, over the full height of the sketch and as wide as its proportions require (cropped if it is longer than the sketch). Read it as Shapes (dark or light areas) or as Lines (edges); the threshold and the lines per column set how much of it becomes strokes. Add as track turns it into strokes and can keep the picture as a faint guide; Guide only keeps just the picture, to draw over.

Exporting Sound and Score

Canvas → Export WAV renders the sonification faster than real time, with the engine you choose, the mixer levels and the effects. It can include the original audio at the mixer levels, bring both to the same loudness, and also save them as separate, aligned files for a DAW. Export SVG saves the sketch with all its tracks as a vector image.

Settings & Export

Canvas Settings

Open with the Canvas button in the bottom toolbar.

• Dark Mode — Switch the canvas and UI to a dark theme.
• Scrollable Canvas — Enable horizontal scrolling for pieces that exceed the viewport width.
• Progress Line — A moving vertical line synchronized with playback. The adjacent color picker customizes its color.
• Hz / Time Axis Ticks — Overlay frequency (Hz) or time (s) grid lines on the canvas.
• Tooltips — Show feature values when hovering over sketch elements.
• Auto-Redraw — Automatically re-sketch whenever mapping settings change.
• CLAP Contour — Overlay the semantic CLAP embedding contour over the sketch.
• Objectifier Mode — Enable object-based symbolic rendering.
• Noise Threshold (ZCR) — Filter out high zero-crossing-rate frames to suppress noise from the sketch.

Data Settings

Open with the Data button in the bottom toolbar.

• Window Length — Analysis window size in samples. Larger = more frequency resolution, less time resolution.
• Overlap — Window overlap percentage. Higher = smoother feature curves.
• Normalize Audio — Normalize the signal before feature extraction.
• Apply Filtering — Pre-filter the audio before analysis.
• Clipping (Hard / Soft) — Clip outlier feature values to prevent extreme mappings.
• Configure Features — Per-feature clipping and scaling controls.
• Recalculate — Re-run analysis with the current data settings.

Export

Export buttons are in the Canvas panel.

• Export PNG — Save the current canvas as a PNG image.
• Export JSON — Save the underlying feature data as JSON for external analysis.
• Export SVG — Save the sketch (including drawings, shapes and images) as a vector image.
• Export WAV (sonification) — Render the sonification, optionally with the original audio (see Drawing & Sound Shapes).
• Print — Open a print dialog for the current sketch view.

Eraser Tool

Press the Eraser button in the toolbar to enter Erase Mode. Click any stroke on the canvas to mute or unmute it. Muted strokes are hidden and excluded from sonification. Press Restore All to un-mute every erased stroke at once. Exit erase mode by pressing the Eraser button again.

Keyboard Shortcuts

Troubleshooting

File won't upload

• Ensure the format is supported: .wav, .mp3, .ogg, .flac.
• Check the file is not corrupted or zero-length.
• Try a different browser or disable extensions that may block file uploads.

Sketch looks blank or odd

• Check the Mapping panel — a dimension mapped to “None” produces no strokes.
• Adjust the range sliders; very narrow ranges can suppress all output.
• Press D to reset mappings to defaults, then re-sketch with A.

No sound from the granular engine

• Make sure the engine is armed — press the ARM button in the Sonification panel.
• The first Play press may need a user gesture to unlock Web Audio in the browser.
• Confirm the Mixer channels are not all muted.

Sketch renders slowly

• Long files with small window size generate many frames. Increase window size or reduce overlap in Data settings.
• Close other browser tabs to free GPU and CPU resources.

Browser Compatibility

Best experience on the latest versions of Chrome or Edge. Firefox is supported. Safari may have limited Web Audio API performance for the granular engine.

About

Soundsketcher is a research and creative tool developed at the Aristotle University of Thessaloniki, with support from collaborating institutions.

Development

• Konstantinos Velenis — Lead Developer, Aristotle University of Thessaloniki

Research Team

• Emilios Cambouropoulos — Aristotle University of Thessaloniki
• Danae Stefanou — Aristotle University of Thessaloniki
• Maximos Kaliakatsos-Papakostas — Hellenic Mediterranean University
• Dimitris Maronidis — Aristotle University of Thessaloniki
• Konstantinos Giannos — Aristotle University of Thessaloniki
• Asterios Zacharakis — Aristotle University of Thessaloniki
• Savvas Kazazis — Aristotle University of Thessaloniki
• Alexandra Karamoutsiou — Aristotle University of Thessaloniki
• Konstantinos Velenis — Aristotle University of Thessaloniki
• Eva Matsigkou — Aristotle University of Thessaloniki
• Vicky Zioga — PHENO
• Nikos Kostopoulos — PHENO

International Advisors

• George Athanasopoulos — Humboldt University, Berlin
• Richard Barrett — Institute of Sonology, The Hague / Leiden University
• Emmanouil Benetos — Queen Mary University of London

Contact

Email: kvelenis@gmail.com

Website: soundsketcher.web.auth.gr