VioTech · ISMIR 2026 poster (A0 landscape)

Why frame-level & multi-scale?

Playing techniques shape expressive violin performance, yet few detection models exist. VioTech labels three dimensions every 10 ms:

Note-level pipelines segment notes, then classify them; on violin, soft onsets and continuous bowing make segmentation unreliable, and errors cascade.
Frame-level prediction skips segmentation, but techniques are multi-scale: no single window fits all.

VioTech120 dataset

The first manually annotated violin technique dataset of real solo performances, with the first explicit legato (bow-change) labels.

120 étudesaligned audio–MIDI [5]multi-label2 violinists, peer-reviewed

Annotation interface (released)

VioTech annotation interface: waveform, piano roll with legato regions, and per-note technique, string, position and finger selectors

Audio waveform with playback

MIDI piano roll aligned to audio, with legato spans

Per-note wheels: technique, articulation, string, position, finger

Fine-grained MIDI–audio alignment and temporal control; released to extend the dataset beyond techniques, e.g. fingering.

Multi-scale mixture-of-experts

Note-aware transcription cues and a frozen MERT encoder feed 8 temporal experts; a gating network per task routes them.

Temporal experts

Each expert learns its own weighting of the 25 MERT layers, then a dilated depthwise conv sets its window $k_{i} \cdot d_{i} \cdot 10 ms$ , spaced exponentially from 20 ms to 2.56 s.

𝐡_{i}^{'} = \sum_{l = 0}^{24} softmax {(𝐰_{i})}_{l} \cdot 𝐇_{l}

Per-task gating & heads

𝐦^{task} = ReLU (LayerNorm (\sum_{i} α_{i}^{task} 𝐞_{i}))

Tonal (3-way) & articulation (2-way): cross-entropy with focal loss + load balancing.
Legato: frame-wise regression to Gaussian targets at bow changes.

TrainVioTech120 + MOSA-VPT [1] (76 h synthetic, file-level labels) mixed 8 : 2 · 10 s segments · Adam, lr 5×10⁻⁴ Modeltranscription: HPT [4] as in VioPTT [1] · MERT-v1-330M [3], frozen · L = L_trans + L_tech TestRWC [6] violin recordings (cross-dataset, unseen) · Bach10 & URMP for note transcription

Results: cross-dataset evaluation on RWC

90.49%

macro accuracy on RWC

+10.45 pp over VioPTT [1]

+36.5 pp

pizzicato F1 from MoE

57.46 → 94.00 (vs. no MoE)

+30.7 pp

spiccato F1 from MoE

59.78 → 90.50 (vs. no MoE)

+12.1 pp

note F1 on Bach10

50.7 → 62.8 (vs. VioPTT)

Table 1. Technique accuracy (%) on RWC; legato excluded to compare with prior work. Bold best, underline second.

Model	TN	TP	TF	AN	AS	Macro
VioTech (ours)	79.07	89.60	88.84	97.78	97.16	90.49
w/o MERT layer weight	86.11	93.68	65.44	98.74	83.66	85.53
w/o transcription cue	79.83	95.28	83.28	99.57	64.51	84.49
w/o MoE	85.98	40.85	69.62	94.43	61.41	70.46
VioPTT [1]	77.22	88.80	71.89	77.22	85.08	80.04
MERTech [2]	53.36	43.27	95.77	53.36	15.61	52.27

VioTech has the best macro accuracy, led by spiccato (97.16%), while staying competitive on pizzicato and flageolet.

Table 2. Technique F1 (%) on RWC with legato.

Model	TN	TP	TF	AN	AS	LS	LB	Macro
VioTech (ours)	87.40	94.00	30.10	98.69	90.50	97.61	40.02	76.90
w/o MERT l.w.	91.28	85.35	35.29	98.34	86.46	97.89	43.42	76.86
w/o trans. cue	88.09	95.59	30.03	97.58	76.86	97.98	47.17	76.19
w/o MoE	88.38	57.46	30.87	94.72	59.78	98.92	64.09	70.60

Trade-off: bow-change F1 drops vs. no MoE (64.09 → 40.02): sparse, phrase-level events need longer context.

Table 3. Note transcription (%), mir_eval; F1_w/o: without offsets.

Model	Bach10				URMP
Model	P	R	F1	F1_w/o	P	R	F1	F1_w/o
VioPTT [1]	48.7	53.0	50.7	62.5	44.4	44.9	44.6	54.9
VioTech w/o MoE	54.8	58.0	56.1	65.2	39.1	57.5	46.3	59.7
VioTech (ours)	60.9	65.0	62.8	79.1	45.6	60.4	51.9	69.3

How many experts?

MoE helps most on rare, expressive techniques, especially cross-dataset (3-seed means on RWC: pizzicato +32.8 pp, spiccato +15.5 pp), while near-ceiling classes barely move. The best expert count depends on technique and dataset.

What do the experts learn?

Gate routing by technique (VioTech120 test)

Tonal techniques lean on short windows; articulation and legato shift to ≥ 320 ms, about 70% for legato.

Learned MERT layer weights

Tonal classes rely on lower-to-middle layers (local timbre); articulation and legato weight middle layers (broader context).

Frame embeddings on RWC

Pizzicato and spiccato separate cleanly; bow-change frames form their own clusters.

Take-home

Frame-level, segmentation-free violin technique detection at 10 ms; new state of the art with strong cross-dataset generalisation.
Experts specialise by time scale and MERT depth per technique, making the model interpretable; long-context experts for bow changes are next.
VioTech120 + annotation interface released for violin technique and fingering research.