Page 141 - Weiss, Jernej, ur./ed. 2025. Glasbena interpretacija: med umetniškim in znanstvenim┊Music Interpretation: Between the Artistic and the Scientific. Koper/Ljubljana: Založba Univerze na Primorskem in Festival Ljubljana. Studia musicologica Labacensia, 8

ai and musical interpretation
And, as Shumailov et al. stress, once the AI starts training itself on
AI-generated content model collapse becomes ultimately inevitable.
Carl Franzen presents another instructive animal-based example of
model collapse:

a machine learning model is trained on a dataset with pictures of 100
cats — 10 of them with blue fur, and 90 with yellow. The model learns
that yellow cats are more prevalent, but also represents blue cats as more
yellowish than they really are, returning some green-cat results when
asked to produce new data. Over time, the original trait of blue fur
erodes through successive training cycles, turning from blue to green-
ish, and ultimately yellow. This progressive distortion and eventual loss
of minority data characteristics is model collapse. 28
The basic problem in the examples of both dogs and cats is that rare
yet existing cases are more and more neglected and eventually weeded out
completely in the process of generating new output. Hence Franzen insists
that

it’s important to ensure fair representation of minority groups in data-
sets, in terms of both quantity and accurate portrayal of distinctive fea-
tures. The task is challenging due to models’ difficulty learning from
rare events. 29
So far cases of LLMs operating entirely on the basis of input already
generated by AI only exist in artificially constructed situations like the
ones described in the articles referenced above. Yet the percentages of ful-
ly AI-generated or at least partly AI-influenced data on the web is continu-
ously increasing.
I could not find examples of model collapse exemplified through mu-
sic, yet given that generative AI creates texts, pictures and music based on
the same principles, it is to be expected that the concept applies to AI-gen-
erated music in the same way. In the real world, within the data sets that
are training the AI, fully AI-generated content may not be prevalent yet,
but the percentage of partly AI-generated content it is certainly higher than
many of us may guess. Constantino quotes a study by Amazon Web Servic-
es according to which 57 % of all content on the internet that is available in
three or more languages has very likely been translated by AI, also referring
28 Franzen, “The AI Feedback Loop.”
29 Ibid.

141