Metadata for audiovisual heritage: Semiotic considerations

Indrek Ibrus, PhDAssoc. Prof. @ TLU/BFM

Advisor @ Ministry of Culture

Digitisation campaigns

• 2K and 4K resolutions equal the quality of analogue film

• New initiatives are emerging throughout Europe to digitise film and TV heritage collections.

• Costly: to digitise all of Europe’s audiovisual heritage will cost at least 5 billion euros

• Cost of digitising Estonian film heritage: 6,5 million euros

Benchmarking initiatives• France: "Grand Emprunt": €125 M to digitise 2500 films +

€400 M to digitise 7000 films

• UK: Film for ever €3.8 M per year to digitise 10,000 films

• BBC is working towards digitising and enabling free access to all of its audiovisual content by 2022.

• British Film Institute together with multiple British film archives is working towards integrating all British film heritage metadata databases into one searchable system.

• Dutch project “Images for the Future” focuses on experiments with metadata creation and with finding innovative uses for both the heritage as well as for its metadata.

Goal: repurposing and innovation

• EU policy emphasis: Not only preservation, but reuse and repurposing – innovation in terms of remixed content as well as new services, applications, etc.

• Memory institutions – limited budgets, understaffed – no capabilites for innovation

• Private institutions have no incentives.

• Governments need to lead the way for innovation.

Standards fragmentation

• AV content metadata standards are in the typical early era of fragmentation.

• But interoperability is needed, therefore also further standardisation.

• Yet, at which cost?

Analogue and digital archives

• Hartley (2012) distinguishes between modern archives, postmodern archives and network archives.

• Network archives = probability archives = uncertainty

• Baron (2014) “Experience of reception”

• Compared to written documents audiovisual documentary content offer more of interpretative freedom.

• Due to their indexical nature their form and messages are less designable – more democratic and less controllable.

Probability and curation

• Answer to ‘probability’ is ‘metadating’ (Ernst)

• Question: appropriate balances?

Different interests for functions

• Librarians concerned about interoperability and having standardized access to descriptors

• Producers and rights owners interested in IPR management and in content ratings and access controls

• W3C develops XML and related standards for metadata formatting and semantics in order to develop ways for web-based metadata creation (and use)

• Educators have specific needs within particular domains, e.g., tagging video by curriculum or grade level

• VOD providers – developing proprietary recommendation systems aimed at motivating further consumption and service loyalty

• New public online-archives - Europeana, EUScreen – service operability

Standards• MARC – developed in 1960s - path dependent on the functions of 20th

century memory institutions

• MPEG-7 - technologically one of the most comprehensive metadata technologies, born out of the industry co-operation processes

• Netflix ‘genres’ – proprietary solutions

• PBCore (Public Broadcasting Metadata Dictionary) – created by The Corporation for Public Broadcasting (CPB) in US – standard now being used as an asset management system for all kinds of audiovisual industries

• EBU Core – designed by EBU as part of the development of its “Linked Open Data” program to service EUScreen. Since it was designed to work with audiovisual ontologies for the seminatic web much of it was eventually incorporated into W3C Media Annotation ontology (W3C MAWG)

• Variety of in-house standards and description systems (EFIS).

While many of the standards have emerged out of the interests of specific kinds of institutions there has been more recently some convergence of these interests and relatedly also more of cooperation and dialogues.

Standards evolution

• Standards as forms of modeling, of auto-communication

• Auto-communication in complex environment

• Results in dialogues

• Dialogues of standardisation

• Research on standards evoluton as a semiotic study

Ideologies of standards and archives

• If the evolution of standards is conditioned by the power struggles between institutions this suggests that the standards also express these power relationships and contribute to their sustenance - i.e. they function ideologically.

• Recognition, that ‘archive is the first law that can be said’ has been part of cultural critique since Foucault’s “Archaeology of Knowledge”

• Derrida demonstrated how archives are structured according to the logics of power that determine which objects are preserved, stored, and revered and which are excluded.

• Contemporary media archaeologists such as Parikka and Ernst have investigated ways how ‘power still resides in the archive’ - i.e. how it is embedded in architectures of software and in the political economy of social media platforms.

Ideologies of standards and archives

• The most immediately ideological feature is how the protocols select, connect or delimit cultural texts and sub-domains

• Ernst: although indexes or tags of digital archives are primarily search oriented, unlike traditional archive repertoirs, they are not passive, but constitute a logistical document containing links to the pertinent data records – a process through which the archive starts functioning self-referentially and in the process defines a penumbra of sub-domains within this whole/culture.

Ideologies of standards and archives

• Cultural semiotics: how meta-languages for cultural domains emerge and how they affect the evolutionary dynamics of culture?

• Understood to emerge as resulting from a mesh of dialogic and auto-communicate communications of existing institutions and social-sub-systems.

• But the more the dominant systems produce descriptions of themselves or their environment the more these descriptions also fix culture and codify its inner structures. Until such descriptions get standardised in ways that they become too inflexible to respond to ongoing cultural dynamics and innovations.

• At such instances new and possibly more pertinent metalanguages for culture may emerge at the dynamic peripheries of culture.

Example: genre-systems• Different fields of arts and entertainment are

having different genre systems, some of them are stricter and more standardised (videogames), some constantly evolving (films).

• Neale: film genres are industry production, but also evolve via copying.

• Vidogame genres: evolved during the era of network based sale

• Result of metadating?

• Netflix genres: fixing for global audience – merger of industry and network dynamics?

Metadata as cultural auto-communication

Need to study digital metalanguages as cultural construction aparatuses.

• A new form of auto-communication – at an aggregate level

• How fixed or open the system is?

• How dialogic the system is?

‘Folksonomies’• Folksonomies reflect a communial view of the

attributes associated to items, essentially supplying a bottom-up categorisation of resources.

• Folksonomies stand in contrast to ‘ontologies’ which by definition are expert defined explicit specifications of the conceptualisation of a domain.

• Folksonomies are in effect ‘lightweight’ ontologies characterised by not having any preset vocabuliaries for annotations.

Folksonomies• McKee (2011): YouTube’s metadata is more intuitive than that

of Australia’s National Film and Sound Archive.

• Gligorov et al (2011): In case of Dutch video labelling game Waisda? large part of the added tags were Dutch words not used by professionals (later used for further developing the indexing in Netherlands Institute of Sound and Vision.

• Although studies indicate that folksonomies may not create perfect metadata, still the general conclusion is that even bad metadata is useful.

• When designing tagging (shot level) services for highly motivated fans (of Twin Peaks series) the results may be valuable for providing new capabilities for analysing the structures of that content (Geisler 2010).

• Folksonomies may service the new forms of digital humanities.

Content analysis algorithms• Manual metadata creation is very laborious and expensive.

• Folksonomies are difficult to organise, may take considerable amount of time to carry out and due to ‘network effects’ it may be difficult for smaller archives and memory institutions to make happen.

• Therefore, automated algorithms become native components of digital archives and will contribute too all metadata creation processes.

• PrestoSpace Project: There is an increasing number of techniques.

• With the help of automated analysis visual images a lot of questions on patterns of visualisaton of social phenomena could be investigated (for instance representation of women in films) – method for digital humanities.

• Question: ideologies of such algorithms?

• Ideologies of all schemes of metadata creation?

• Tension between curated and ‘free’ cultural memory

• How free is the free use?

Political economy of user-created metadata

• What makes the network-based archives especially dynamic is that into the (meta)data-sets is also included information about all kinds of behavioural aspects of users.

• Through their queries and usage, users create further archive elements to be digitised and stored.

• Object-oriented archive takes shape cumulatively, entailing a shift from read-only paradigms to a ‘generative’ form of archival reading.

• Classical file-oriented archive practices yield to the use-oriented (“to be completed”) “dynarchive” (Ernst)

• Such ‘generation’ often means user tracking without their explicit and full consent.

• Highly probable that this information on the actions and preferences of users may subsequently not only be used for perfecting the public services, but may be also be commodified and traded in media marketplaces.

Economic curation?• Question: How the political economy of ‘heritage

metadata markets’ affects the operations and functionalities of digital archives and what are the effects to culture structuration.

• If the aim is innovation and (commercial) repurposing then what are its effects to cultural auto-communication?

• The effects of memory marketing to cultural memory?

• How free is free use?

• Digital metadata – a new field for cultural policy

Thank you!indrek.ibrus@tlu.ee