“We have to take into account the ways digital humanities more broadly have taken up computational techniques and then consider the specificity of visual art objects and their particular requirements and points of resistance” [Drucker 2013]. Drucker remarks on a process very common in the digital humanities: methods are developed first or existing approaches are simply taken and specificities and requirements of digital data are considered afterwards. However, this approach as stated does not consider characteristics of digital data during the phase of development and therefore restricts the potential of digital data and methods. Although we have transferred analog image processing into a digital space, we restrict ourselves to knowledge which could have been obtained with analog data and methods. Instead, we must identify special characteristics and requirements of digital data first and create corresponding methods afterwards. Examples of research projects in the last section of this article highlight the importance and validity of this thought process. But what are the specifics of digital data in comparison to analog data which should be considered and explicitly exploited by new methods?

Digitization through, for example scanning, describes the translation of an analog, continuous into a digital, discrete signal. Digital images are thus available as numeric representations. This means that certain aspects in images such as composition, spatial relations between objects and figures or color can be evaluated statistically and expressed in descriptive statistics. Through a statistical analysis we are able to summarize the content of one or multiple images in representative statistics. These provide factual and objective descriptions of digital artworks, which are impossible to create for analog images and with analog methods. By looking at these image statistics, art historians are able to study the popularity of a composition, the degree to which artists relied on compositional models or when a color originated and how frequent it was adapted by artists.

In addition digital images can be easily duplicated because they are stored as numerical representations. These digital copies are reproductions of existing artworks and not computer-generated images. In contrast to the analog image, the digital reproduction is transportable and easy to distribute. Digital image copies can be shared with colleagues or used for multiple projects, thus supporting collaborations and the founding of a global network of scholars. Digitization then supports the transformation of art history from a locally and nationally determined discipline, where each country favors different methods and topics, to an international and dynamic field. While the reproduction of images offers great potential for society and research, it also holds disadvantages: reproduction quality varies for images; color, illumination, sharpness or contrast are often misrepresented in digital images and falsify the appearance of the original artwork. These visual alterations must be considered in an interpretation or when developing computational methods. An erroneous color reproduction might establish a visual similarity of actually dissimilar images and a blurry image conceals details otherwise visible. Duplications of artworks also raise questions of originality, uniqueness and authenticity. German philosopher and art critic Walter Benjamin (1892-1940) famously addresses these issues in his essay “The Artwork in the Age of Mechanical Reproduction” first published in German in 1935:

Even the most perfect reproduction of a work of art is lacking in one element: its presence in time and space, its unique existence at the place where it happens to be. This unique existence of the work of art determined the history to which it was subject throughout the time of its existence. This includes the changes which it may have suffered in physical condition over the years as well as the various changes in its ownership. The traces of the first can be revealed only by chemical or physical analyzes which it is impossible to perform on a reproduction...  [Benjamin 1969]

Therefore the digital artwork challenges the uniqueness of the original artwork and detaches it from any place or time restriction. According to Benjamin, the originality of an artwork and its existence at a particular time and place is the prerequisite of authenticity [Benjamin 1969]; for the viewer and in particular the scholar this attachment has consequences. The function and role as well as the physical effect of the artwork can never be fully comprehended through digital images. Instead, we are only able to guess and hypothesize, thus providing a first insight but never a full picture of the artwork in its original context. To do so we have to see it in person, if possible, in its actual environment: the museum. While the removal of authenticity, uniqueness and original context are shortcomings of digitization, it allows to preserve art and cultural heritage in digital form. This is especially relevant for works which are exposed to weather conditions or pollution or which are made out of fragile material, examples include weavings, fresco paintings or stained glass windows. Street art is a current art form which faces — but also consciously plays with — these threats. As Benjamin mentions in his essay, reproduction techniques further blur physical damages such as cracks on the original artwork [Benjamin 1969]. The damaged artwork is thus restored in digital form. For art historians this aspect is ambiguous: on the one hand, it enables to study the unharmed artwork and to reason about its original appearance, on the other hand it gives a false impression of the artwork’s current condition.

The representation of digital data as values makes it modifiable and editable through computer technologies. While this article focuses on the potentials of image editing and manipulation for art historical research, we feel propelled to express caution. The image observer must always be aware of the possibility of image manipulation and must challenge and critically evaluate the image content. We advocate for a transparent manipulation of digital art images, because the benefits for art historical research are too significant as following examples show. Editing programs such as AdobePhotoshop enable art historians to perform simple operations: they can crop details from images, alter the format and resize the image or change color or saturation. These alterations already offer new insights: a change of color emphasizes its importance for an artwork and illustrates how its perception and general meaning changes through the modification of color. By altering formal properties of images we produce different versions of the artwork and present new perspectives. Section four includes research examples which also produce unknown views of the object or artwork, however exceeding these early operations by far; computational methods not only change color, but alter textures, brushstrokes, content or the shape of objects.

Through editing programs scholars might also enlarge images to study brushstroke and texture of artworks or details, which would be overlooked or remain unnoticed otherwise. By enlarging image regions, art historians are able to study Jan van Eyck’s Arnolfini Portrait (1434) in detail. The mirror depicted in the background was particularly interesting for art historians. Essays discussed the identity of the persons shown in the reflection or the frame depicting the ten stations from the Passion of Christ. The observer is further propelled to appreciate the haptic quality of the dog’s fur, the material of the chandelier, the flowers seen through the window or the decorative details of the wooden bed [Postel 2017]. All details give proof of the mastery of van Eyck and can only be studied through the availability of the digital image by most scholars as they do not have access to the respective museum or are not allowed to see the physical painting up close (see Figure 2).

In contrast to digital images, analog images are restricted, often attached to a specific space and thus can only be enjoyed by a selected group of people who have access to specific spaces, museums or can afford entrance fees. The access to digital images is offered to much more people and simplified by computers and the internet. Seemingly audiences can immerse themselves in art and culture through digital images regardless of geographic location, nationality or social status. A study of the “World Bank” conducted in 2016 however has shown that while technologies have connected people on large scale, sixty percent of the world’s population still have no access to the internet. This means that there is a growing digital divide between the rich and poor and rural and suburban regions and that advantages of digital technologies are mostly beneficial for the wealthy and educated [Elliott 2016]. Thus large parts of the population are still not able to enjoy art through digital images. Other factors further limit the access to images, this includes fees to image collections, copyright or political regulations. While art has become more inclusive through the wider and simplified access to digital art images, limitations and exclusions of specific demographics are still present.

Essentially the numerical representation of digital images facilitates their duplication, modification, availability or accessibility — all these are the specific characteristics of digital data. Aforementioned operations such as the changing of color, saturation or cropping of details are possible because of these characteristics, but only present a small fraction of current possibilities. The availability of digital images and their (numerical) representation also support the employment of computational methods for art analysis and their subsequent processing through neural networks. Computational methods by far exceed previous operations and capabilities and are able to produce even more knowledge. The specific characteristics of digital data and the presence of automatic methods in computer vision highlight that it is not enough to simply simulate analog methods in order to produce new knowledge for the field of art history but that computer scientists in collaboration with art historians must create new approaches.

The previous section has emphasized that analog and digital data are characterized by different qualities, the latter for example is reproducible and modifiable. These differences suggest that although both formats still carry the same information, methods to extract information and produce new knowledge must differ. Analog images are often visualized in pairs; artworks are placed next to each other on a table or pinned on a board to compare visual properties. Heinrich Wölfflin’s method to study artworks was essentially comparative: pairs of images, which embodied different sets of visual principles (linear versus painterly, plane versus recession, closed versus open, multiplicity versus unity, absolute versus relative clarity), were used to emphasize stylistic and historical shifts between the Renaissance and Baroque period [Hatt/Klonk 2006]. Aby Warburg extended this comparative approach with the Mnemosyne Atlas; his wooden panels provided space to visualize multiple images. An analog visualization might be further enlarged, but at one point the physicality of analog images demands too much space and the visualization is hardly comprehensible and understandable to the observer anymore. If we adapt this visualization method for digital data, it means that images are viewed — although on a computer screen — in pairs or a limited number. However, the presentation in pairs neglects the fact that a digital space offers to display significantly more images: thousands of images might be viewed and compared simultaneously; the user can change perspectives, zoom in or obtain an overview of the data. Visualizations of digital data are much more interactive and adaptable to needs and requirements of scholars. The t-SNE Map project, created by Cyril Diagne, Nicolas Barradeau and Simon Doury for Google’s Arts and Culture Experiments, uses machine learning algorithms to map thousands of artworks in an interactive 3D landscape according to visual similarities [Diagne et al. 2018]. The project enables a virtual tour through the digital collections currently included in the Google Arts and Cultureonline website. The platform currently hosts collections from 1,200 museums and cultural institutions worldwide, including the Museum of Modern Art (New York), Uffizi Gallery (Florence), The Art Institute Chicago or the Kunsthistorisches Museum in Vienna [Google Arts and Culture 2011]. Although the t-SNE Map project focuses on data visualization and exploration and neglects a thorough data analysis, it enhances the possibility of computer technologies to access large image collections and study artworks from various angles and at different scales. If we continue to use techniques which resemble analog approaches to visualize digital images, we do not embrace the fact that computer-based visualization techniques significantly enhance our viewing practices.

The digital presence and reproducibility of digital data also enables its localization in many data networks or the combination of multiple networks. “The point is that we could situate a work within the many networks from which it gains meaning and value” [Drucker 2013]. This affects the analysis of single images, since its function and role within many networks can be studied, revealing its ambiguous meaning and adaptiveness to new social or political contexts.

In the past analog data such as prints, photos or books were often stored in rigid systems, organized according to categories, such as genre, technique or topic, or specific data models thus limiting the retrieval of records, exchange of data between different systems or users. Data models define the organization and meaning of the respective data collection [West 2011]. There are various models which are used to store data in databases and other information systems: popular examples are the network-model, relational model, entity-set model or hierarchical model. The former three were used as a basis for the entity-relationship model (ER-model), introduced in a paper by Peter Pin-Shan Chen in 1976 [Chen 1976]. The ER-model “incorporates some of the important semantic information about the real world” [Chen 1976] and provides information about entities and their relationships. The model offers different views of the data because it combines the network model, relational model and entity-set model. The network model consists of divided entities and relationships, which are presented in a non-hierarchical structure [Chen 1976]. The relational model is based on relation theory, where the term relation refers to its usage in mathematics: “Given Sets S1, S2, ..., Sn..., R is a relation on these n sets if it is a set of n-tuples each of which has its first element from S1, its second element from S2, and so on” [Codd 1970]. These relations if straightforward are expressed in arrays of columns [Codd 1970]. Lastly the entity set model is concerned with the properties of entities. Every entity belongs to a different entity set such as secretary, engineer or department; to test whether or not an entity belongs to a distinct set, we have to look at the properties attached to an entity set. It is assumed that if an entity is in the set secretary, it shares properties with all other entities in the set secretary as well [Chen 1976]. Although the ER-model superimposes structure onto digital art data, the value of early data models to separate elements, model their relationship and define common attributes among a set of similar elements must be recognized. The hierarchical model organizes data in a tree structure. This means that records are classified from general to more specific categories, which are predefined by scholars. For art history categories include but are not restricted to epoch, style, artist and topics recurrently painted by artists. The selection of a data model depends on the following aspects: the database in which it is integrated, topics and format of the data, its subsequent function, or the discipline which uses the data. Essentially models should simplify the storage, retrieval and exchange of the data. However because listed data models operate on a fixed, rigid structure and databases often facilitate different models, it is difficult to link various databases or share data [West 2011]. Moreover we also limit their points of access: then images are only accessible through predefined categories, which prohibit a free and individual exploration of the data. After digitization techniques had produced large image repositories, scholars had to promptly decide on a way to store the data meaningfully. Consequently scholars simply applied traditional models to databases such as the hierarchical model and classified digital data accordingly. But in order for data to be accessible by scholars from different disciplines asking diverse research questions, databases have to be flexible, dynamic and open-ended. By assigning digital images to these rigid structures, we limit their flexibility and usability for different, personal visual networks. Consequently (digital) artworks should not be organized according to existing data models, which would describe and predefine them according to terms of the past, but according to flexible and adaptive models.

New forms of analysis become possible through the digital format of the data. Methods are manifold, varied, adaptive to individual research questions and often easier to use than analog methods. The latter is due to the fact that a computer-based analysis is often unsupervised, which means that the processing of images requires minimum human intervention and no descriptive image labels (i.e., genre or style). Computational methods are usable by a general audience, because they do not require expert knowledge to establish for example similarities between images; the link is suggested automatically by the computer, which is capable of processing thousands of images in a short amount of time. There are manifold and varied computational methods which combined provide a multilevel view of the data and make complex interpretations. In contrast to analog methods, computer-based approaches offer a statistical evaluation and visual analysis, the latter is performed by computer vision which transforms images into visual representations (and not numbers). Analog methods are not capable of a statistical analysis, because image information is not stored as numbers, which are easy to read for the computer but not for the human observer. For digital images, we might analyze and measure color distributions, saturation or even the image composition to answer questions such as what colors were preferably used in a given time period or how artworks are composed in general. The study of visual properties with computer technologies reveals overarching or unusual visual patterns. All this information is inherent in digital images and obtainable by computational methods.

Eventually, most analog methods were developed by art historians or adapted from other disciplines: in the early days of digital humanities, traditional methods from art history or computer vision were simply adapted and applied to digital art data without considering their specific needs [Lang and Ommer 2018c]. Aby Warburg’s Mnemosyne Atlas and corresponding panels are models for early approaches: similar to the computer screen, Warburg’s panels displayed numerous images (instead of pairs) and included hyperlinks and a non-linear, dynamic and open-ended structure [Kalkstein 2019]. However Warburg’s space was restricted by the physical dimension of the wooden panels, which measured approximately 150 x 200 cm [Cornell University Library 2016a], and the number of links between images and hyperlinks to external sources were limited to the information and photographic reproductions available to Warburg. While digital approaches resemble Warburg’s Atlas and his way of grouping images, computer technologies exceed Warburg’s abilities and encourage a display of complex image structures and numerous possibilities to access image collections. At present art historians and computer scientists consider these new capabilities and jointly develop genuine computational methods for art historical research. Thus we must encourage collaborations between disciplines to ensure that new methods truly assist art historical research and reduce skepticism towards computational methods. An increased interest is not least visible in the considerable number of digital humanities conferences and journals worldwide; the Digital Humanities Conference, which takes place annually and is organized by the Alliance of Digital Humanities Organizations (ADHO), receives contributions from scholars working in the humanities and sciences, thereby demonstrating an increasing intersection of both disciplines [ADHO 2019]. However reactions of art historians towards computational methods and a collaboration have been ambiguous: while most use digital images for presentations to visualize research hypotheses or show image details, or online databases to gather images and information, comparably few actually engage in a collaboration with computer scientists and use computational methods for their research. While digital humanities as a subject has been increasingly established at universities worldwide, both art historians and computer scientists are still skeptic towards a general collaboration and the usability and validity of methods and results. A survey among art historians and computer scientists conducted in 2014 amplified these observations. It “...intended to shed light on field members’ knowledge of the capabilities and applications of computer technology, attitudes and perceptions about the use of it, and reactions to the meaning of this type pf digitization of the humanities” [Spratt and Elgammal 2014]. The questionnaire was completed by fifty-nine art historians and sixteen computer scientists and allowed two responses, namely approvals and disapprovals. While we are unable to discuss the survey in its entirety, we nevertheless want to highlight some findings. There was an overall consensus regarding the digitization of humanities: the majority of participants from both fields answered positively when asked about possible collaborations. However more than fifty percent of art historians did not view the implementation of AI technologies in humanities as a positive shift; in comparison, the majority of computer scientists did. The survey also brought forth that most art historians were unaware of the current possibilities of computer technologies for a visual analysis and only knew about the most basic applications, including a physical assessment of art or image retrieval based on descriptive labels [Spratt and Elgammal 2014]. “Why would I entrust my life’s work to a computer just so I could be made irrelevant” [Spratt and Elgammal 2014], was one response by an art historian. Although the survey was conducted five years ago and the field of digital humanities has massively enlarged and evolved since then, the fear of being replaceable and a general skepticism is still perceptible. A solution to further reduce skepticism and encourage collaborations might be to be more transparent with each other, so that the other discipline, its intentions and methods do not remain a black box. Another solution could be to already approach students of art history and computer science and inform them about the potentials of a collaboration. So far students in art history are less affected by computational tools as access has not been widely available. Their use of digital methods and images has been limited to simple visualization techniques and online databases from which they obtain images and information, sources include WikiArt, Wikipedia, museum collections or university catalogues. Digital images have been mostly used for presentations and term papers and altered through basic editing programs, which allowed them to change color, saturation or size. In turn students of computer science utilize art data as another source for the classification or detection task or to increase the robustness of algorithms. However we must emphasize that a collaboration promises much more: it produces new methods to visualize data or to perform an automatic data analysis, which “draws upon multiple forms of visual analysis that together make complex interpretations of a given object or group of objects” [Spratt and Elgammal 2014]. Consequently methods enable new questions relating to the popularity or the spatial and temporal spread of a motif. In turn computer science develops robust algorithms and produces methods, which are not only capable to perceive what is depicted but also how. Lastly a collaboration enhances and creates new work fields for art historians and computer scientists.

Previous sections emphasized that it is not sufficient to simply translate analog into digital methods, because they are not reflective of characteristics and possibilities offered by digital data. It is only when we develop new computational methods to process digital data that we are capable of producing new knowledge, which is not possible with analog data and respective methods and which in turn affects not only art historians but also earlier generations of scholars, namely students.

Based on research projects such as the Time Machine or Google’s Arts and Culture Experiments and previous remarks we argue that computational methods are especially beneficial to access, link and edit digital images. This section discusses these aspects in more detail and concludes that especially operations of accessing and linking are closely related.

“While access and availability of the art historical corpus have been facilitated by the development of online repositories, tools for... computational processing have been slow to emerge” [Drucker 2013]. Only in recent years have scholars spent increasing efforts to create sufficient tools and methods to access digital image collections. However most are still inspired by analog methods. We have already noted that many databases use data models, such as the network-model, relational model, entity-set model or hierarchical model, which structure collections according to predefined concepts but hinder individual access and a free data exploration. In contrast, computational methods independent of traditional database structures, provide access to large data collections on individual terms. Such methods, which allow easy access and a free exploration, have already been hypothesized at the middle of the twentieth century. In 1945 Vannevar Bush (1890-1974), American engineer and head of the US Office of Scientific Research and Development during World War Two, envisioned a “future device for individual use, which is a sort of mechanized private file and library... A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his (the user’s) memory” [Bush 2015]. At the end of World War Two Bush already demanded what is at the forefront of digital humanities’ research today, namely to build storage spaces for digital data and methods, which retrieve records fast and according to individual demands. Although Bush remained vague about how these devices should look — supposedly because necessary technologies were missing — , his request that repositories should be consulted with exceeding flexibility supports the current urge to be able to access image collections independent of predefined terms. As we will see in section four current computational methods for object detection, often implemented in interfaces, operate on purely visual information. These methods enable a search for objects, compositions or other arbitrary image regions in large data collections and do not require tedious and error-prone annotations to access data. Thus users formulate search queries based on visual input and do not need to provide text queries, which require knowledge about art historical terms. Instead of searching for examples of a “still life” users can simply mark an arrangement of fruits, cut flowers or vases to find similar instances. Access to repositories is not only granted through interfaces but also through data visualizations. The digital space is capable of displaying thousands of images at the same time, allowing a direct comparison of image content or style — this has been exemplified by the t-SNE Map project of Google’s Arts and Culture Experiments [Diagne et al. 2018]. Through visualizations art historians can recognize patterns such as groups of artworks sharing a similar motif or formal property and identify gaps, which indicate that certain visual features were not popular at a certain time.

There have been art historians in the past, who explicitly focused on how art collections can be accessed through visualizations to be comprehensible for the viewer. Hamburg-born art historian and cultural scientist Aby Warburg and his Mnemosyne Atlas are often referred to as a starting point for current practices in digital art history. Digital Media professor Stefka Hristova compared present methods of data visualization with Warburg’s method to examine “shifts and continuities that have shaped informational aesthetics [...] and data-driven narratives” [Hristova 2016].

Warburg accessed image repositories by establishing visual lines of continuity, mainly linking Western antiquity with the Renaissance. He coined the term “Bilderfahrzeuge” to describe the migration of images, objects and other visual features throughout time 4. The term and concept was materialized through the Mnemosyne Atlas: the scholar pinned reproductions of artworks on wooden panels covered with black cloth to group images along a common theme [Hristova 2016]. These images, which consisted of a relatively fixed number of 2000 reproductions, were often regrouped or completely removed emphasizing a great dynamic of the Atlas [Warnke 2000]. Warburg was interested in two main concepts, namely the expression of human emotion (Pathosformel) and the afterlife of antiquity (Nachleben der Antike) [Impett and Süsstrunk 2016]. Both concepts were traced from antiquity to the Renaissance through their depiction in paintings, drawings, sculptures and objects. He visualized this continuum by merging images and presenting them on one panel. In this way, the Atlas not only operated across genres and media but demonstrated an efficient way of information processing. The Atlas enabled Warburg to access a substantial part of the repository of art by linking images based on specific properties content. He focused on “‘images of great, symbolic, intellectual, and emotional power’ from the art and culture of Western antiquity through the Renaissance, and up through his own day” [Kalkstein 2019]. Although Warburg was interested in non-Western cultures — in 1923 he gave a talk on images from the Pueblo Indian region — his Atlas focused on Western art and specific periods which are indicative of Western art history such as the Renaissance [Hvattum and Hermansen 2004]. This restriction might be due to the fact that he selected most reproductions from his own library or photographic collection, which both focused on Western art [Kalkstein 2019]: in 1909 Warburg wrote to his assistant Wilhelm Waetzoldt (1880-1945): “I have now got a library for cultural sciences (with a focus on Italy) of ca. 9,000 volumes, to which 600 are added every year, several thousand photographs and a few hundred slides. The aim: a new methodology of cultural science based on the ‘reading’ of the pictorial work. Area: Europe in the fifteenth century” [The Warburg Institute 2018]. Although his library was comprehensive, it focused on specific geographic regions and times, namely Europe and the Renaissance, and thus mainly neglected images of non-Western cultures. This is further indicated in his introduction to the Mnemosyne Atlas written in the late 1920s; the text presents Warburg’s thoughts on “social memory, the origin of artistic expression and the psychological drama driving the history of European culture from classical antiquity onwards” [Warburg and Rampley 2009]. Although Warburg’s Atlas utilized a restricted and mostly consistent number of images, it remains an important attempt and role model for current visualization techniques and research attempts: he not only viewed the single image but placed it within a larger artistic network to make sense of visual continuities, to identify and understand similarities between artworks, which testify to artistic relationships and adaptation processes. The latter referring to the circulation of distinct motifs, compositions, styles and other visual properties over time and space.

The process of linking to find analogies is a central human behavior and studied in detail by neuroscience. To make sense of new experiences and impressions, our brain also uses associations: Neuroscientist Moshe Bar emphasizes the importance of analogies for associations and consequently to make predictions of the future. Bar proposes a framework which studies how the human brain “...generates proactive predictions that facilitate our interactions with the environment...” [Bar 2009]. He proposes that incoming information is not analyzed isolated in order to be understood, but that links between new and existing, already familiar information are used to make sense of the unknown. Like Warburg, Bar’s theory is based on the presence of analogies and the search for similar information stored in our memory. He famously declared that when seeing a new object, we should not ask “What is this?” but rather “What is this like?” [Bar 2009]. Essentially the concept of linking is central for computer science: section four illustrates how computational methods based on neural networks find visual correspondences automatically; but even earlier data models are interested in links. The ER-model for example models entities and their relationships, both of which are expressed in a given data collection [Chen 1976]. The model considers various possible relationships, namely one-to-one, many-to-one or many-to-many relationships. The example Secretary (entity) works for engineer (entity) indicates one-to-many relationships: While every engineer can have only one secretary, a secretary can work for multiple engineers. This diagram is further enriched by adding descriptive attributes to entities such as name, color, or height [Teorey et al. 1986].

Both methods proposed by Warburg and Bush established links manually and based on knowledge which the scholar had prior to his research. Warburg’s Atlas was mainly used to visualize his arguments and hypotheses. Thus most visual links were already known or at least suspected and therefore produced little new knowledge. In contrast, computational methods establish links automatically and independently of the art historian. Doing so, these methods hold the potential to identify links between thousands of artworks, which have not been known by art historians before. In addition new approaches overcome spatial limitations which characterize previous visualization techniques such as Warburg’s Mnemosyne Atlas. Thus, art historians are able to establish comprehensive relations between artists, determine influences and the adaptation of motifs or styles by different artists. Accessing and exploring data collections through linking means that the artwork is not studied in isolation but always seen in comparison to others; by searching for predecessors or successors, scholars aim to make sense of short-term and long-lasting phenomena in the history of art. Where and how did a motif such as the skull evolve? In which context was it presented? Can we find predecessors for the Cubist style? These and similar questions can be answered by establishing visual links. The idea of linking is extended by, for example, hyperlinks, where connections to other data even of different media can be established simply by clicking — the internet consists of such hyperlinks.

Warburg’s Mnemosyne Atlas has shown how the process of linking allows us to open up image collections of art, thereby creating visual trails and lines of continuity. The idea of linking was taken up by Bush for his memex device which was intended to operate on associations and therefore resembled the workings of the human mind [Bush 2015]. What has been performed by Warburg to a limited extent and only hypothesized by Bush has become reality through computational methods: thousands of art images can be visualized in one space, thereby revealing visual patterns common in essentially distant artworks. Consequently past works can only provide suggestions, guidelines and validations for current practices in the digital humanities, because the physicality of photographic reproductions and space limitations do not resemble current conditions.

Previous sections have provided more information about how computational methods are beneficial to access and link images; this last paragraph focuses on the aspect of editing. Eventually the combination of digital data and computational methods enables an editing of images. The format and reproducibility of digital images allow the modification of images without actually altering the physical artwork. One can change formal features such as color, saturation, brightness, contrast with easily available editing programs such as AdobePhotoshop. Art historians are thus able to improve the quality of reproductions or study the importance of color, etc., for an artwork. Through image editing, scholars might also crop digital images to compare details or reconstruct the hanging of artworks in past exhibitions in a digital space to study viewing practices or exhibition contexts. More complex computational methods such as style transfer or image synthesis, which are introduced in the following section, describe a more conspicuous editing of digital images. Recent works by Gatys et al. and Sanakoyeu et al. have used computational models to stylize a real photo in the style of a specific artist, this process is referred to as “style transfer” in computer vision [Gatys et al. 2015] [Sanakoyeu et al. 2018]. Visual features of a style such as color, object shape, texture and brushstroke are learnt by a neural network and transferred onto the real image. Image stylization thus holds the potential to generate new perspectives and visualize how an artist would have painted a modern scene. Generative models which are used for style transfer also offer the potential to generate unseen views of an object by for example synthesizing two images. Through image generation, lost, damaged or unfinished artworks can be reconstructed, producing new knowledge for art history. Both examples indicate that generative models enable an editing of images unimaginable with analog data or analog methods.

Previous sections have identified the potentials of using computer technologies for accessing, linking and editing images. The discussion has highlighted that methods to access collections or link images based on visual similarities have already been suggested by art history and studied by other fields such as neuroscience. While these attempts provide valuable guidelines for current computational methods, examples given in the next section emphasize that models based on neural networks exceed previous attempts by far.