In recent years, numerous projects have been initiated and funded that aim to digitize cultural artifacts. Archives, museums, libraries, and other heritage institutions are now facing the challenge to make digitized inventories publicly accessible and provide new modes of engagement with the cultural content. At the same time, there is an increase in visualization research related to the digital humanities that explores computational and data-driven approaches to literature, art, and other cultural domains. The asset of computational analysis and visualization is often reduced to their benefit in regards to quantitative analysis, which implies a distancing from the singular object or phenomenon. At the same time, the object-centered nature of museums, art history, archeology, and archives is fueling a growing interest in harmonizing digital approaches with qualitative and interpretative methods, i.e., “close viewing” or “close reading,” an approach that we aim to extend with this work.

The case study[1] that we present shows the potential of applying techniques largely drawn from information visualization to an art historical source. With a zoomable visualization environment we seek to explore how a “distant-viewing” approach can be combined with a “close-viewing” mode in a dynamic and contextualized arrangement (see Fig. 1). We aim to examine how visualizing the quantitative structure of a collection can be complemented by allowing for examination of the qualitative texture of the individual object. We describe a design space for visualizing cultural collections that also does justice to the aesthetic value of the collection and objects. Our contributions further include an evaluation of the prototype in regards to usability, user interaction, and audience groups.

Our work builds on prior research on information visualization in the context of digital humanities, the emerging field of digital art history, and visualizations particularly developed for cultural collections.

The analysis of textual data has been an important method for the largely language-centric digital humanities [Jänicke et al. 2015] and a major strand of research in information visualization [Kucher and Kerran 2014]. While early text visualizations were created for relatively abstract use contexts, more recent work focuses on specific needs of particular user groups such as humanities scholars [Jänicke et al. 2015] [Koch et al. 2014]. The scope of visualization tools for textual analysis ranges between single-purpose tools [Wattenberg and Viégas 2008] to comprehensive analysis environments  [Muralidharan and Hearst 2013]. Moretti’s notion of “distant reading” [Moretti 2005] has become a provocative yet productive term to negotiate quantitative and qualitative methods of literary history, and more generally for interpreting cultural data. The distinction between close and distant readings has informed a wide range of visualization projects aiming to support analysis that ranges from individual words and sentences to documents and entire corpora [Jänicke et al. 2015].

Apart from these predominantly text-based efforts in digital humanities, the broader context of digital cultural heritage has attracted a variety of experimental visualization research and innovative design of collection interfaces. Visualizations of cultural heritage collections promise to offer “rich-prospect interfaces” that represent an entire collection at once, afford interactive control over its arrangement, and provide detailed access to its elements [Ruecker et al. 2011]. This approach is based on the principle that the default interface should “show a meaningful representation […] of every single item in the collection”  [Ruecker et al. 2011] that is apparent to the user and helps them to understand, on a visual basis, what is available in a collection. Image plots of thousands of Instagram photos arranged by time and hue fulfil some of these aspirations in that they blend the representation of the metadata with the visual qualities of the images [Hochman and Manovich 2013]. The resulting visualizations offer impressions of large stocks of photos that were not visible in that way before. The Viewshare tool was explicitly designed for rapid creation of visual interfaces that can be used to explore and share patterns in collection data using established techniques such as maps, timelines, and lists [Algee et al. 2012]. In contrast, the “Bohemian Bookshelf” interface contains deliberately idiosyncratic visualizations each offering a unique point of entry to a visually rich and multidimensional collection [Thudt et al. 2012]. The principle of “generosity” can be used to qualify to which degree an interface reveals the material and semantic qualities of a collection and encourages its exploration [Whitelaw 2015]. Our research aims to contribute to this line of work on generous and rich-prospect interfaces for cultural collections in particuarly by supporting the shift between abstract, distanced explorations and highly detailed close-up views.

Related to these developments in the wider cultural sector, digital art history is gradually emerging as a research area in its own right [Bentkowska-Kafel 2005] [Drucker 2013] [Promey and Stewart 1997] with a distinct set of use cases, tasks, and requirements for digital tools and data structures. While early mentions of the term focused on teaching [Promey and Stewart 1997], there has been a continued interest in making sense of digital technologies and exploring their potential for art history, especially with regard to purposes like preservation and presentation [Bentkowska-Kafel 2005]. Museums and art history, although preoccupied with visual resources, have been predominantly text-based fields of research in that sense that the visual object of study had to be translated into text (e.g. on index cards or in cataloguing systems and databases) to be able to retrieve relevant information from rich collections, either analog or digital. Efforts in computation and digital humanities research thus also include the improvement of ontologies and text-based information retrieval systems [Isemann and Ahmad 2014] or the use of folksonomy or crowdsourcing for textual annotation of cultural heritage [Ridge 2014]. Consequently, some approaches in digital art history research build upon the prior developments in text-based humanities, introducing approaches that allow for “distant viewing” and quantitative analysis of art historical data [Bender 2015]. Computational non-textual analysis of art historical resources, on the other hand, include methods for image analysis that are able to group or cluster artworks on the basis of similarity of artistic style [Shamir and Tarakhovsky 2012] or the development of computer vision algorithms classifying fine-art paintings by style, genre, and artist [Saleh and Elgammal 2015]. Although the application of digital methods in art historical research still represents a small subset of disciplinary practice, digital art history is on the rise [Zweig 2015].

Despite the dominance of “distant” approaches in digital humanities and digital art history, there is a growing recognition that digital methods in these fields should incorporate qualitative and interpretive modes of analysis [Schnapp et al. 2009]. Drucker argues for a fundamental shift in visualization design from viewer-independent representation of data to viewer-codependent interpretation of “capta”, i.e., data that is purposefully collected and constructed [Drucker 2011]. Data in the arts and humanities are not about absolute truths, they are constructed to develop novel interpretations and advance arguments. Visualizations in this sense should be seen as propositions for a certain reading (or viewing) of a phenomenon [Galey and Ruecker 2010]. Current visualization research in the context of literary studies reflects this need for interpretive, experimental, and even playful approaches to data [McCurdy et al. 2016] [Hinrichs et al. 2016]. Similarly, our intention is to conceive intriguing visualizations that make rich art historical collections available for casual analysis and exploration while still supporting detailed inspection.

Much of digital art history, and more generally digital cultural heritage, is reliant on textual information about artifacts. Additionally, computationally processed, clustered, and interpreted image data, or sometimes a combination of text and digital image processing, are used to organise and make sense of digital cultural heritage. Metadata, understood as textual information on (digitised) cultural heritage, connects the expert’s knowledge to the visual resources and helps to address entities via title, author, date oder descriptive elements like keywords or classification [Schumacher 2016]. As part of the efforts to also develop visually rich representations of and new modes of interaction with digital cultural heritage, there is a growing interest in the question of how the underlying textual-digital fragmentation of an artifact influences our perception of digital cultural heritage on a more general note. These discourses engage, amongst others, with the question of how to address, represent or preserve the materiality of objects in digital representations. This is especially relevant in the context of museums that, for a long time, have given primacy to the materiality of objects. Nonetheless, within museum studies, and the study of material culture in general, the notion of “object biographies” has come to acknowledge the ways in which objects become invested with meaning through the social interactions they are caught up in [Gosden and Marshall 1999]. This shift away from a strict focus on materiality and towards a notion that understands the accumulation and transformation of an object’s meaning and values as a distinctly sociocultural process can now also be accentuated in the context of digital cultural heritage. Objects, in this sense, can be recognised as being in a state of motion, that might occupy or migrate through different states and media [Parry 2007]. Nonetheless, debates on the meaning, value, and presence of digital objects, quite often still rely on the object-centeredness of museum culture and thus argue from the standpoint of the “superior” physical counterpart of the digital object [Cameron 2007]. In this context, we would like to acknowledge that the materiality of a museum object could also be problematised even in regards to how it is conveyed in a museum display. Although museums can be regarded as “material institutions par excellence”  [Dudley 2012], they often limit the extent to which people can engage with the things on display and thus likewise limit the access to the objects’ materiality [Dudley 2012]. Naturally, this restricted access is a due measure of precaution that ensures the preservation of valuable and unique objects. Nonetheless, neither in a physical nor digital display can we touch, feel, smell, taste, or weigh the objects with the help of all our senses. What we have to rely on in both instances is our sense of vision and, if provided, the “knowledge” assigned to the (digital) object by the museum as metadata or other additional information [Dudley 2012]. Concerning the influence of new media on our way of perceiving and engaging with cultural heritage, Jeff Malpas has remarked that we need to retain a sense of the materiality of objects “understood in terms of the way in which human life is essentially formed and articulated, and so also understood, in relation to spatio-temporal formation and articulation”  [Malpas 2007, 19]. He emphasises the need to differentiate between what is interpretation and what is the object in its spatio/temporal materiality in order to not “lose a sense of the object”  [Malpas 2007, 24]. With our research we are attempting to address some of the challenges arising from the material and interpretative complexity within cultural collections.

With the expanding use of digital methods in the humanities, tool creation is certainly a valuable scholarly practice [Davis and Kräutli 2015] [Schnapp et al. 2009] that expands the ways a phenomenon can be seen and examined. Prompted by limited adoption of more advanced tools in the digital humanities, the results of a survey with historians highlight the critical importance of interface design, instructions, and ease of use [Gibbs and Owens 2012]. Besides their immediate function, visualizations also act as arguments and rhetorical expressions [Galey and Ruecker 2010] [Hullman and Diakopoulos 2011] that can serve critical approaches towards cultural collections [Glinka et al. 2015]. Accordingly, to benefit the research agendas in the humanities, it is the scholars themselves who should be much more involved in setting the parameters for the design of tools and formulating questions for their interpretation and critique [Drucker 2013]. Taking these findings into account, we are pursuing an interdisciplinary research process that brings together humanities scholars, designers, and engineers to collaboratively create and make sense of visualizations of cultural collections.

The temporal dimension is often of particular importance in cultural collections. When reflecting this by visualizing cultural data along a timeline, it is advisable to also consider its genealogy [Davis et al. 2013], going back to the design decisions made in the 18th century when the first timelines were conceived [Davis and Kräutli 2015]. Even though the timeline is almost omnipresent, there can be good reasons to subvert its linearity, for example, to counter absolute notions of time and advance more complex models of times [Drucker 2011], expose the uncertainty in the dating of artifacts [Davis and Kräutli 2015], or to pique the interest of an exhibition audience about the oeuvre of an artist [Hinrichs et al. 2008]. The temporal dimension can also serve as the basis to trace the evolution of visual features of art such as color usage in paintings [Haber et al. 2011]. Following the aim to provide users with a visual basis for understanding what is available in a collection in rich-prospect interfaces, Ruecker et al. have identified the temporal arrangement of collections as one that helps identify meaningful connections between items [Ruecker et al. 2011]. In our research we continue this line of work of visualizing temporal patterns in cultural collections and further seek to integrate the material and thematic qualities of the artifacts.

There is a traditional emphasis on the overview in information visualization [Shneiderman 1996], which is typically understood to be distinct from the preview of individual resources in a collection [Greene et al. 2000]. However, the primacy of overviews is increasingly being challenged, for example, by visually rich “innerviews” as local perspectives that show both the visual material and its dense metadata [Whitelaw 2015]. Akin to blending or alternating between distant and close reading, there is growing interest in bringing bird’s-eye and close-up views on collections together. For shifting from high-level aggregations of an entire collection to the detailed views of individual resources, zoomable interfaces prove particularly powerful [Bederson and Hollan 1994] [Dörk et al. 2012] [Hochman and Manovich 2013] [SFMOMA 2008]. Typically, however, the arrangement of elements in zoomable interfaces is relatively static and highly detailed views of the artifacts are seldom provided. Our intention is to develop dynamic arrangements of rich, visual artifacts that make full use of interactive and graphical capabilities of modern web technologies. Thus, we build upon and refine visualization techniques in the context of the challenges and potentials of cultural heritage and digital art history.

Within the scope of this research project, we are collaborating with the Prussian Palaces and Gardens Foundation Berlin-Brandenburg or SPSG, a cultural heritage foundation that administers several historical buildings, palaces, gardens, collections of paintings, furniture, sculptures, porcelain, drawings, and other historical objects, which have only partially been digitized, so far. Motivated by the lack of research combining quantitative and qualitative analysis of art historical collections, we proceed with the ambition to select a data set that would allow us to examine a collection along its structure, i.e., based on the artefacts’ metadata, as well as their texture, i.e., visual, physical, or aesthetic qualities. Additionally, we seek to incorporate context, i.e., frame the collection in such a way that would allow non-experts (or interested laypersons) to explore the collection. In the following we detail our framework of collaborative, interdisciplinary research and the design and development of cultural heritage visualizations in cooperation with art historians, designers, and engineers.

Our collaborators repeatedly emphasized the temporal and thematic patterns to be particularly important for making sense of the collection. In addition, there was an interest in incorporating the actual reproductions of the drawings in the visualization. This corresponds with the assertion that visualizations, especially in the context of digital humanities, should take into account the unique visual and material qualities of digitized collections to reflect their physical counterparts [Hinrichs et al. 2016]. Obviously, the digital cannot surmount the restricted access to materiality posed upon a museum object. Nonetheless, we can try to at least tap into the full potential of displaying a digitized museum object and still be aware that much of what constitutes the display and the ways of engaging with the object is based on the interpretation and not on its spatio-temporal materiality. Reflecting on these conditions, we have found it to be productive for our research to try and exhaust the given technological potentials in making high-resolution reproductions of fragile (and mostly non-public) objects available to the public. Thus, we hope to at least maximise the visual information about the material qualities that can be extracted from the digital collection at hand, maybe even surpassing the accessibility of the original that would be shielded from the public with “glass cases, picture frames, ropes”  [Dudley 2012], or other visually distancing precautionary measures. At the same time, we concede that it is not solely the physical materiality of the objects that affect the conditions under which we can engage with the collection by subjugating the arrangement of the visual resources to their interpretation (of temporal, contextual, or thematic attributes). The aforementioned challenge of conveying the materiality of digitial cultural heritage also influenced our decision-making in regards to the treatment and processing of the digitized drawings. We decided to preserve and highlight the traces of materiality that are conveyed through the high-resolution digital reproductions, such as ripped-out corners, faded-out strokes, and the proportions of the sheets and differences in size. Following our aims and research questions, we have devised a zoomable visualization environment that arranges a collection of drawings based on their temporal attribution, allows for thematic filtering, provides historical and art historical context, and enables the close inspection of the high-resolution images and their visual texture.

The interface contains a keyword visualization, a zoomable canvas and timeline, and information panels (see Fig. 3). The visual design of the interface with a dark screen canvas and muted colors takes into account the collection’s texture. Many prints, engravings, or drawings are monochromatic and low in contrast. This also being the case for the collection of drawings by Frederick William, a light screen canvas would have blended the drawings with their background. Since the original sheets were furthermore digitized mounted to a dark carrier paper, the contrast between the light sheets of paper and the dark carrier paper, that extends to the screen canvas, makes it possible to discern the cut-outs and texture of the paper more easily. Additionally, the physical context of the original drawings can be associated with the interface, since the design reminds of the dim and concentrated aesthetics of print rooms in museums. In the following we describe the specific design decisions behind the visualization along the three main facets of our visualization: time, themes, and texture.

To make the visualization of 1492 images immediately responsive and interactive we take several optimization measures[3]:

According to our research questions, we studied the visualization in different settings and with a range of audiences. Besides regularly discussing the prototype with our collaborators, we also presented it to other domain experts at network meetings, public talks, or research colloquiums and collected their responses and questions. At an earlier stage of development, the visualization was shown during an exhibition in Berlin under the title “Past Visions of the Future City–Exploration of drawings penned by Frederick William IV of Prussia,” putting an emphasis on the subset of drawings that depict architectural and urban planning visions for Berlin and Potsdam. Visitors mainly identified as being from the visualization and design community, but also as groups of museum professionals. Still, there was a good balance with spontaneous visitors from the neighborhood and surrounding areas with no expertise in neither one of the fields. Most exhibition visitors had no, or only little, knowledge of Prussian history or the King’s life. Accordingly, one particular insight from conversations with visitors was their desire to also learn more about Frederick William’s personal life or the political developments of his time that had an influence on the drawings and sketches.Thus, we decided to supplement the timeline with general historical facts, biographical events, and other relatable information that would help to comprehend the collection with little prior knowledge about the subject matter. By presenting these earlier stages of the prototype in controlled environments and with the potential to engage in conversations with domain experts or the general public, we were able to continuously react to feedback by our peers during the development process and learn more about how different types of users react to the visualization and interface. Finally, we deployed the visualization as a web application, logged user interactions, and linked a questionnaire in the interface.

The importance of close collaboration with domain experts in fields like digital art history [Promey and Stewart 1997] was self-evident throughout the process of our research and the development of the visualization. Only by including expert knowledge from the humanities as well as from interface design, computer science, and data visualization we were able to reach a level of mutual understanding for the relevant topics and challenges that were crucial to our work. Pursuing the notion of “cultural sensitivity,” the content of the collection was the driving force during the design process. Although the resulting visualization was not intended as a one-fits all solution in the sense of a visualization tool that could be used unmodified for a broad range of data sets, we recognized some promising aspects in the zoomable and filterable canvas that can be transferred to other use cases beyond this specific collection. In the following, we discuss some of our insights and critically evaluate our use case with regard to the areas that would call for additional research and refinement.

The collection subset used for the case study consists of about 1500 data records of a collection of drawings that will eventually include more than 7000 sheets. Knowing that the collection will be gradually digitized, we had to take into account the scalability of our approach. Although we have already run some preliminary tests with 7000 images, we would have to assess the performance of the visualization when scaled to such a size more thoroughly. These questions of scalability pave the way for transferring our approach to other image datasets of varying sizes and metadata structures. While the use case was specifically designed and developed for the drawings of Frederick William IV, the relatively monochrome drawings with low contrast are weaker in a ‘distant’ viewing mode, compared to images with higher contrast and a broader palette of colors. Thus, in order to assess the general viability of transferring our framework, we ran two trials with other data sets. For one, we explored about a thousand paintings, drawings, and other artworks by Vincent Van Gogh[6] (see Fig. 9). For the other, we loaded 3915 items from the New York Public Library collection of Detroit Publishing Company postcards[7]. Initial results indicate that images with a higher contrast and broader palette of colors lend themselves well to exploration in a zoomable and filterable image plot. For example, in the case of Van Gogh, not only quantitative insights about the thematic distribution of works are visualized, but also their visual texture can be examined, e.g., the dominance of darker colors in his early paintings compared to brighter or more colorful shades in the later years becomes clearly visible (see Fig. 9). Similarly, the potential of alternative layouts came up during the iterative development. On the one hand, these layouts could be particularly relevant to the specifics of a collection, for example, the geographical distribution was suggested by participants of our study. On the other hand, we are interested to see how the collection would behave in similarity-based layouts (such as t-SNE) or grid layouts [Klingemann 2015] based on the metadata. Since the metadata is based on in-depth art historical research along a controlled vocabulary, similarity-based layouts may support the exploration of drawings by affinity. In that sense, such layouts could translate the qualitative insights from the art historical classification into a spatial arrangement.

Cultural heritage data is often characterized by incompleteness and uncertainty. While we were able to employ a subset from a collection that had a relatively high level of completeness in regards to required metadata fields, depth of assessment, and a standardized taxonomy, it lies in the nature of qualitatively sourced data to implicate uncertainty, especially with regard to time [Davis and Kräutli 2015]. This also being the case for the subset of drawings by Frederick William IV, discussing the issue with our project partners again highlighted that this is not a problem that is possible to be solved, but that needs to be addressed in the design of what might be called “humanistic interfaces”  [Drucker 2011]. During the next iteration of the prototype, we hope to be able to futher discuss and explore how timeline visualizations could visually reflect temporal uncertainty. Even though we already decided to integrate additional contextual information in the timeline — thus including data that had to be specifically sourced and authored manually — some responses by participants in the questionnaire showed that there is a desire for even more contextual data and content from sources that go beyond the defined collection data set. Although the problem was addressed as a general one in regards to cultural heritage interfaces, considering the expansion of used sources and maybe even implementing a layer for community co-creation and annotation (by using folksonomy or crowdsourcing [Ridge 2014]) could be valuable during the next phase of the project. Nonetheless, the question of missing data also has to be seen with regard to the reality and actual state of digitization in most collecting institutions. The sheer amount of collection items, the costs of the digitization process, and the need of expertise when assessing an item and its metadata often make it hard for collecting institutions to meet such requirements just yet.

With this research we have explored the potential of zoomable visualizations of cultural heritage collections. A collaborative design process with a heritage institution resulted in a visualization technique, which is aimed at supporting a “distant-viewing” approach coupled with a “close-viewing” mode for the exploration of a historical collection. We showed how visualizing the quantitative structure of a collection can be complemented by allowing for examination of the qualitative texture of the individual object. This work contributes insights to discourses on digital heritage as well as on information visualization. In the early years, research in the field of digital cultural heritage and the digital humanities was often dominated by computational or technological solutions originally developed in other fields and for other purposes. Following a shift towards digital heritage research that puts the needs, challenges, and requirements of cultural heritage in the focus and then seeks to find technological and computational solutions for a mainly content-driven approach, we introduced our research in a sense of “cultural sensitivity.” At the same time, humanities research that employs computational methods is often perceived as dominated by a solely quantitative, positivist, or distant approach. Quantitative research in the humanities, in turn, is often questioned on the notion of the alleged necessity of qualitative examination in order to be able to truly grasp the sources’ significance. This “quantitative-qualitative divide” in the humanities, though, is not inherently a digitally inducted discord. Empirical or quantitative methodologies existed in the humanities long before the establishment of computers and web technologies in these fields [Drucker et al. 2015]. Even in pre-digital humanities research, the tendency to either value quantitative research over qualitative research or vice versa already existed [MacNealy 1999]. Rather than carrying on an antagonistic discourse of “close vs. distant,” “qualitative vs. quantitative” under the slogan of “digital vs. analog,” there is an increasing recognition of the potential in experimenting and engaging with digital technologies that can help to actually unify these seemingly opposed strands, predominantly in language-centric research [Jänicke et al. 2015] [Koch et al. 2014] [Muralidharan and Hearst 2013]. With our research we expand on this notion and apply it to a collection of visual resources. Our contributions thus are relevant for the ever growing research activities in digital art history, as well as a foundation for further work on visualization in the context of digital cultural heritage.

We wish to thank the reviewers of the Digital Humanities Quarterly for their thoughtful feedback and suggestions. We would like to acknowledge the German Federal Ministry of Education and Research (BMBF) for their generous funding of our research project VIKUS - Visualizing Cultural Collections (Visualisierung kultureller Sammlungen). We thank our project partners SPSG and Programmfabrik GmbH for the productive cooperation. We wish to thank Matthias Graf for his work on a previous version of the visualization, Jörg Meiner for his scientific consulting and for valuable ideas, Carsten Dilba for his continuous feedback, and – last but not least – we are thankful for the support from our colleagues Sebastian Meier, Till Nagel, Stephanie Neumann und Jan-Erik Stange.