Many previous text visualizations focused on some form of text. For instance there are visualizations of documents [Collins 2009a] [Paley 2002], of selected subsets of words [DeCamp 2005] [Hetzler 1998], and repetition in context [Hearst 1995] [van Ham 2009] [Wattenberg 2008]. There are also visualizations for grouping and clustering documents [Collins 2009b] [Wise 1995]. However, they do not approach whole language visualization. Most approaches to text visualization whether coming from the Information Visualization (InfoVis) community or the digital humanities, are often aesthetically pleasing, but have not been demonstrated to have utility for literary criticism or linguistics. This limitation can be seen in text visualization projects such as tag clouds [Lee 2010] [Viegas 2008] [Viegas 2009] word associations [Yatani 2011] [van Ham 2009], topic modeling [Chen 2006] [Gardiner 2010] [Gretarsson 2011] [Hall 2008], and text document comparison [Collins 2009a] [Lin 1992] [Smalheiser 2009] [Wise 1995]. There are text visualization projects that fall into the category of linguistics that have had some success. [Rohrdantz 2012] provide a visual analysis of suffixes in English, and the Google N-Gram[1] viewer has given access to historical usage data previously unattainable.

Within the domain of digital humanities, there are three main streams of visualization that tend to pervade the literature. All three types are based in text analysis but approach the problem from different directions. The first is the GIS type tools for organizing spatial data in the humanities [Jessop 2006] [Jessop 2008] [Wood 1992], specific examples of this type of work include Bingenheimer et al.’s visualization of the biographies of Buddhist monks [Bingenheimer 2009], as well as Valley of the Shadow [Valley 2015], in which there are examples of multiple types of quantitative and qualitative data presented in a GIS format.

The second category involves tools used to augment reading, text analysis tools that highlight relationships in texts, often with visualizations. This category includes work such as Clement’s distant reading of Gertrude Stein [Clement 2008], and Wattenberg et al.’s work on using tree visualizations for visual concordances [Wattenberg 2008]. Projects such as TAPoR [Tapor 2015] also provide data mining tools for literary and textual analyses. The aim of these projects is to allow for interesting portions of large texts to be marked up and visualized for comparison.

The third category of research is the straight visualization projects which are of two types: The first are the projects that use toolkits such as Voyant tools [Voyant 2015] to accomplish their research task and the second are the projects that create bespoke visualizations specific to their research problem. The second type of these includes Moretti’s work on visualizing interactions and character movements in literature [Moretti 2005], Bingerheimer et al.’s social network visualization from TEI data, which uses text encoded data to produce networked relationships among textual elements [Bingenheimer 2006], and Gould’s use of network tools for historical data analysis [Gould 2003]. This can also be seen in the TextArc project, which uses radial graphs to show contents and relationships between words in texts [Paley 2002], and Writing without Words [Posavec 2015], Posavec’s beautiful graphs of sentence structures and themes in Kerouac’s “On the Road”. Perhaps the most exciting and cognate study is Alexander’s work on the Oxford Historical Thesaurus and mapping sonic relationships in language over time [Alexander 2012]. Also, we find a movement to projects that use art to approach these problems, highlighting that aesthetics may be itself a critical tool. Installations such as TextRain [TextRain 2015], an interactive art piece where gallery goers interact with falling text, and Word Collider [WordCollider 2015], an artistic project modeled after pictures from CERN’s Large Hadron Collider, where parts of words when “smashed” together produce effects like those seen in images from particle accelerators.

The development of our technique was motivated by the fact that existing visualizations and text analysis tools, while usually aesthetic pleasing, cannot easily be used for the types of analyses expected or desirable for literary criticism or linguistics. Our Language DNA is an attempt to build a system that can handle multiple levels of information to display complex structural and content-related relationships within texts.

First, a visualization algorithm is needed to encode language such that we can create a space that is both consistent and reversible. In mathematical terms, this would be referred to as one-to-one and onto. The need for consistency and reversibility arises from the requirement in the analysis process of preserving the ability for human interpretation of the words. In order for the visualization to be “readable” by a critic, each word must be consistently mapped and that mapping must be reversible into the original work. Most existing visualization techniques distort the original texts without providing an avenue for reconstituting them. This is problematic when studying things like poetics where the spatial component of the text is integral to its meaning.

Second, it is important that the algorithm uses a plotting space that is infinite. Imagine that we were to approach the problem of metaphor. As an example, in theory we do not know if the chain of meaning created by metaphoric relations between words is finite. It stands to reason then, that without an understanding of the full requirements of a system that an infinite space is a safe decision. If we are to start to approach these types of questions our lack of knowledge should not limit the size of possibilities. Specifically, the critic must be able to analyze words and literature that are perhaps not known to the visualization designer. With our technique, we can encode anywhere from one letter to an entire language, to the entire literatures of one language, and even multiple languages, in a space where each point belongs to one individual piece of original information. The infinite space means that in creating “experiments” the literary critic is not limited to our present understanding of language.

The third requirement is the need for the ability to layer symbols in order to make comparisons. For instance, it should be possible to overlay a poem within the context of the entire language or other poetry. The need for layering arises from the analogic basis of most types of literary critical and linguistic inquiries. This should extend to any types of symbols, as comparisons are not always rooted in the Latin alphabet. Based on what we recognize as the possible requirements of such a system, we designed our Language DNA visualization with the three characteristics of consistency and reversibility, infinite plotting space, and layering in mind.

The mapping above describes how an arbitrary word can be mapped onto a number line, which already allows the visual mapping of words onto an axis in 1D space (similar to a lexicographical axis). Here, we describe a method, inspired by Cantor’s Diagonalization to map an individual word onto 2D space directly (and more generally onto n-dimensional space).

We can split the word in two by considering every other character, for instance InFoViS would become IFVS and noi. Thus, we can take the base 27 representation of the mapped word and create two dimensions as follows:

𝑓2:𝑊  →   (0,1)2
𝑓2(𝑤) = 𝑓2(𝑥1𝑥2 ... 𝑥2𝑛) = (𝑓(𝑥1𝑥3 ... 𝑥2𝑛-1) , (𝑓(𝑥2𝑥4 ... 𝑥2𝑛))

For example, if our word is “applesauce” (Fig. 3):

𝑓2 ("applesauce") = (0.𝑎𝑝𝑒𝑎𝑐, 0.𝑝𝑙𝑠𝑢𝑒)

which, in base 10 would be:

𝑓2 ("applesauce") = (0.0592410,0.6100587)

This mapping can easily be extended to n-dimensions by taking every nth character in the base 27 representation of (𝑤). f2 is also clearly a bijection, because every word can be split into alternating characters to generate two base 27 representations (one-to-one) and each pair of numbers between 0 and 1 can be converted to base 27 to recover the two parts of the word, which can then be reassembled (onto). Thus, every word in the English language can be mapped onto a 2D space using f2, and every 2D point can be mapped to a “word”, where a word is a sequence of possibly infinite letters which may well not have associated semantics. Note that this mapping does not account for things like homonyms, but with a simple addition to the mapping we could easily differentiate words for any number of their ontological characteristics.

Since whole natural languages are immense, it is important to discuss scale, both of what is being visualized and the size of the resulting visualization. We can base a visualization size calculation on the number of words being visualized, and then determine the length of a 1D L-DNA visualization that draws at a density of a single pixel for each word or unit (it is important to remember that these calculations are for orthography, they will change depending on the symbol system used). Two measures are needed to accomplish this: the smallest and the largest distance between two words. Since our algorithm already normalizes words in 1D to be between 0 and 1, we can assume that the difference between the largest and smallest words is approximately 1.0 (with the words ‘a’ and ‘zygote’, this is already correct to 1 decimal place). In our analysis of words from the Oxford English Dictionary (OED), the two closest words using our algorithm are “abandoner” and “abandoning”, with the first seven letters in common and the next being very close in the alphabet. The difference in values from our algorithm for these two words is:

0.abandoning27 - 0.abandoner27 = 1.37 x 10-1110

Thus, to present a number line from 0 to 1 with numbers only 1.37×10-11 apart represented as different pixels would require:

1.0 ÷ (1.37 x 10-11) = 73.1 billion pixels

Note that in 2D, our algorithm fairs far better. This same pair of words would be broken down into two pairs of coordinates:

(0.aadnn27, 0.bnoig27) and (0.aadnr27, 0.bnoe27)

which has at most four letters in common for each dimension and would require only:

1.0 ÷ (0.aadnr27 - 0.aadnn27) = 1.0 ÷ (2.79x10-7) = 3.6 million pixels

To put this into perspective, a 1D visualization using our algorithm would require the width of 38.1 million 1080p screens (1920 × 1080 pixels) placed side-by-side, and a 2D visualization would require 6.2 million 1080p screens arranged to form a rectangle.

Fig. 4 shows all 370,624 words parsed with criteria that eliminate diacritics and punctuation the Oxford English Dictionary[3] rendered using the algorithm described above (i.e., using the coordinates provided by f2. The result is a mapping that privileges the first two letters of each word. That is, the x-axis can be read as an alphabetical ordering of the first letters of words, and the y-axis can be read as an alphabetical ordering of the second letters. This makes the top left box “AA”, where you would find words such as “Aardvark” (note that few words in English begin with two A’s, which is why this box is quite sparse). This property is recursive, so that within each box, the third and fourth letters are similarly privileged. For example, in the “BA” box, there is an “NA” box, which has another “NA” box that contains the word “BANANA”. This initial visualization shows how we can start to understand where each word belongs within the 2d whole.

Because our algorithm privileges the spelling of words, this 2D representation can be thought of as a form of 2D orthography (specifically spelling rules). It is essentially a two-dimensional layout of alphabetical order. This version of L-DNA reveals a “bird’s-eye view” of the language that was not previously available to the literary critic, linguist, or lexicographer; a critic could previously flip through a dictionary’s pages or even a list of ordered English words, but this visualization instead provides a new 2d spatial location for each word in this dictionary.

Our second example compares multiple languages (English, French, German, and Spanish). Fig. 5 shows these four languages each represented in 1D on the 0 to 1 number line using our algorithm, stacked for comparison. Visual inspection reveals a similar sparseness in the ‘Q’ portion of the line for all languages (i.e., few words in any of these languages begin with ‘Q’ and any letter other than ‘U’), but additional sparseness in French, German, and Spanish exists near the end of the alphabet (‘W’, ‘X’, ‘Y’).

Fig. 6 also shows a side-by-side comparison of multiple languages in 2D, and Fig. 7 overlays these four languages. Fig. 8 shows a close-up of the AL region of the overlaid image. These side-by-side comparisons or overlays allow for elementary analogic comparisons and can be expanded on with more complex symbol encoding.

The above images were generated with the constraint that we only had access to open source dictionaries [Dict 2015] for languages other than English (for which we have university-wide access to the OED). The French (red) has 197,954 words, the English (blue) has 370,624 words, the German (yellow) 425,501 words, and the Spanish (green) has 160,442 words.

We chose the next example (Fig. 9), English phonemes, to demonstrate the robustness of the technique to arbitrary symbolic representations of language, and to create an analogue between the spellings of words and the sounds of words. The mapping is organized in like sound units: vowels (e.g., “AA, AE”), semivowels (e.g., “W,Y”), stops (e.g., “B,D,K”), affricates (e.g., “C, H, JH”), fricatives (e.g., “D, SH, V”), aspirates (e.g., “HH”), liquids (e.g., “L, R”), and nasals (e.g., “M, N, NG”).

By organizing words into phonemes, some interesting observations can be made. It appears clear that a portion of the phonemes are used primarily for the first syllable and another distinct set is used primarily for the second syllable. This can be observed through the densely populated top and right columns, with the majority of the bottom-left part of the image containing almost no English words. In addition, the top-right corner is mostly empty, with the exception of a few very dense groups, representing the few phonemes that are used for both the first and second syllables.

The final example that we created was to insert a single poem into the space that we created for English words and phonemes. This is a first step in being able to use these spaces for analogic comparisons.

Some interesting patterns can be observed in the poem through visualizing it in this manner. Firstly, in terms of orthography (Fig. 10) it becomes possible to identify visual rhymes by cluster groups within the image. In Fig. 11, this same phoneme visualization can be used to identify rhyming patterns within a poem. As the phonemes group together it is possible to see the types of sounds being repeated in the piece. Although this is easy to do with a 16 line sonnet, it becomes much more difficult with a poem of any significant length (e.g. Milton’s 10,000 + lines of verse in “Paradise Lost”) and this technique could help to highlight “macro-structures” in poetry. Each diagram below is laid out in two dimensions. This is a decision made during the encoding process and can be n-dimensional based on the amount of information you wish to build into the model. In this case we have chosen to show 2d representations for simplicity. In Figure 10 we present what we label our alphabetical order visualization where we represent words by their spellings. A visualization of spelling alone may not lead to many insights, but is useful for simple demonstrations. One area where this simple encoding could be used would be to visually compare irregular spellings in Elizabethan drama. It would provide quick visual access to the places in texts that differed and needed an editor’s attention. The real power in this method comes from being able to encode as many connections as desired. Work has been done in the digital humanities and computer science in the last few years in word embedding models and the consistency of our method could aid in the process of detecting connections in texts by using vectors. In figure 11, we have graphed phonemes in two dimensions. Any highlights that form vertical lines are showing alliteration in the poem. An example is in the line from Donne’s poem: “Or like a thiefe, which till deaths doome be read”. The words “deaths” and “doom” line up vertically to indicate alliteration in this particular encoding. If we wanted to visualize rhymes we would simply encode the phonemes in reverse, privileging the final phoneme and we would generate a similar graph. The n-dimensional nature of the models allows for as much or as little data coding as needed, including relations between words. The only limitations on the questions that can be asked are the imagination of the analyst.

We intentionally sought participants from a variety of disciplines. We had 14 participants which included 1 visual artist, 3 literary critics, 1 rhetorician, 2 digital media critics, 1 database programmer, 1 business analyst, 1 linguist, 2 interaction designers, 1 graphic designer, and 1 marketing specialist.

Each participant took part in a thirty-minute interview and was first shown L-DNA visualizations that we had intentionally left void of any legends or axis labels, so that we could ask questions about their initial interpretations. After showing the image in Fig. 4, the mapping of the OED in two dimensions, we asked what they thought the image might be. We then showed participants Fig. 7, four languages plotted in the same space, and the interviewer gave a thorough explanation of the how the algorithm works and what they were seeing. We took time to make sure they were comfortable with the explanation and asked about their understanding. We then showed them Fig. 8 to be able to further explain L-DNA and asked questions based on the participants’ understanding. We also asked participants to complete a post-interview questionnaire. Five questions were asked on a 5-point scale, with an opportunity to provide free form answers:

Q1. Once explained to what extent is the visualization readable?

Q2. Do you think the white space has meaning?

Q3. Is the white space necessary?

Q4. Does representing languages by colour and words as points work well?

Q5. Does this spatial representation of language trigger new ideas?

The following three questions asked for free form answers only:

Q6. What are your initial interpretations of this visualization?

Q7. Can you imagine a more suitable or readable structure?

Q8. Please provide any criticism you have about this visualization.

Interestingly, when shown the images without any labels, participants tended to engage in metaphoric comparisons of what they were seeing. The omission of a legend led each participant to find something that was cognitively analogous to what they were being shown. For example, some responses were:

“Is it zoomable? It has a DNA look or sort of ummm a matrix data flow and I feel the urge to zoom it. It looks like I am really far away from an unbound book, like how a book would be printed in sheets. It has that type of aesthetic.”

“Well, it reminds me of DNA, like a screening test, it also looks like a stamp, like someone has stamped something. It is a very tactile image, I want to touch it.”

“It’s kind of like DNA. Like, uh, people show these images that visualize DNA.”

In response to this process, five out of nineteen (26%) of our participants from varying backgrounds and fields, with no explanation of what they were looking at, described Fig. 4 as appearing like DNA — the inspiration for the name of our technique. This result also demonstrates the power of representation held within L-DNA. Some of the responses received from participants included references to stamped or fading paper, city grids, abstract art, and digital clock faces. Because we were trying to create a space that could handle an ontology of words, the DNA metaphor was highly applicable based on the implications of describing parts of a long chain of information.

In our interpretation of the study data, the questions about whitespace (Q2-Q3) produced perhaps the most interesting results. It was during this question that most of the participants began to hypothesize about the “space” in the languages they use every day. Essentially the parts of the visualization with an absence of marks inspired thought, because they were in stark contrast to the actual dots drawn on the screen. This result strongly indicates the analogic or comparative possibilities of this technique — the literary critic can begin to understand what makes a word English by recognizing what is not a word, or by investigating what words poets or writers use that push the boundaries of language. This result is encouraging for the domain-specific problems of literary criticism. One of our requirements is a space for analogues and with the whitespace in this simple mapping there is significant affect. The response of our participants to the relationship between the whitespace and the space occupied by words creates a relationship that gives insight into what sets of symbols we use and which we do not use in our language. The sheer size of the whitespace in comparison forces an understanding of how few of the possible arrangements of letters we actually use. With further work we think it is possible to show that more complex mappings will produce more complex analogues.

It was generally agreed upon that it was the relation of the empty space to the marked space that created meaning and inspired insight into what the visualizations were showing and what they could show.

“the white space gives you a sort of ground against which it makes sense... without it it might be even less evident”

It was in the white, or the lack of spellings, that our participants saw the potential for growth in the language, or commented on the enormous range of letter combinations that we do not use in English. This is encouraging because as we build “meaning” into these visualizations we expect the response to be comparative, and we expect new interpretations will result from these comparisons. Initial reasons were as follows:

“Every letter can start a word, but does that mean that there’s combinations of letters that don’t turn into words... you don’t have words that start with trsz is that why there would be space... Maybe it means that language is primitive, not as evolved as it could be.”

“I’m almost more interested in the dots in the white space... if there's one I want to know what that is...the outliers are more interesting.”

“What it does is show what the common letters are and common overlaps.”

For the researchers this result was surprising, but was explainable. Without “meaning” being built into this version we were simply showing a part of orthography (spelling) and, in this stripped down version of the possibilities of the space, it was the comparison between what was empty and what was marked that sparked the interest of our participants. This is the exact response we were interested in and we anticipate with more complex representations we will be able to see more complex analogies. Some of the responses to this whitespace analogy are as follows:

“I think the uniformity of the gaps are startling. It seems oddly uniform and consistent. I think that potentially symbolically the gaps can sort of be a formative quality of language and words will start to fill in those gaps.”

“Well, I guess it kind of goes to show how constrained we are in language, which shows how some things just are not possible with spelling, which is kind of cool. There is so much blank space particularly along certain lines, you get some sense that our alphabet constrains us, which is why we have poets.”

These responses were typical of all our participants and are very promising for future work.

When asked if the images they were seeing inspired any new thought processes, they proved to be exciting to our participants and the answers that follow show the breadth of their thinking:

“You could map to any narrative, I would like to see this map out, A Tale of Two Cities, it looks like a genetic footprint, like a genetic phonetics, you get to see this formal genetic blueprint, it is more like an autopsy of the text.”

“Yes, from a design and a fine art point of view. I think it is, again, if you look at it as a design issue does it solve any problems, not yet, but then again you are deconstructing something that already has that problem solved so you are raising questions instead of solving problems and you are raising interesting and meaningful questions.”

“Well, I mean my initial instinct is YES (emphasis given by the participant), but I am not sure what that would be yet, I think it could lead to a lot of productive conversations about how language operates or the way someone creatively uses language.”

“These are important questions. Literature departments have always performed as if they were in the shadows of the sciences and…[i]t seems to me that this kind of work, although seemingly scientific, should be the domain of literature department. It would be very important work for us to do.”

Interestingly the 1 person out of 14 who said no to being inspired (included below) touched directly on the fact that the simplicity of our representation of orthography was limiting but suggested in his negative response that it could be interesting for literary criticism if we could find a way to include meaning, which is fruitful ground for future work and possible with the three criteria we laid out above for the development of the technique:

“I’m not sure it’s useful for literary theory or criticism because it seems to explicitly set aside the question of meaning in favor of orthography”

“Oh yea, um this is only for spellings well things get really interesting when you get into phrases, rhetorical aspects of language organization, the whole question of nuance which we like to think we are studying as literary scholars. Once you get into these things and away from mere spelling and into points that have meanings you can start to clump together interpretation to different meanings.”

Our participants were in some cases critical of the design of L-DNA. In particular, the most common criticism centered on making the technique interactive, which is a clear (and previously planned) next step in our iterative design process. Another criticism was that the encoding we used was arbitrary, and its meaning was not immediately clear:

“The mechanism by which you map words to spatial locations, it’s kind of arbitrary (maybe that’s not the right word). Mostly you see the first two letters, and when you zoom in you’re seeing the space of subsequent letters. I guess what this mapping is lacking is the Meaning of words, or the semantics. They end up being distances from one another but … the distance between related concepts, synonyms. I’m not necessarily saying this is a bad visualization because of that. It just doesn’t encode the meaning of words. Which is much harder. But, this is a legitimate way of putting words in a consistent space.”

“The way that it is right now, that it’s static, it’s getting in the way of itself. It demands explanation. It needs to be paired up with something very practical, like reading a word or reading a sentence and how that gets paired up in the system.”

We see this interactivity as the next steps in developing an application for these types of interactions and it is in that interactivity that the objections to the arbitrariness of the design will be addressed. We hypothesize that being able to investigate the space dynamically, and by defining other symbol systems to encode, the literary critic will be able to explore the types of meaning and associations being looked for by our participants.

In response to the discussion with participants about the interest in whitespace and the lack of density in certain regions, we produced a density and inverted density map to highlight the white spaces, shown in Fig. 12 and Fig. 13.

This was in direct response to comments from our participants such as:

“Yeah. But, one thing I was going to say… the UNIFORM inclusion of the whitespace is interesting. But I wonder if there’s a better representation of density and overlap.”

In this iteration of our design, instead of rendering each word in the language we instead cluster groups of words into the boxes representing pairs of words (“AA”, “AB”, etc.), and render the box using transparency that corresponds to the density of words therein. The inverted version of this mapping highlights all of the non-English words, which were clearly of interest to participants.

We have discussed how the blank spaces are compelling and how the absence of words in these spaces generally seems to intrigue people.

It is interesting to note that this space is and has been filled in many interesting ways. For example, E. E. Cummings poem ygUDuh does not contain a single English word, yet it can be read as English, where the “words” take on the sonic characteristics of English when read aloud. For example, the first few lines of the poem are:

That when read allowed becomes a phonetic map for a type of early 20th century urban slang exemplified by the poem:

In Fig. 14 we have plotted ygUDuh overlaid on the OED grid. Note how these “non words” exist largely on the edge of the word spaces and the white spaces. This may be because, while they are not English words — hence the white space proximity — they have similar vowel and consonant structures to English words. By seeing these words overlaid on the whole language, we can see a visual representation of Cummins’ craft, of the attempt to make non English words that sound like English when read aloud. The fact that all of the non-words are situated on the edges of heavily populated space tells us that these arrangements of letters that we try to make into words when reading the poem are “closer” to English words than we think, at least in terms of spelling.

Another example where new types of “words” or at least English communications are evolving is in instant messaging, text messaging, and social media. It seems that for ease and speed, we can give up many letters — chiefly the vowels — in words and still retain meaning. Fig. 15 shows MSN “words” overlaid on the OED visualization. It is interesting to note that many of the new “words” (marked with red dots), fall in the spaces where very little or no words exist. This demonstrates that even in a type of shorthand, like the one used in instant messaging (e.g., “btw”, “lol”, “ttyl”, etc.) that many of the newly created words are spelled with letter combinations that simply don’t exist in the language. This is partially a result of the volume of acronyms used in instant messaging but it becomes obvious by “reading” the image that many of the words used fall on the top line of each row suggesting (such as with row A, and row I) that many of these “words” and acronyms begin with those letters. In this way our technique produces visuals that allow us to ask further questions about the organization of our data.

We have also begun to integrate interactive elements into our visualization, some of which were planned prior to our qualitative study, and some of which were inspired by our results. In particular, we have already created a version of L-DNA which incorporates a brushing technique that presents the “words beneath the cursor”, both when dragging across words and when dragging across the whitespace. We have also created a version of the density map (Section 9.1) that allows zooming into the recursive letter pairs. For example, it is possible to click or tap on the “BA” square, then the “NA” square, then another “NA” square to then see the word “BANANA” as shown in the static image of figure 3.