DHQ: Digital Humanities Quarterly
2009
Volume 3 Number 3
2009 3.3  |  XML |  Discuss ( Comments )

Picture Problems: X-Editing Images 1992-2010

Abstract

After centuries of image deprivation, we now bathe in a sea of pictures, most of them digitized at some stage. In the 1990s, as humanists began to sense the advantages of networked computing on the web, they conceived major new editorial projects that would depend to an extraordinary degree upon the documentary power of pictures. Despite evident progress in devising sturdy and responsive standards, images, and tools, stubborn problems persist in several key areas that are explored here through an overview of issues that arise as the William Blake Archive acquires images, prepares them for reproduction, and makes them available for manipulation by its users. Editing electronic images in so unsettled and unsettling an environment generates the provisional success — weak success — that is utterly characteristic of X-editing, electronic scholarly editing in our time. Our dependence on current technology and the expertise of others is not a remediable condition. We must play the game as it presents itself, making the compromises that are necessary, and move ahead.

After centuries of image deprivation, we now bathe in a sea of pictures, most of them digitized at some stage. In the 1990s, as humanists began to sense the advantages of networked computing on the web, they conceived major new editorial projects that would depend to an extraordinary degree upon the documentary power of pictures. That move seemed logical enough at the time and, indeed, at the prime spots there were, even then, white beaches and pristine visual fluids: artifacts so scrupulously reproduced that by 1993 — remember, this was in the dark days when a "high-end personal computer" was a "Macintosh Quadra, or an IBM-compatible 486 running Windows"  [Robinson 1993, 31] — Peter Robinson was already marveling at the level of detail captured by the "archival" (versus "transmissive") images of the VASARI and MARC projects [Robinson 1993, 77, 91–2]. Such reports of delicious archival images digitized to within an inch of their lives, like Richard Preston’s in The New Yorker about the team of conservators, photographers, and mathematicians who digitized the Metropolitan Museum’s "Unicorn in Captivity"; or Stephen Mihm’s report in the New York Times Magazine about North Korean counterfeiters who use digital technology to master the visual complications of the new U.S. supernotes; or even our own hours squandered on the global fascinations of Google Earth or on the celebrity faces of Riya can make our hungry eyes glisten like Ray Kurzweil’s in a futurist rapture. Perhaps all things are possible, almost already.
But it would be a mistake to suppose that pristine beaches and pure water are the norm. Nearly fifteen years since Robinson’s report, day-to-day editorial reality on the picture front remains rocky and polluted. For texts there is, and has long been, the Text Encoding Initiative to offer reassuringly systematic recipes — but no counterpart in an Object Encoding Initiative. To give you some sense of what I mean, I want to take up an indicative selection of picture issues as they arise for us at the William Blake Archive. We acquire images, prepare them for reproduction, and make them available for manipulation by our users. Each little story that follows will be an installment in a larger narrative of modest success mixed with failure and uncertainty — so far neither indubitable success nor abject failure but always considerable uncertainty. This is the dim, disturbing landscape that I have taken to calling, inelegantly, X-editing.

Acquisition

In the past there was print, and a modest few rules of thumb that governed the practice of many public and private collections when it came to the reproduction of their property, usually as monochrome photographs that were converted into a highly constrained number of mediocre halftones. But in our post-print era, intellectual property requirements are a highly charged package of changing laws and anxious, opportunistic, tentative interpretations.[1] We are surrounded by blockbusting digital book projects and digital photo and video collections of wondrous scope, but even industry giants like Google are hindered from doing what they want to do by a snarl of intellectual property restrictions. The constraints when it comes to valuable images that live outside of books — paintings, for instance — are in many cases even greater. Where does that leave us humanities-computing smallfry?
In the early days — say 1993-1995 — of trying to clarify our own foggy founding notion of somehow harnessing the power of computers (to overcome some of the obstructions that had prevented Blake scholars from seeing the full range of Blake’s artistic output), we landed eventually on the slightly more mature idea of a William Blake Archive. We — co-editors Robert N. Essick, Joseph Viscomi, and I — knew simply that we had to start somewhere, so we began on the strength of cooperation from only three collections and some beautiful new photography left over from a just-completed six-volume print facsimile project [Bindman 1991-95]. At the time, most art collections knew just enough to be scared of the word digital and threatened by the global exposure on this web they had heard about. We used what little influence we could muster in the face of a lot of inertia and pockets of strong resistance. Gradually, experience taught us to conserve our energy by focusing on targets of opportunity and putting the hardest cases to one side. We learned to count on curators, whose sense of participation in the life of scholarship helps them understand what we’re up to. Five years later we had, I suspect, perhaps ten collections willing to contribute their images. We sensed a critical mass of participation that was shifting the balance of the picture owner’s decision-making from why? to why not? Today we have almost thirty contributors, including the world’s largest and best in our field ("Contributing Collections," http://www.blakearchive.org/​blake/​public/​institutions.html).[2] Awkwardly, we must always ask them to make very large special exceptions to the stringent rules that institutions have devised in the past decade to control reproduction of their images on the web. Even so, we have been able to maintain our original principle: pay for photography but not for the privilege of reproduction and offer everything free on the web. We could not have continued otherwise.
From one perspective this coalition of the willing is a powerful, even inspiring, and perhaps unprecedented lesson in the collective power of international scholarly responsibility. Most collections understand the Internet better now than they once did. Less optimistically, however, our coalition is a house of cards propped on a rickety network of formal and informal understandings — much less secure than, say, the scaffolding of intellectual property law. To carry the day we bank on a combination of intensive diplomacy, collector guilt, precedent (the critical-mass factor), and performance. But in the worst of times we may be putting our faith in the wrong place. The coalition may deteriorate. We don’t know, we don’t think we can know. We don’t have the knowledge or money to pursue legal issues. Here as in so much else, we persist in the vague hope that the Archive’s community will endure long enough to provide a useful bridge to the next great editorial settlement, whatever it may be.

Reproduction: Ours

The Archive is known for its meticulous care of images and adherence to a strict set of established guidelines. Perhaps the most significant features of the Archive are its uniformity and its conformance to explicitly stated standards ("Editorial Principles: Methodology and Standards").[3] In the beginning we determined that each image would be extracted from a source image produced in excellent conditions, either directly from a high resolution digital image file or indirectly from a large film transparency (4x5 or 8x10 inches, with color bars and gray scales) that we then scan. No batch processing: as we worked out our initial division of labor, Joseph Viscomi took on the responsibility of overseeing the color correction of every image on calibrated equipment. The technical details of this journey from the picture owner to the Archive would accompany each image (see the Info button on the bar beneath every image). Users could calibrate their systems accordingly. Under optimal conditions, the results would be — and have been — better overall than the best printed reproductions (see Image Enlargement from the Show Me pulldown menu to the left beneath every image).
These days we scan transparencies on far better scanners, and the best museum photographic services can now produce far more accurate and uniform image files from their own digital cameras than they could a decade ago. But we still correct images one by one. And they must still be compressed for transmission over the web, despite greater bandwidth and faster machines. And, in a world where digital technology seems to advance far more rapidly than many of us can adjust to it, the compression format of choice is still JPEG, which was a real step forward when the standard was first fixed...in 1992.
The JPEG image-compression algorithm is ubiquitous. We’ve been using it for over ten years to create our thousands of images in pairs: a lower resolution 100dpi "inline" image for main pages (our Object View Pages, that is, all pages that reproduce an image that Blake created — whether a print, painting, drawing, or manuscript) and, two clicks behind it, a 300dpi enlargement for detail junkies. (In normal circumstances, most collections limit the reproduction of their images to 72dpi for the web.)
But poor, reliable, "lossy" [4] old JPEG is tuned to the requirements of average images from the natural world, which are dominated by tone and color, not by "edges," which are associated with "graphics," as distinct from "photographs" on the one hand and "texts" on the other. For capturing the natural world as we perceive it most of the time, this calibration is not bad. But JPEG groups pixels to render lines far less clearly and subtly than it can render colors and tones.
Blake, on the other hand, was devoted to the idea of artistic lines and convinced that "Natural Objects always did & now do Weaken deaden & obliterate Imagination in Me Wordsworth must know that what he Writes Valuable is Not to be found in Nature"  (annotations to Wordsworth’s Poems, E 665). As one might expect from the hand of a trained engraver, Blake’s art is full of strong edges — including the sharp, rich ones produced by engraving tools and etching chemicals on metal — that then often structure complex overlays of color and tone. For JPEG this is the worst of all possible pictorial worlds. So we need a compression algorithm that can capture edges as well as it captures tones and colors. That, as it turns out, is still a tall order.
Our best hope was the long-anticipated JPEG 2000.[5] JPEG 2000 can capture the full spectrum of graphic and pictorial elements a little better than JPEG, but its chief advantage, its basis in wavelet compression, gives it many other advantages that conserve memory and improve speed, stability, compression, scalability, and editability, while giving users more control through "progressive downloads," which ingeniously draw, from a single image file, images at selected levels of quality from faster and lower to slower and higher. With JPEG 2000 we would no longer have to produce image pairs at all; one file per image would suffice for everything, including details.
Hence we welcomed the chance to participate in clever experiments with JPEG 2000 that Vladimir Misic conducted for his dissertation in electrical and computer engineering at the University of Rochester. Using what was then the latest draft of the core code of JPEG 2000 and assembling a group of problematic Blake images, he devised a system that separated each image into its elements, processed the tone and color with JPEG 2000 and its edges with MRC (Multi-Raster Content) technology — developed by Xerox for sophisticated processing of graphic elements in fax documents — and then recombined the result for display in a JPEG 2000 viewer plug-in [Misic 2003]; [Misic and Kraus 2002]; [Misic, Buckley and Parker 2002].
At the time — 2002-03 — JPEG 2000 seemed just over the horizon. We planned to mount a JPEG 2000 demonstration on the Archive site. But imaging on the web then stalled at JPEG. The latest news is mixed: by some accounts, JPEG 2000 is held back by submarine patent issues and low browser support.[6] I’ve heard more optimistic news to suggest that the new (though aging) algorithm is slowly gaining ground in the cultural heritage community. The National Archives of Japan, for example, has experimented with JPEG 2000 and its film/video counterpart, Motion JPEG 2000. Other parts of the standard have been published as ISO/IEC standards in the years since 2000 (see http://www.jpeg.org/jpeg2000 and the relevant Wikipedia entries for further information). But for now, in practice we seem to be stuck with JPEG and an ever-swelling bank of several thousand pictures and manuscripts full of tricky tones, colors, and edges. That legacy becomes a drag on any eventual change to a better system.

Reproduction: Yours

But what we supply, fine JPEG images in pairs, is not, in any case, what our users receive. Along a perilous chain of transmission lie a host of image robbers: service providers, operating systems, browsers, video cards, display settings, displays, and the user’s own highly variable eyes and brain. Restricted bandwidth squeezes file size, while humanists hang further back on the technology curve than scientists and thirteen-year olds. Color control across systems has improved, driven by such powerful forces as digital cameras and web retailing — will that shirt I’m ordering look cool with my new jeans? (Try, for example, the compare function on the North Face site: http://www.thenorthface.com/​na/​gear-shop-category-2.html) Even so, color control is far from perfect, and many users still have slow connections and bad monitors with who-knows-what settings. We re-learn this lesson every time we proofread our own forthcoming publications on computers spread across the country. We may try to reach a collaborative decision about one of Blake’s etched, amorphous punctuation marks: comma, period, exclamation point, question mark? Tiny differences make all the difference as these squirrelly marks morph from instance to instance, copy to copy. Or we try to decide whether Joe Viscomi should readjust the "paper tone," the color of the paper on which Blake printed or painted or drew his images — one of the most telling colors in a picture, and often starkly wrong in otherwise impeccably printed images — to reach a tolerable compromise among our own few machines, which may be showing yellowish, brownish, greenish tan, and ivory, depending on the machine. The pull here away from Joe’s carefully controlled results and down toward the lowest common denominator becomes instantly obvious. Control of color, tone, and texture could be far better.

Manipulations

Blake Archive images arrive on the desktop in the embrace of a highly manipulative environment that allows users to use and abuse their pictures according to their scholarly needs. They can zoom in and out. With a click or two they can adjust images to actual physical size or something handier. They can read a transcription of any verbiage we have been able to extract from the image. With two clicks they can easily compare versions of the "same" work — any plate from (now eleven) copies of Songs of Innocence and of Experience, for instance. They can read elaborately detailed descriptions of the pictorial content of the image. If they question our account, they can check it themselves against the high-resolution enlargements. Blake may not have liked the way we’ve buried his works in a kind of poke-and-probe forensic lab, but he had his dreams, and we have ours.
However, our dreams of seamless, fast, sophisticated scrutiny hide hard realities. The lower-resolution 100dpi OVP image quickly pixilates when zoomed, forcing the user to load the 300dpi enlargement. The ImageSizer, which allows a user to calibrate the scale at which images are delivered to the desktop, can’t cope with Blake’s largest images. The Compare function is rigid, creating a lot of cannots: the user cannot resize images within it nor choose to compare images that we haven’t already chosen. Compare was devised to juxtapose multiple "copies" of the "same" illuminated books. Anywhere outside that category — watercolor drawings, paintings, engravings, printed works — we have to hard-code the application to tell it what to include in a comparison. The kinds of comparisons a scholar is likely to want are far beyond the capacities of our little Compare window.
Our eyes are on the Virtual Lightbox, a clever Java application and applet developed at the University of Kentucky and the University of Maryland by the Maryland Institute for Technology in the Humanities and the Human-Computer Interaction Lab. The Lightbox promises much greater flexibility to users with less frontloading at our end http://www.mith2.umd.edu/​research/​?id=12. Our project manager, Will Shaw at the University of North Carolina, has extensively revised the code for Lightbox and successfully installed a test version in our Work in Progress site. Add: And there is more good news in a function we have labeled "Related Works," which will allow us to connect objects and sequences of objects in complex and sophisticated ways. Soon we expect to incorporate the Lightbox and Related Works in a redesigned version of the Blake Archive.
The biggest drag on our imaging system, however, may come from Java in general and Inote in particular. All our OVP images are delivered to the page in a Java environment. Images that would otherwise take almost no time to arrive on the desktop take several seconds in Java time. The performance of Java has markedly improved, though the various claims and counter-claims for its speed are hard to sort out. But the "wait for me, I’m starting up" icon — that cup of coffee with a wisp of steam (http://java.com/en/download/index.jsp) — will become all too familiar to any regular user of the Blake Archive (Wikipedia, "Java [programming language]," "Java Performance," and "Criticism of Java," accessed 3 Jan. 2008). Worse, Inote, an innovative if confusing little image-annotation program when it was written at the Institute of Advanced Technology in the Humanities — the original home of the Archive — soon after the invention of Java (Java 1.0 1995, according to Wikipedia, "Java"), is a product of the primitive Java code of the 1990s, an era ago in computing time, as is our second Java application, ImageSizer. Early Java implementations slowed things down. In any case, there are no plans at present to rebuild Inote, though Will Shaw has made improvements. So for the time being our images have their feet a bit mired in it.

The Search

In her memoir, Julia Child writes that she "began to suspect that French bread was the recipe I worked hardest on that the fewest people bothered to try"  [Child 2006, 183–84]. For the Blake Archive that recipe would be Image Search, probably the least used and appreciated feature in the Archive but the one that has devoured the most editorial attention per image. Image Search should be the star among all our imaging features. In a good mood we think it’s both the smartest and the dumbest feature we’ve concocted. It allows users to search the contents of images by chaining up to nineteen keywords, which, in combination, provide access to those elaborately detailed descriptions of images that I’ve mentioned, anchored to one or more quadrants of the image itself — with the description as annotation.
Why we did we bother to invent an image searching tool? Simply to help restore the palpable imbalance — which we have endured in the print environment for centuries — between our sophisticated means of access to texts and our very crude and rudimentary access to the contents of images. Search 10,000 text files? No problem. Search 10,000 images at a level deeper than artist, title, general subject, medium, and location — a huge and heretofore unsolved problem. Or, we wondered back in 1993, did a system for doing that already exist?
In the early stages of the Blake Archive we got word of Iconclass, which seemed to pop up as a spoiler on grant applications (why aren’t these Blake people using Iconclass?). We soon discovered that Iconclass was freely traded as a label — it seemed to name the image-searching game for art historians, but we’d never heard of it. Humbled, we investigated and found many signs pointing to Iconclass, but the closer we got, the further away it seemed. We discovered that, in art history, 20th-century attempts to make images as searchable as texts had been dominated by efforts to solve this problem by textual means that had eventuated in Iconclass, whose noble lineage included the Index to Christian Art. In the library we discovered a visionary reference work designed to make all images in the world searchable by their iconographic details. Begun during World War II by a Dutch art historian and librarian, Henri van de Waal (1910-1972), who spent the rest of his life on it, the first full published list of verbal details and numeric codes ran to seventeen fat printed volumes when completed by van de Waal’s assistants and successors in 1985.
By the mid-1990s the Iconclass camp promised electronic tools that would ease the pain of assigning elaborately hierarchical alphanumeric codes to actual images (the Libertas browser represents the current state of play: http://www.iconclass.nl/​libertas/​ic?style=index.xsl). So we looked around for existing applications and realized that the user base for purposes like ours is miniscule. Some cataloguers had used a handful of top level descriptors, and a very few had gone deeper (see the list at http://www.iconclass.nl/​libertas/​aboutbb.html#image). But the full potential of Iconclass was untapped.[7] So was it a mirage, or a giant sleeping beauty?
We still don’t know, but at the time we paused over the huge form of Iconclass long enough to consider the consequences of kissing it. We certainly shared a major goal: objective description of images as distinct from the "interpretive" aims that had dominated image "description" in Blake scholarship.[8] Methods of study for Blake’s nonstandard images have tended toward rough and ready combinations of the art historical and the literary. Critics had often leapt past what is in a picture in order to say what the picture means — understandably, because the meaning of Blake’s work is so elusive. One of the hardest writers in English, his pictures make his texts more difficult rather than less, and his texts have comparable effects on his pictures. He had a special genius for creating obscure combinations that promise to deliver, ultimately, a kind of total meaning — and an equal genius for keeping the meaning always at bay. The difficulties, instead of turning off the audience, drive its most dedicated members toward ultimate questions about the very nature of words and pictures.
But we wanted to regress to a system that is primarily a means rather than an end, to provide tools that could make basic identifications possible: to find a picture by its content (a bearded old man with a walking stick that I saw on a CD?), to figure out what something is (sheep or dog? sky, water, or dirt?), and, serving the researcher’s needs for comparison and contrast, to find out if there is other similar content elsewhere in Blake’s work. What questions usually also extend to what is happening and where: standing or walking, in London near St. Paul’s or on the Lake of Udan-Adan? Answering such questions can be hard: is that globe a sun, and is it rising or setting? Satisfactory preliminary answers can be: a human couple, perhaps identifiable with character names, embracing on top of the Lilly of Havilah, or in Golgoonooza, or hell, or on the London streets, or — are those eighteenth-century-style druids strolling among the trilithons of...Stonehenge or Avebury or some invented place? Once you start playing this game, you quickly see that other details come flooding in: how are the characters arranged, what are their head positions, is their hair long or short, curly or straight, with or without beards. What are those objects they are holding? Are they standing on…are those white rocks the cliffs of Dover? Is the character with wings an angel, a moth, or one of Blake’s Spectres? Is character X offering help to character Y or threatening her? Are her eyes open or closed? Is that a (naked) man or a woman (turned away from the viewer) standing beside a dog or a sheep?
As entangled as the spiraling processes of identification and interpretation are, and for all the hermeneutic loops that entertain academic minds, we didn’t seriously doubt our ability to make rough but useful distinctions between descriptions and interpretations that would suffice for making the descriptions and keywords that would support Image Search. Using visual and literary cues, we could make educated guesses in descriptive sentences and tie those to keywords that a user could select from a list of search terms. But we would not allow ourselves to go the next step and create a running commentary, a coherent interpretation of the meaning of these elements in the artistic work. We would therefore not feel any obligation to, let’s say, tie our descriptions of individual images in Blake’s Book of Urizen to any overall narrative schema. As a matter of policy, we would be conservative even in our identifications: if there were a fairly persuasive bit of nearby textual evidence for identifying a nude male with a big hammer as Blake’s hero Los, we would venture that identification and give the evidence. If not, then we’d stick with "nude," "male," "curly hair," "blacksmith," "hammer," "standing," "facing left," and so on.
We had seen that Iconclass could handle a myriad of minute details. Its structure may have, as one of van de Waal’s assistants wrote in a posthumous tribute to the master, "the simplicity of genius"  [Fuchs 1972-3, 7]. But our stronger impression was that the system’s universalistic ambitions could pose a big problem for any particular set of images with rare or unique features. Though the Iconclass system eventually spread far beyond its roots in medieval and Renaissance iconography, to encompass, at least in theory, all images, it adapts most readily to typical images with typical elements employed by a recognizable school or era of image production. Blake’s images are so Blakean, so typical in confusing ways, so idiosyncratic in others, so literary and yet so visual, driven so fundamentally by metaphorical processes, and we ourselves had such defined aims, we concluded that the problems of adjusting Iconclass to our purposes would not be worth the time and effort.
We would instead make our Image Search tool an inside job — custom tailored to Blake’s work and to confirmed scholarly habit. Rather than start with a preliminary list of keywords, we grew the list of terms image by image: we needed "male" for the first plate of the first work we marked up, Blake’s Book of Thel, but we didn’t need "Job’s wife" until several years later, when Bob Essick was marking up Blake’s Illustrations to the Book of Job. Every new group of images would expand the "book of terms" just a bit. We would link our descriptions of details, composed in normal sentences, to those keywords, and the keywords to the quadrants of a visual grid laid over the image in Inote. A user would search the keywords — either one by one or several at a time — and that would lead to a list of hits, which would lead in turn to descriptions, which would lead to the images, which would display the descriptions as verbal annotations to the visual evidence.
As the list of keywords grew, it had to be organized. Like everything else about the Image Search tool, the categories that have emerged are rudimentary but fairly intuitive. They more or less track the ancient hierarchy of the great chain of being or the game of Twenty Questions. Within those divisions, they are alphabetical. The more terms we stuff in, inevitably, the more puzzling and daunting the interface becomes to new users — if there are any.
As a proof of concept, we thought our experiment worked — to alleviate, if not solve, a stubborn picture problem. We were excited to be able to find what we had tagged in the images, excited to do something unprecedented in Blake scholarship, and excited to see our image search engine grow along with the Archive, so we kept at it despite the labor involved.[9] As it stands today, Image Search is no great achievement in software design, nor the next big thing in imaging, even in the village of humanities computing, and it remains raw, nonstandard, unlovely, and trapped by its design in its own local neighborhood. But it is, finally, quite useful.
The dependence of Image Search on in-house skill and judgment is its Achilles heel. But we faithfully kept it up to date until we came to a crossroads where new editions that were in every other respect ready for the Archive outran our ability to generate the descriptions and keywords for Image Search, and we faced a long, painful conversion of the site from SGML to XML. So we fenced off the category that Blake scholars call the illuminated books. We have maintained Image Search in that one large category only. For all others, including drawings, paintings, prints, manuscripts, and printed books, we created Preview, a stopgap label for new works that incorporates every Archive feature except Image Search. The word preview betrays our frustration and ambivalence. Would there be more Image Search to come? We weren’t able to commit; but we weren’t able to just sink the thing. So the resort to Preview felt a bit like failure, caused by our devotion to what was perhaps a flawed concept. And we knew that most users would neither notice nor care.
Then a series of major developments put us back on track. Blake Archive XML dawned in 2006. After more than a decade of close collaboration with the Institute of Advanced Technology in the Humanities at the University of Virginia, we moved to the University of North Carolina at Chapel Hill, where Joe Viscomi had assembled a superlative new team, and we began to publish our big backlog of finished work and launch major new projects.
Meanwhile, out of the blue, came an inquiry from a highly respected Blake scholar and colleague, Alexander (Sandy) Gourlay, that brought Image Search back into focus: would we be interested in publishing his "commentary...as compact as you like, primarily objective/descriptive"  (email 11 May 2006) on Blake’s 537 watercolor illustrations to Edward Young’s once-popular poem, Night Thoughts?
We were intrigued. The designs are striking and significant; the Night Thoughts project was an artistic laboratory for Blake. We had already published the few engravings that he himself executed for the one printed volume that appeared before the project was aborted by the publisher, and those 537 preliminary watercolors were in our plans. We had already photographed the entire Blake collection — the world’s largest — of the British Museum and had permission to publish it.
So for us Sandy’s question came down to this: could we teach our system to a visiting Blake expert who would apply it to a huge batch of pictures that were outside the illuminated-books category? We supplied a simple template and a review of Image Search. Sandy’s first tries ran aground in ways that were familiar to those of us who had suffered through the same difficulties: too many new terms, too little consistency, and too much effort to capture meaning. He complained that writing the descriptions left him "thoroughly depressed about prospects...It’s partially the ratio of labor to useful productivity" and partly the sense that Blake, in pictures, and Young, in words, lived in "two different universes...each with its own language...very different from the illuminated books."
We felt his pain. We knew, as Bob Essick wrote, that "trial and error is the best way to learn this sort of thing"  (email, Essick to Eaves and Viscomi, 13 May 2006). So we responded with empathy: "Everything about the terms and the descriptions feels arbitrary...and the sea of markup-able elements seems too deep to fathom." Ultimately Sandy decided to create a separate web site devoted to his project. We were sorry to lose his expertise, but we understood the frustration of a scholar-critic with something to say about the meaning of Night Thoughts. Saying it — or rather, not saying it — in Image Search would be like running a marathon in a prison-yard maze.
But, perhaps perversely, the failed little experiment inspired us. We continued our discussion. Was Image Search worth salvaging? Yes: the experiment reminded us that it worked pretty well to meet the limited demands we had in mind. Perhaps it even met the "good enough" standard that engineers sometimes apply. Image Search could accommodate any of Blake’s works, even those as different from the illuminated books as his Night Thoughts designs. So was it time to bring Image Search off the sidelines? Yes. Rather than flee our own beast, we decided to kiss it and revive it.
Was it time to generate new descriptions for the several copies of as-yet unpublished illuminated books? Yes, as a high priority, to get us back in the groove. Members of the University of North Carolina team have begun with our backlog of illuminated books, because new learners can use existing templates to guide them. But would we extend Image Search to works beyond the illuminated-books category and eliminate the Preview stamp? Yes, gradually. And, most formidably, was it time to revisit, refine, and reform the system itself — exploiting, for example, hundreds of working notes we had already made? Yes. Eventually, we hope to see Preview reserved for a very few special cases.
We may live to regret our decision. Image Search is parochial, but it is fairly self-consistent and built on standard bones — imaging bones, markup bones — in a project that has been highly aware of standards and best practices when they exist and compliant with them whenever compliance works for us. We hope to fulfill our original aims of creating an image search engine that will sustain advanced research. But here as elsewhere we proceed as shopworn veterans who have learned one more hard lesson about pictures and their problems.

Computer Vision

Beyond even our low and localized aspirations for the near term, what might we consider? Internet pioneer and Google vice-president Vint Cerf recently imagined the arrival of an exciting moment less than a decade away — September 10, 2017 — when, "In a breakthrough for Web searchers everywhere, new indexing tools have been announced that allow images, video, audio clips, and other nontextual content to become part of the organized information of the World Wide Web"  [Cerf 2008]. In the scholarly world even the rehabilitation and revitalization of Iconclass seems not utterly out of the question; theoretically, it would boost enormously the value of big academic imaging databases such as ARTstor http://www.artstor.org/index.shtml, which has again highlighted the challenges of gaining access to the content of images on a par with the access we expect for texts.[10]
And, of course, there is always automated searching. Computers have inspired the thought that image searching could be automated in a way that would make it easy for users of, say, Corbis http://pro.corbis.com, the Bettmann Archive http://www.corbis.com/BettMann100/Archive/BettmannArchive.asp, or AP Images http://apimages.ap.org/ to locate anything they might be looking for in any of the millions of images in their commercial databanks. Research proceeds apace on multiple fronts with enormous stakes. The co-founder of Palm is now the co-founder of Numenta http://www.numenta.com/, one of whose pilot projects calls for it to "learn" the properties of a large set of images and search their contents automatically. There are bits of promising (if proprietary) news from other quarters — demonstrated by Riya http://www.riya.com/, for example, Blinkx http://www.blinkx.com/, and Ask http://www.ask.com/. Google recently filed a patent application for a technology that would recognize texts within images (street and store names on maps, for instance) — with potential application for searches in Google’s Book Search and Street View [Claburn 2008]. (Recognizing store names and printed text is a first step toward the far greater challenge of recognizing handwritten text in manuscripts.) The 2009 update of Apple’s iPhoto application includes face recognition so that users can search their photos by the faces in them as well as by date, event, and so on. The uses of image searching in national security operations involving vast, swift data streams are many. The potential commercial value of success is equally vast, and no doubt progress in solving the fundamental problems will come. How computing humanists might eventually inherit the results of success is unclear.
So far we have seen no evidence that any automatic system in the foreseeable future will be able to shoulder any of the work that our modest little Image Search does. But there is yet another kind of potential in visual searches. Once, as I was trying to explain to a member of the JPEG Consortium why in the world such an elaborate array of keywords is required to respond to the questions about Blake’s pictures that most scholars ask, he broke in: "Yes, but do they ever want the answer to another question: 'Is there anything else in this group of pictures like this?' " He made a quadrangle with his fingers. Not "is this a gowned or nude male or female standing or walking with arms raised in Golgonooza, Stonehenge, or London?," but "is this [visual area] on this painted or printed or drawn surface 'like' [any other image areas]?" Is this [combination of texture, color, light, and shade] an instance of a technique that Blake used elsewhere or only here? Do these colors, or these crosshatching patterns, appear frequently? In that arena the potential for automatic searching in the near future or perhaps even now seems far easier to envision.
Meanwhile there is no reason to stop worrying about the problems — the smaller problems of digital humanities projects, the larger problems of scholarly communication as a whole — that pictures bring to the table of our troubles. At this point, between ten and fifteen years into the game, our imaging standards, images, and imaging tools are responsive and sturdy enough to continue to provoke the combination of uncertainty with optimism (even, occasionally, vision) that has been the fundamental dynamic of web-based humanities computing. In technical areas we are, much of the time, pathetically dependent on the expertise of others whose investments are elsewhere. In print media scholars have been dependent on typesetters, printers, and publishers, of course, but in a technological environment that changed very slowly by comparison. Our uncertainty and dependence are not remediable conditions of our work — and supposing otherwise could be paralyzing. We must simply accept the expertise, make what we can of it along with the compromises we must make, and move ahead. It is never enough to say this once. We must say it again and again.

Notes

[1]Useful accounts of the complex status of images as intellectual property appear in Bielstein’s recent book and Howard’s Chronicle update on the difficulties that art historians face when they want to reproduce pictures, along a few emerging if very limited remedies. Consider that the popular e-reader Kindle, developed and marketed by the powerful corporation Amazon.com, must eschew most images in its electronic books: "[Y]ou are unlikely to find on Kindle any books that benefit from illustrations. Permission is so difficult to obtain for online books that most presses aren’t trying — and many believe that Kindle doesn’t yet provide optimal viewing for all illustrations"  [Jaschik 2008].
[2]Their full Blake collections are also outlined at http://www.blakearchive.org/blake/resources.html. Future contributing collections whose images we have in hand but have not yet published are not listed.
[3]With credit due entirely to the wisdom and foresight of John Unsworth and the team he assembled at the Institute for Advanced Technology at the University of Virginia, which guided our development until 2007.
[4]To minimize the size of image files and thus improve speed of access, a lossy algorithm discards image data that is less important to the normal habits of human visual perception.
[5]As originally planned, the JPEG 2000 standard was ultimately to have eleven parts, as follows (quoting directly from the JPEG site):
  • Part 1, Core coding system (intended as royalty and license-fee free - NB not patent-free)
  • Part 2, Extensions (adds more features and sophistication to the core
  • Part 3, Motion JPEG 2000
  • Part 4, Conformance
  • Part 5, Reference software (Java and C implementations are available)
  • Part 6, Compound image file format (document imaging, for pre-press and fax-like applications, etc.)
  • Part 7 has been abandoned
  • Part 8, JPSEC (security aspects)
  • Part 9, JPIP (interactive protocols and API)
  • Part 10, JP3D (volumetric imaging)
  • Part 11, JPWL (wireless applications)
  • Part 12, ISO Base Media File Format (common with MPEG-4)

 (http://jpeg.org/jpeg2000/index.html, accessed 26 June 2009)

[6]According to Wikipedia, for instance, "JPEG 2000 has been published as an ISO standard, ISO/IEC 15444. As of 2008, JPEG 2000 is not widely supported in web browsers, and hence is not generally used on the World Wide Web (http://en.wikipedia.org/wiki/JPEG_2000 accessed 26 June 2009)"  (http://en.wikipedia.org/wiki/JPEG_2000, accessed 26 June 2009). From a Library of Congress site dedicated to image preservation: "[I]n early 2007, some commentators on the Web called attention to the fact that JPEG 2000 encoding is not being built into camera chips nor is JPEG 2000 decoding native to Web browsers. This has led them to compare the adoption of the format in unfavorable terms to JPEG_DCT, the earlier JPEG codec, which is native to virtually all digital cameras and browsers"  (http://www.digitalpreservation.gov/formats/fdd/fdd000143.shtml, accessed 16 Nov. 2007). On the other hand, as the same Library of Congress page notes, "Implementations of JPEG 2000 have been increasing steadily during 2005 and 2006. Michael Gormish, as part of his Gormish Notes on JPEG2000, now maintains a small 'wiki' for his tracking of JPEG2000 adoption, rather than a single page, because developments to record are increasingly frequent"  (http://www.crc.ricoh.com/~gormish/jpeg2000adoption.html). The mix of favorable and less favorable news on Gormish’s page is, realistically, openended; the latest entry in his tracking wiki is Dec. 2006 (as of 16 Nov. 2007).
[7]The rights for Iconclass software were acquired in 2006 by the Netherlands Institute for Art History (Rijksbureau voor Kunsthistorische Documentatie, RKD) in The Hague, which also took on responsibility "for the daily management and the further development of the Iconclass System"  (http://www.iconclass.nl/).
[8]Julia Thomas offers an interesting account of the frustratingly "entangled" character of looking-at, looking-for, and the quest for meaning in "a technological environment that allows access to more visual images than ever before"  [Thomas 2007, 193]. Among her subjects is the attempt to use words to describe pictures.
[9]To try it — do try it! — simply check off a few search terms in the list of keywords — preferably but not necessarily terms that seem to fall naturally into a group — and follow the results pages until you reach an image with its annotations. You may begin at any Archive OVP - any page that bears an image from the Archive - or at the Image Search main page (http://www.blakearchive.org/blake/imagesearch.html).
[10]Not that ARTstor uses Iconclass, of course, or any systematic image-searching method as far as I know — my impression is that it chiefly depends on whatever metadata come with the images from their home collections — but it has rapidly acquired a vast fund of experience with many different kinds of collections that scholars will want to search systematically.

Works Cited

Bielstein 2006 Bielstein, Susan M. Permissions, a Survival Guide: Blunt Talk about Art as Intellectual Property. Chicago: U of Chicago P, 2006.
Bindman 1991-95 Bindman, David, gen. ed. Blake’s Illuminated Books. 6 vols. Princeton, NJ: William Blake Trust and Princeton UP, and London: William Blake Trust and Tate Gallery Publications, 1991-95.
Cerf 2008 Cerf, Vint. "Way, Way beyond Web Search." PC Magazine 27.1-2 (Jan. 2008): 77.
Child 2006 Child, Julia, with Alex Prud’homme. My Life in France. New York: Knopf, 2006.
Claburn 2008 Claburn, Thomas. "Google Patent Imagines Robots Indexing The Grocery Aisle." Information Week, 4 Jan. 2008: http://www.informationweek.com/news/showarticle.jhtml?articleid=205208105
Eaves 2006 Eaves, Morris. "Crafting Editorial Settlements." RoN: Romanticism on the Net. http://www.ron.umontreal.ca/ University of Montreal. Issues 41-42 (10th anniversary issue). (February-May 2006): http://www.erudit.org/revue/ron/2006/v/n41-42/013150ar.html
Erdman 1988 Erdman, David V., ed. The Complete Poetry and Prose of William Blake. Rev. ed. Berkeley and Los Angeles: U of California P, 1988.
Fuchs 1972-3 Fuchs, R. H. "Henri van de Waal, 1910-1972." Simiolus: Netherlands Quarterly for the History of Art 6.1 (1972-73): 4-7.
Howard 2006 Howard, Jennifer. "Picture Imperfect." Chronicle of Higher Education 52.48 (4 Aug. 2006): A12.
Jaschik 2008 Jaschik, Scott. "University Presses Start to Sell via Kindle." Inside Higher Ed 24 June 2008. http://www.insidehighered.com/news/2008/06/24/kindle
Mihm 2006 Mihm, Stephen. "No Ordinary Counterfeit." New York Times Magazine. New York Times 23 July 2006: section 4, 36-41.
Misic, Buckley and Parker 2002 Misic, Vladimir, Rob R. Buckley, and Kevin J. Parker. "Encoding and Processing of Color Engravings (Using MRC)." International Conference on Image Processing 3 (2002): 773-76.
Misic 2003 Misic, Vladimir. "Mixed Raster Content for Processing of Colored Engravings." Diss. University of Rochester [Electrical and Computer Engineering], 2003.
Misic and Kraus 2002 Misic, Vladimir, and Kari Kraus. "Digital Representation and Compression of William Blake’s Hand-Colored Engravings." Conference presentation. Museums and the Web, 2002: http://www.archimuse.com/mw2002/papers/misic/misic.html
Preston 2005 Preston, Richard. "Capturing the Unicorn: How Two Mathematicians Came to the Aid of the Met." The New Yorker 11 April 2005. 28-33.
Robinson 1993 Robinson, Peter. The Digitization of Primary Textual Sources. Office for Humanities Communication Publication No. 4. Oxford: Oxford University Computing Services, 1993.
The William Blake Archive The William Blake Archive. Ed. Morris Eaves, Robert N. Essick, and Joseph Viscomi http://www.blakearchive.org
Thomas 2007 Thomas, Julia. "Getting the Picture: Word and Image in the Digital Archive." European Journal of English Studies 11.2 (August 2007): 193-206.
van de Waal, Fuchs, Tholen and Couprie, 1973-1985 Waal, Henri van de, R. H. Fuchs, E. Tholen, and L. D. Couprie, eds. Iconclass: An Iconographic Classification System. 17 vols. Amsterdam : North-Holland Pub. Co., 1973-1985.
Wikipedia Wikipedia http://en.wikipedia.org/ Entries "Java [programming language]," "Java Performance," "Criticism of Java," "JPEG," "JPEG 2000"