Understanding the searching and browsing needs of a digital library’s core users is important for anyone building a new online resource or refreshing an existing one [Agichtein et al. 2006a]. In 2014, British History Online (BHO) was seeking to rejuvenate its navigation and architecture after a decade in production and sought the direct feedback from its users to shape that process via a voluntary user survey of individuals who had registered accounts with the project. Volunteers were recruited through an email sent in May 2014 to all 6,301 registered users of the site, which included those with paid subscriptions, but also those repeat users who opted to sign up for a free account to take advantage of bookmarking features. 1,439 people responded to the survey call, representing a response rate of 22.8%. This paper discusses the findings of that survey as well as how they provide us with new understanding of the different needs and expectations of academic historians and genealogical users of digital libraries and archives, providing a basis for future conversations on identifying user needs more broadly.

BHO is a digital library of key printed primary and secondary sources for the history of Britain and Ireland, with a principal focus on the period between 1300 and 1800. The project is an initiative of the Institute of Historical Research, part of the University of London. The collection currently contains 1,250 volumes, and 120,000 web pages of material (each comprising the equivalent of many “pages” of original printed work). The site has particular strengths in the late medieval and early modern periods up to about 1660, as well as a strong local history collection.[1] As such it is both unique and typical of digitisation initiatives that contain primary or secondary source material.

BHO is a product of an evolution in digitisation stretching back to early attempts at microfilming and microfiching in the last century, but also early digital text collections such as Project Gutenberg [Hart 1971] as well as multimedia endeavours including the Electronic Beowulf project [Kiernan et al. 1999]. It is also specifically the product of a set of British digitisation initiatives from the turn of the twenty-first century that focused on digitising British historical source material. These included The Clergy of the Church of England Database, [Burns et al. 1999-2016], The Old Bailey Online [Hitchcock et al. 2002], and the Charles Booth Online Archive [Donnelly et al. 2002] as well as Eighteenth Century Collections Online [Anon 1999] and Early English Books Online [Anon 2000] (see [Hitchcock 2008] for a fuller discussion of the digitisation landscape at the time).

The approaches to digitisation, as well as searching, browsing, and access, naturally informed a subsequent wave of British historical digitisation projects, including most notably London Lives [Hitchcock et al. 2010] and Connected Histories [Hitchcock et al. 2011] and a number of commercial collaborations between the British Library and various partners, resulting in archives such as the digitised Nineteenth Century Newspapers [Anon 2007] and Burney Collection [Anon 2009]. The project is also an important part of the technical revisionist reactions in the form of discrete re-curated datasets that eschew searchable databases entirely and instead focus on a mutable interpretation of a set of records as the new unit of dissemination [Boulton and Schwarz 2007] [Crymble et al. 2015] [Howard 2016].

These and similar projects have been engaged in ongoing conversations about search and browse that are more often expressed as websites or datasets than as journal articles. This paper takes those digital expressions and looks at it in the context of the extensive body of literature on user needs, as well as the findings of BHO’s own user survey on the search and browsing desires of its large user-base.

After having launched in 2003, by early 2014 the site was beginning to show its age. At the time of writing, readers can find semi-functional archived copies of BHO via the Internet Archive.[2] Figure 1 shows a screenshot of the main page of the pre-development site. This old version of the site was navigable via keyword searching, or via browsing. Searching was the most popular option. The search feature compared to those found on most websites. At the time the rebuild began in early 2014, the site was navigable via a search box, powered by a physical Google Mini search server and based on common search algorithms used in 2010. Survey respondents from all three groups rated the search an average of 4 out of 5, suggesting they were happy (or at least comfortable) with the existing search capabilities.

The search itself was possible because of the high quality transcriptions available in the BHO collection. The core collection had been built entirely using double-rekeying, a process that involved two typists individually transcribing the texts, with the resultant work compared and differences between them reconciled manually. This resulted in a very high level of accuracy, as it is unlikely that two typists would make the same mistakes. The team believes that the content produced through double-rekeying has an accuracy level of more than 99.995 percent.[3] Errors in the original printed volumes have been transcribed verbatim, leaving a level of ambiguity to the number of “errors” in the collection. Because of the double-rekeying approach, the corpus was relatively easy to keyword search - with the limits of keyword searching in mind, such as archaic spelling, abbreviations, and the occasional error in the original volume [Beall 2011]; [Badke 2011].

Users also had the separate option of browsing the collection through a series of tags that were manually created by the editors when the content was first uploaded to the website:

These tags are indicative of the historical nature of the collection, the British focus, and of early decisions about acquisition strategies, which were tied closely to the interests of some of the early project partners. These include, The Centre for Metropolitan History at the Institute for Historical Research, and The History of Parliament project.[4] This is why “administrative and legal” has been categorised distinctly from “parliamentary” amongst the subject tags. There was no way to combine searching and browsing in a meaningful way, or to filter results by more than one tag. The options had become what Marcia Bates described as “a hodgepodge of system elements working at cross-purposes” rather than an integrated searching and browsing environment [Bates 2002].

In order to improve these navigation options, the BHO team decided to seek feedback from its users on the types of features they wanted to be able to use to identify relevant records within the collection. Though the Internet is relatively young, a number of designers, computer scientists, and library and archive professionals have conducted research and written strategies for improving searching and browsing experiences for users of online collections. The studies include surveys such as the one conducted by BHO, as well as indirect monitoring of users with unobtrusive technologies such as eye-tracking or video recording [Granka et al. 2004]; [Tullis 2007]; [Hill et al. 2011]. There are also a number of studies that implicitly gather feedback, using techniques such as what Agichtein and his colleagues called “clickthrough interpretation” (measuring which links someone chose to click on a given page) [Agichtein et al. 2006b]; [Joachims et al. 2007]. Other forms of discrete monitoring include asking people to use computers fitted with tracking software [Grace-Martin and Gay 2001], analysing server logs [Bates 1996]; [Jansen and Spink 2006], or monitoring subjects in a lab setting [MaKinster et al. 2002]; [Tsai et al. 2012]; [Hsu et al. 2014]. Claire Warwick and her colleagues, for example, conducted this form of discrete monitoring of web logs for the “Log analysis of Internet resources in the Arts and Humanities” project, in an attempt to determine the digital use patterns of scholars in those fields [Warwick et al. 2008]. The growth in this implicit feedback analysis has contributed to the web analytics craze, in which free services such as “Google Analytics” allow website owners to monitor their traffic and gather statistics on user demographics.[5] Whether direct questionnaires or indirect monitoring are the most appropriate way to gather feedback is up for debate. Paul Samuelson argued instead for the power of “revealed preference”, which monitors what people want by looking at what they spend their money on [Samuelson 1948]. As Harley and Henke warn, all approaches have their advantages and disadvantages; surveys may be inexpensive, but may not provide representative opinions, whereas analyses of transaction logs are less human and thus make it nearly impossible to infer the goals and intentions of users [Harley and Henke 2007].

BHO opted for a survey because it was inexpensive and provided opportunities for a large cohort of users worldwide to offer their opinions in a targeted manner. This mode also allowed users to express preferences for new types of services that did not exist on the current website and may therefore be impossible to monitor by collecting data on current use. We also believed that it was important to listen to our users, many of whom had been using the service for a decade. As each new mode of searching or browsing requires additional metadata to be generated and stored about the records in the collection asking users in advance of their preferences made it possible to prioritise development.

Searching and browsing within the context of information retrieval have long roots within discussions about libraries that extend far further back than the advent of the Internet. Should one search or browse to find the most pertinent resources for a project? The limits of the library itself – be that physical or digital – often impose limits on the researcher and make that decision for them. In the fictional fourteenth-century monastic library in Umberto Eco’s The Name of the Rose [Eco 2004], it was the librarian who decided if you were worthy of finding what you sought. What you received from the restricted stacks in the labyrinth above the reading room came down to his judgment alone; he was the keeper of the knowledge as well as its search interface. As Barclay argued in 2010, that power relationship between librarian and reader remained until the middle of the twentieth century. He noted that browsable open stacks were not an “ancient scholarly right”, but an invention that was “no older than the baby-boomer faculty who so often lead the charge to keep books on campus” [Barclay 2010]. Barclay sought to dispel what he called the myths of browsing, noting that the academic value of browsing the stacks was already compromised by the large number of missing and stolen books, books currently in use by other readers, and by the very fact that many people cannot be bothered to look at items on the top or bottom shelves [Barclay 2010]. Barclay believed that new electronic search capabilities would enable scholars to continue to find books held in remote storage.

Some researchers long before Barclay saw the potential of search, including Bruce Schatz, who in 1997 predicted that by 2010 we would have “concept searching enabling semantic retrieval across large collections”  [Schatz 1997]. Schatz was a bit premature in his prediction, but concept or semantic searching has recently become more important, as it becomes clear that what we want from a collection is a relevant document rather than a document that contains a particular keyword. For Dan Cohen, to the question, “is Google good for history”, his answer was simply: “of course”  [Cohen 2010]. While Jesse Shera, writing in 1964 noted that “the automation of libraries through the use of computers and book-finding robots is 'pure fantasy'”, the digital era has certainly needed to meet some of those challenges [Shera 1964]. Users of online archives and libraries need to be in possession of all the tools that they need to help find what they are after.

Not everyone agreed with Dan Cohen’s enthusiasm; even as early as 1985, Champlin warned of the “perils and pitfalls” of online search, arguing for the importance of print indexes and abstracts for researchers [Champlin 1985]. A survey of historians’ preferences by Ian Andersen in 2004 showed that little had changed, with print-based finding aids and “informal leads” from colleagues or librarians the preferred information retrieval strategy of scholars in that study [Anderson 2004]. Many scholars used browsing and shelf proximity as part of their information retrieval strategy. Richard Mott countered Barclay’s arguments about the limits of browsing by highlighting the practice of obtaining call numbers as a way of identifying an appropriate shelf to begin browsing. Mott believed that it was “not uncommon for this browsing to yield relevant sources that electronic searching alone misses”  [Mott 2010]. This is the principle of serendipity in discovery. Stephen Ramsay calls it “screwing around”. For Ramsay, browsing is an important part of the scholarly process, which prevents us from focusing solely on the predefined literary canon. He urges us to take time to wander the library and grab things that seem like they might be interesting, as a means to discovering new knowledge or unexamined connections [Ramsay 2010].

Within the world of online libraries, Maxwell argued that simple browsing was more important than keyword searching [Maxwell 2010]. Grey and Hurko echoed those same warnings, suggesting that controlled subject searching was more effective than keyword searching for research, which of course necessitates a good set of controlled subjects built into the collection [Grey and Hurko 2012]. Collins and her colleagues noted that researcher uptake of new digital modes of research depended heavily on what was considered normal and acceptable within their discipline [Collins et al. 2012]. These disciplinary standards in history were well entrenched; in 2003, a survey of 278 historians by Dalton and Charnigo found that while 58% of respondents used websites for their research, only 3% believed it was the most important way they conducted research, lagging far behind traditional finding aids, footnotes, and archival or library catalogues [Dalton and Charnigo 2004]; [Hamburger 2004]. The four million annual users of BHO suggests that acceptance of digital resources may have moved forwards in the past decade. Genealogists, despite tending to be older, have always been more willing to engage with digital sources than their academic historian cousins [Duff and Cherry 2001]. Nevertheless, the question remained: how do these users want to be able to explore the collection in ways that they could not do on the pre-development version of the BHO website?

Since its launch in 2003, BHO has developed a strong user-base that has been attracted to the historical content on the website. In the project year 2013-14 (August – July), the site received more than 4-million visits from 2.7 million unique visitors. Most visitors are anonymous, however, BHO's users can largely be classified into three categories: academic historians, genealogists, and casual users. Each group comes to the site with different expectations and needs from the collection.

Academic historians use the digital library to conduct historical research that is destined for peer-reviewed publication or postgraduate theses. Many of these users are attracted to the subscription-only materials such as the Calendar of State Papers, or the Calendars of Close Rolls. This subscription content comprises twenty percent of the material on BHO, and is targeted at these academic users.[6] 354 survey respondents (25%) classified themselves as academics or students. The majority of academic users were between the ages of 55 and 74 years, with people under the age of 55 accounting for most of the outliers (see Figure 2). Nearly six in ten identified as male.

Genealogists or family historians are the second main group of users. They tend to use the site to look for details of their family's past, and are generally attracted to the Ordnance Survey Maps, as well as the local history resources about the communities in which their relatives lived. 737 respondents (51%) classified themselves as genealogists. Genealogical users were typically older than the academic users. The majority were between 55 and 74, but the largest group was older than 65, and there was a large group older than 75 years. Very few users were under 45. The split between male and female respondents was equal, unlike amongst academics.

The third group is best described as casual users, a large number of whom arrive via search engines or sites such as Wikipedia [Blaney 2013]. The group includes journalists looking for stories, business owners interested in the heritage of their premises, historical video game makers researching historical context for upcoming projects, and people reading for enjoyment. 348 respondents (24%) decided not to classify themselves under one of the above headings or felt the headings did not apply to them. Their age profile most closely mirrors the academic users, and suggests that most of them are not amongst the genealogical group. Given the fact that most users heard about the survey via an email to the address they used to register with BHO, it is likely that many of the people in this category were repeat users. Because the needs and interests of the casual users are so diverse, they are the most difficult to pin down. While the discussion in this paper will include the views of the respondents from this group, they will not be the study’s core focus.

As the average age of BHO users is older than the typical web user, age may be pertinent to a full understanding of the survey results and search and browse preferences. There is a large body of literature that studies the web needs of older web users, or “silver surfers” as they have been called in the past. Much of this research was done in the early years of the 2000s and focused on the cognitive, physical, and sensory decline of the elderly, as well as on ways to address the alleged “fear” older people have of technology [Darin and Kurnaiwan 2000]; [Morrell et al. 2000]; [Zajicek 2001]; [Millward 2003]; [Aula 2005]; [Kurnaiwan and Zaphiris 2005]; [Kurnaiwan et al. 2006]; [Chadwick-Dias et al. 2007]; [Dickinson et al. 2007]; [Gao et al. 2007]; [Tullis 2007]; [Fairweather 2008]; [Hill et al. 2011]. This literature, most of it presumably written by academics under the age of 75, while good intentioned, can at times be patronising and is perhaps less apt for today’s “silver surfer” who may have been in his or her forties when the web first became popular, and thus has decades of experience with the Internet. From the perspective of the BHO project team, it was not so much the age of these users that was important, but how these different groups wanted to navigate online collections like BHO.

To get a better understanding of respondents, users were asked how frequently they used the site. Results can be seen in Figure 3. The most common answer in all three groups was that they used the site “occasionally” (31% academics, 46% genealogists, 32% casual users). Academics were considerably more likely to think of themselves as regular users, opting for the “often” or “very often” categories twice for every genealogist respondent (37% academic versus 15% genealogist and 21% casual users).

A majority of all users were based in the UK or Ireland (66%), with casual users showing even higher levels of local use – See Figure 3. North Americans comprised 19 percent of respondents, with academics proportionately more likely to be North Americans taking advantage of research materials that for them may have been overseas. Only 9 percent of respondents were from Oceania, with a clear majority of them using the site for genealogical reasons. Europeans were rare and generally were academics. Respondents from the rest of the world were very rare.

The respondents of the survey are therefore of an older set of Internet users, typically over the age of 55, and mostly British and Irish. Academic users are more likely to be heavy users of the site, and are more male than female. Genealogical users are occasional visitors, and have no discernible gender variance.

The rest of this paper discusses the project team’s findings derived from the survey.