Muninn

I have already had occasion to mention my mother’s online historical project warsailors.com but I wanted to write a short post to congratulate her on the completion of a remarkable achievement.

My mother’s ship lists contains detailed information on hundreds of Norwegian merchant marine vessels from World War II that played a key role in supplying the allies in Britain and elsewhere during the war. She also contains information on many of the ships in the home fleet under German control.

In addition to reference information about the ships such as the years they were built, tonnage, and their fate during or after the war, she has assembled an incredible collection of anecdotes, crew lists, prisoner of war information, and other valuable information about the ships that are useful not only to historians but also the thousands of descendants of sailors who served and in many cases died on these ships.

The primary task she has dedicated herself over the past months is to organize, process, and upload thousands of images of voyage lists of these vessels taken from the Norwegian national archives and post the images as links from the various ships where they can be viewed directly and compared to similar voyage records compiled and already available online by an Arnold Hague for fact checking purposes.

I have picked a ship completely at random, M/T Dageid, which shows you the kind of pages that have grown out of her years of work. I’m sure she has other ship entries that she is particularly proud of, but this random entry is already impressive. The ships now contain up to a dozen or so links to pages of the voyage records so, for example, descendants can determine where their merchant marine, or at least his ship, was at a given time during the war.

Although the links to these images are small and may go unnoticed, they represent many hundreds of hours of work by my mother. She now returns to her previous task of meticulously assembling and organizing information about the hundreds of convoys these ships sailed in. You can see her considerable progress so far on her convoys page.

Last week I paid a visit to a wonderful archive in a medium sized city of Shandong province, China. There I looked up various documents from the 1940s for my dissertation research that are a bit more local in scope than those I have been looking at in the Shandong Provincial Archives here in Jinan.

The archivists were incredibly friendly, and warned me in advanced that they didn’t think they would have too much from the period I was looking at. After providing the letters of introduction that are required at most archives in China and having the way paved for me thanks to a phone call from a contact I made in Jinan, I was allowed to search for documents using their digital database. They even gave me a free lunch from their cafeteria on the first day and a free copy of a book they had published that I was interested in getting containing documents from the wartime period.

Unlike the provincial archives, this archive found their collection manageable enough to scan and store digitally copies of all the files and make them available for viewing by visitors in place of the originals. Unfortunately, I was not given the option of looking at the originals instead. Also unlike the provincial archives, the online search of their database seems to return results from a much larger proportion of materials that are found by searching for the same on their internal database. (( When I asked one of the archivists at the provincial archives why they did not provide full online access to the database, rather than a very small sampler of the full internal database so that visitors could come prepared with a list of documents to request, I got a bewildered and serious look, “Do you want to put me out of a job?” This answer only makes sense if you realize that one of the primary duties of two of the archivists is to sit at the database search engine and help first time visitors search for documents. Given the fact many of the, especially older, visitors are completely computer illiterate, however, I still believe their services would continue to be required to help elderly comrades who come to search for their records. )) They did not allow me to save any of the digital TIF image collections of individual documents onto a USB drive (( though, as was the case with the Korean national archive, it would have been simple enough for a less scrupulous person to do this given the access to the “Save As…” option in the file menu and apparent lack of any security on the machine I was given access to. In fact, in the case of the Korean national archive at Daejeon, web browser access was restricted but I was able to confirm, at least as of 2008, the DOS command line still gave me FTP access to my server where I could have uploaded hundreds of pages of Korean archive documents they were requiring me to wastefully print and pay for, had I been so inclined to disregard their rules. )) but I was allowed to print documents and, after their contents was checked over by the archivist (( A bizarre and surely unnecessary step, since the documents have been screened once when they were added to the database for classified information. I could easily note down in my notes anything I read in the documents before printing them so not letting me keep the print outs hardly serves to prevent sensitive or privacy violating information from leaking out. If privacy issues are primary there should be a system, like the one at the Korean national archive, which charges the visitor to process accessed documents to redact out the names of people mentioned. At the Pusan branch of the Korean National Archive I paid about $50 and waited three days to get access to some old police logs. It took that much time because they had to go through and erase the names and provide me copies. However, I’m still grateful I got access at all. Although this is an important issue that deserves consideration, I generally feel that the privacy laws of Korea and Japan are far too strict and that they seriously inhibit serious historical work from the 19th through the period I’m working on in the mid-20th century )), to make off with these environmentally less friendly non-digital printouts.

Unfortunately, almost everything that could have been done wrong with this digitization program and its presentation to the visitor did. So let me list of the issues as a warning to other, especially smaller archives, that might consider going the digital route. I have listed them from the least worrisome to most serious:

1) Environment: The computer designated for viewing of documents had a cheap monitor with little screen brightness (even when set to full) which faced a window where sunlight beamed into the room (even when I convinced them to partially lower shades), providing a horrible viewing experience and harm to the eyes. An uncomfortable mini-mouse, horrible chair, and a table with almost no spare room for visitors to put a notebook or their laptop made this a nightmare to spend any length of time looking at documents.

2) Software: The custom built database software had an advanced query system which is useful for advanced users and archivists but requires multiple stages to search and although I quickly got used to it, I think it would confuse users not used to such systems. Also, when it shows images of archive files, a lot of vertical screen space is wasted on software options and interface components, which leads to a great deal of scrolling at any zoom level that makes reading possible.

3) Page Numbers: At the archive in question I requested a lot of documents where essentially local versions of other documents that I had seen before from other districts. Having seen many originals of this kind I know most of them are one small A5ish sized sheets of very thin paper that are held together with string. Despite the age of these documents, surprisingly I have never run into paging issues at the provincial archives, mostly because I’m seeing them still stringed together. By contrast, pages were all over the place in these documents in their digital form. While it is possible they were already unstringed and in messed up order when the contractors got the documents, I suspect that they got messed up through negligence when the originals were unstringed in order to be scanned.

4) Indexing: This is a very serious problem I found with all but two of the 70 or so documents I looked up during the two days I was at the archive. Before coming to the archive, I used the online database I made a list of file names and file numbers for documents I was interested in. I brought these to the archive and looked up the same numbers in the internal database. Each file number, unfortunately, corresponds to a packet of multiple files ranging, at least judging by what I saw, from 15-50 or so in number. I could then easily locate the appropriate document by its file name and open the images directly in the system. To my horror, in all but two of the cases, the documents in the file images did not correspond to the file name. For each document I would have to hunt through the other dozen or several dozen documents in the same general area to find the images for the file I was looking for. Sometimes I was never able to locate the file, suggesting that those images are probably found in other file groups, if at all. Now, what am I supposed to do as a historian when I cite the documents I did find? I’ll record the correct file numbers, found in the database, but any other historian wishing to confirm the information I am citing will look them up and find a completely different document unless the archivists have gone in and fixed all the indexing issues throughout their scanned collection.

I asked two of the archivists about this issue and I essentially got a, “That is funny. Well, just hunt through the rest of them and find your document. It’s probably like that for this whole collection. We paid a contractor to have it done and didn’t have the resources to check all their work.”

5) Quality: The documents I’m looking at are Communist public security bureau reports and Communist party internal reports. Some of them are hand written or are characters carved onto a special surface that allows a sort of reproduction process frequently used in the 1940s (any printing history buffs know what this ancient photocopying method is called?). In either case, they are very difficult to read, faded with time, on surfaces that are themselves often in poor condition, and most importantly, written in tiny sizes. If you are going to digitize these kinds of documents, then, you need to digitize them with a much higher quality. As I mentioned in my posting on triage in the archives, I have had to sometimes completely skip some of the more hopelessly unreadable documents or those for which the pages per hour drops to a rate that makes the investment of time not worth it. I would say that this happens in perhaps 1/10 documents I look at here.

Now, take these same kinds of documents and scan them. If you scan them well, at high resolution and with color, then you can actually make those difficult to read but important sections more readable thanks to the power of zooming in on parts of the image. However, that is not what happened here.

The contractors here decided to take these extremely difficult to read originals and scan them in black and white (not even in greyscale!). Now I know the evidence seems to suggest that if you are going to run a massive scale OCR program on historical newspapers, for example, then black and white is not significantly worse than greyscale. However, OCR is not even worth trying on these hard documents, unless there are some major breakthroughs in artificial intelligence. If, however, you are trying to use human eyes to read difficult to read handwritten or carved Chinese characters on poorly preserved mediums, you need to preserve as much of the quality of the originals as possible. The cost benefit analysis done in this case resulted, in the case of many documents, in completely unreadable digital copies.

This really left me depressed. In the case of the completely botched indexing described in number four above, an archivist or the hired contractor can go back and meticulously re-index the documents so that they point to the correct images. Since some of the documents have visible page numbers, messed up page numbers might also be fixed in those cases. However, I suspect it is harder to go back and explain to the budget committee, “Ya, our contractor blew the scanning job and made thousands of once barely readable documents in our collection now completely unreadable to visitors. Can we pay to do the scanning all over again?”

I came back to Jinan yesterday morning and felt incredibly happy to go back to reading similar documents in my own hands. (( Note to super friendly archivists: if you encourage a visiting PhD student to eat while looking at the documents by suddenly (and generously) giving him a handful of juicy baby tomatoes, you might end up with a bit of tomato juice on one of the pages of part two of the 1946 treason elimination report from the Donghai public security bureau of the Jiaodong district. )) Digitization can do amazing things for improving access and preservation. When the Japanese national library set about digitizing all Meiji and now Taisho period publications I found myself complaining mostly about the slower speed at which I could browse or skim through the books. I didn’t find that readability itself suffered too much during the process. In a case like these far more difficult to read wartime Communist documents, however, sloppy digitization of these documents, only gradually opening up to researchers and historians, actually reduces rather than increases access.

I’ve been outside of Jinan this week, traveling about a bit. Yesterday I caught a ferry from Penglai (蓬莱) to a group of islands known as Changdao (長島) county which I had been told were well known for their scenic beauty. I had a day left of traveling with no specific plans and it seemed like a nice quiet place to spend a day before I head back to Jinan for my last week in China. I arrived in Changdao late in the afternoon and after checking into one of the only hotels open before the summer tourist season starts in May, I wandered about the town a bit. I didn’t ever get outside the sleepy fishing town in the south of the islands either that evening or the next morning when I caught the ferry back to the mainland. Instead of making it out to see the Changdao National Forest Park and Changdao National Nature Reserve, instead I mostly roamed about the back streets of the town and port.

I couldn’t help noticing that the locals gave me more than the usual amount of attention with a much higher frequency of gasps, cries of “Laowai!” and in one case a mother in a grocery store giving a short lecture to her child, surely too young to understand, about what this monster in their midst was (“You have never seen one of those before, have you? Don’t be scared. A foreigner is someone from another country and they don’t all look like us…”). This is nothing new, of course, to those who have traveled outside the major cities of Asia and I simply attributed this to the natural curiosity for non-Asians I have experienced throughout the countryside of Japan, Korea, and China.

During that first evening, though, I learn something about Changdao almost by accident. Walking back to my hotel late in the evening I passed by a TV shop where my iPod detected a wireless internet connection. I stopped outside the shop to download some email, and, since I really knew nothing about the place I was visiting, at least downloaded the Chinese and English wikipedia articles for the islands on my little offline Wikipedia client on my iPod. When I read the article later that evening, I found the English page had these two surprising paragraphs:

Changdao Island is closed to non-Chinese nationals. Westerners found on the island are swiftly taken to the passenger ferry terminal and placed on the next ferry back to Penglai by the islands Police service. Islanders promptly report all “outsiders” to the islands police service. (First hand experience) Police explain the reasons for this, due to the high number of military installations on the Island.

The Changdao Islands are now open to non-Chinese nationals, including westerners This was agreed by the local and national governments as of 1st December 2008.

Given the fact that non-Chinese nationals have apparently only been permitted on the island since December, and the tourism season hasn’t really started, the relative isolation of these islands may not have been the only reason there was extra surprise at the sight of a (visibly identifiable) foreigner in their midsts.

The next day, I checked out of the hotel, and made my way back to the ferry terminal. On the way, I walked over to the nearby TV shop to download my morning email (I know, I’m an addict). A middle aged man across the street yelled at me to stop. None of the many townspeople I had come across the day before had stopped me but armed with my new knowledge about the island I nervously complied. He came up to me and asked me if I had registered with the police. I told him I hadn’t. He asked me what I was doing on the islands, where I had stayed, etc. I answered honestly. Although he was polite, he said he wouldn’t let me go until he had called the police to ask if I had registered yet. I explained I hadn’t registered but I had only arrived the night before (( I think foreigners are technically supposed to register with the police everywhere in China within 24 hours of their arrival, and I did register in Jinan soon after my arrival, but almost no tourists traveling in China register in every city they stay in, At any rate, this registration he spoke of is not thus a Changdao specific requirement. Technically though, I hadn’t yet reached the 24th hour and I was off the island before my time ran out. )) and, at any rate, was now on my way to the ferry terminal to return to the mainland. “Ah, he said, but why are you going this way, when the ferry terminal is that way?!” Fortunately, a little more explanation made him understand that I simply wanted to walk a few more meters up the road to steal a wireless connection I had come across to check my email before hopping into a cab and going to the ferry terminal. At any rate, I avoided this concerned citizen’s detention, and the potential time-consuming process of going to the Changdao county police station to register myself.

Two notes to the Changdao authorities:

1. If I hadn’t downloaded that Wikipedia article, I never would have known there was any special status for the islands or any kind of military installations. Only the English wikipedia entry, and this 2005 blog entry from someone who was blocked entry some years ago alerted me to the fact, and only after I had checked into my hotel on the island. If foreigners need to take care to register when visiting the scenic islands or are subject to other restrictions, perhaps a sign anywhere in the ferry terminal (( I confirmed there is no special information in either Chinese or English posted about the status of the islands when I returned to Penglai)), or perhaps somewhere on the nice English language website for Changdao county where I am welcomed to the, “peaceful, sincere, civilized and beautiful Changdao for business investment and holiday!” If there is some kind of required registration procedure, can I recommend that one be able and asked to do this upon arrival at the ferry terminal or when one checks into the hotel (the hotel didn’t even look inside my Norwegian passport when I checked in). Finally, if a potentially military adversary like the United States really wanted to send a spy to reconnoiter your military bases on the islands, do you really think it would be a good idea to send an easily identifiable caucasian instead of one of its many citizens of Asian or similar complexion or even better, a hired local?

(more…)

I’m working on my last batch of documents in the provincial archives in Shandong. There are two challenges to doing my historical research here which I often think about. The first is the problem of access to both the archive and much of its contents. I have been very fortunate but I regret that it is more of a result of good fortune than anything else. This posting will focus on the other problem, the need for a kind of triage in the archives and the constant awareness of my own personal limits as a reader. It is a humbling experience, and I suspect many, if not most, historians, come to face it if they have spent much time doing archival research, especially dealing with documents not in a language they speak and read natively.

Language and Detailed Local Knowledge

I enter the archives here with a topic in mind, a relatively good understanding of the regional and chronological context for my topic of study, and a working knowledge of the terminology often used in the kinds of documents I will be looking at, in part thanks to the existence of a published collection of documents from the same archive (山东革命历史档案资料选编). However, I have two major disadvantages that I feel very acutely every day I come to the archives. One relates to my language ability; the other to the limits of my local knowledge.

Though I can read Chinese, especially when it comes to the materials in my particular field of study, I have two huge linguistic disadvantages compared to any native speaker of Chinese (and, to a lesser degree, native speakers of Japanese): 1) I read Chinese much slower, and more importantly, skim Chinese slower, than native speakers. I still have to occasionally look up words that cannot either be understood by context or safely ignored due to probable irrelevancy. 2) I do not have a lifetime of practice reading handwritten documents using cursive or radically simplified Chinese characters, which compose over half of the materials I’m looking at. This means that some of the many handwritten documents I look at here, where I do not have permission to photocopy or take photographs of the materials I am looking at, are partially or in a few cases completely impossible for me to read.

The second major kind of disadvantage I have relates to the fact that, as one archivist here put it to me sympathetically, “This must be overwhelming, since you have only had time to study Chinese history for a year or two before you came.” This makes it seem like every Chinese historian has studied Chinese history for decades and is thus many years ahead in terms of knowledge of the specifics of Communist party anti-treason campaigns in Shandong province, which is simply not the case. However, all other things being equal, I must come to terms with an obvious fact that lies at the heart of what the archivist was trying to point out to me: It is physically impossible for me to have found time to read more than a subset of the Chinese language secondary works or document collections that are related to my field in the short time I have worked on my dissertation, let alone read, as some graduate students and scholars here undoubtedly have, read the many other peripheral works that help one understand the context surrounding my topic. This is even more true since I am doing a transnational and comparative project that also incorporates Korea.

The only way people in my position can walk into the archive each day with some degree of self-respect is to convince ourselves that we have something unique to offer the study of our historical topic that gives us some kind of advantage relative to other scholars and students who might be working on a similar field here. Whatever this might be, our critical question, our comparative approach, our sensitivity to patterns etc. that might not be apparent to those working in other scholarly contexts, and so on, it gives us the confidence to go in and struggle through the historical materials and accept our weaknesses. In my case, I try to tell myself the contribution I can make is largely to be found in the way I “slice” the range of my inquiry and attempt to use that slice to answer particular questions. I remain open to the idea, however, that the “uniqueness of approach” claim may ultimately be an illusion, and as the quality of academic research here in China improves rapidly (I was really impressed with the breadth of reading and fresh approaches taken by some graduate students I have met here), some of the other advantages that foreign scholars coming to study might once have dared to claim are disappearing.

Even if one does avoid falling into complete despair, it remains an incredibly humbling experience to walk into the archive each day and be faced time and time again with one’s own all-so-apparent inadequacies. Below, let me share some aspects of that experience with some examples and the unfortunate but necessary steps I have to take in order to maximize the number of historical gemstones I can mine in the ocean of archival material available to me, despite my weaknesses.
(more…)