The Slaving Voyages KML in GoogleMaps

GoogleMaps can also be used directly, without using Map Engine Lite to upload a spreadsheet for each layer. To create a map of the slaving voyages for the second half of the eighteenth century, go to GoogleMaps, select My Places, and then Create Map. Next, either upload the compressed KML file (KMZ)  generated with GoogleMaps (see previous post) or enter the URL for its location on GoogleSites. It, in theory, is then rendered over the default GoogleMaps content such as Wikipedia and photo links. The base map can also be changed to the  satellite or GoogleEarth view within GoogleMaps, but the time slider is not enabled.

In practice, the KML file uploads but does not render the voyages all at once or in chronological order. The voyages display in twenty-five separate "layers." By opening the map in GoogleMaps by clicking the link below the embeded version, users can advance through the layers with the index at the foot of the panel that displays daily position symbols, dates, and TSTD links.

View The Slaving Voyages KML in GoogleMaps in a larger map

Opening the Web map with GoogleMaps also allows users to save it to their own My Places and modify it in various ways such as adding the commonly used place markers or pushpins as well as lines and polygons or descriptive attributes in pop ups.

Time Enabled Slaving Voyages on GoogleEarth

GoogleEarth provides another free option for publishing Web maps, one that provides an alternative to videos for sharing the temporal function of ArcMap. GoogleEarth 7 can be downloaded gratis here. It is an application that runs on and saves your data to your computer, not the cloud.

I created the below temporal Web map of the British and Dutch slaving voyages by converting the ArcMap GIS file (MXD file) into a KMZ file (a compressed KML file) using the Map to KML Tool found with the other Conversion Tools. GoogleEarth and other Web mapping applications use KML (Keyhole Markup Language) to render point, line, and other types of features. That KMZ file therefore includes not only the features and their attributes but the way I symbolized and otherwise configured them as layers in ArcMap.

Next, I opened the KMZ file with GoogleEarth and configured the time slider settings and symbols to suit GoogleEarth, a rather different type of display than ArcMap. Then I saved the reconfigured layer as one of My Places in GoogleEarth as well as, by right clicking the layer folder, as a new KMZ file. I uploaded that KMZ file to a website that I created on GoogleSites, for free yet again, storing it in a page created with the File Cabinet Template and setting its permissions to allow public access. Then I went back to Google Earth and opened the Embed KML Gadget, configuring the various options such as dimensions and entering the URL for the KMX file stored on my GoogleSite.

The result is a fully functional temporal GIS of the British and Dutch slaving voyages of 1751-1795, first shared using ArcGIS Online. Selecting any of the daily position symbols opens a pop-up box with a link to the Web page for that voyage in the Trans-Atlantic Slave Trade Database. By adjusting the various controls of the time slider, users can view the entire period (1751-1795) or constrain it to a single decade, adjust how many months or years of symbols are visible at once, and move along the temporal sequence one step at a time or as a continous animation.

The control of the time slider, especially over the speed at which the sequence runs, is limited compared to using it in GoogleEarth. The KMZ file can, however, be downloaded and added to GoogleEarth running on a computer or added using the Add Network Link. Doing that will also allow users to access the individual layers for each voyage, the daily position points, and all the attributes as well as to resymbolize points, remove and add data, such as place markers for ports, and finally to save their work as a new KMZ file.

The free versions of GoogleEarth, ArcGIS Online, and GoogleMaps Engine Lite, of course, cannot do nearly as much as the ones you pay for: GoogleEarth Pro is $399 for an annual license; a paid subscription to ArcGIS Online is at $2,500 per year for the minimum of 5 users; and GoogleMaps Engine is "expensive."

The Attelante on GoogleMaps

Another free Web mapping application is GoogleMaps Engine Lite. It allows rapid creation of a basic online GIS, with layers and attribute tables. One way of adding data layers is to upload an Excel worksheet, as long as it has no more than 100 records/rows or 50 columns/fields. Maps Engine Lite also has limited symbolization and other functions, compared even to the free, non-subscription version of ArcGIS Online I have been posting with so far. It does allow use of the GoogleMaps Satellite view, however, and its oblique, 45 degree perspective. Simply click the drop down menu beside Base Map in the legend and select satellite view; it will switch to oblique when zoomed in.

The Attelante was a Dutch Brig that sailed for New York City with a cargo of gin (jenever), copper, and white lead in July 1815, arrived in September, and returned to the port of Schiedam with a cargo of tobacco in December 1815. It actually arrived in November but could not get into the harbor due to thick ice on the river, Schiedam being a port on the Nieuwe Maas River and relatively far inland. Although Schiedam is not named as the departure port on the outbound voyage, it likely was because it was the return port and one of the principal producers of gin in the Netherlands.

Despite the brevity of the voyage, it still requires 127 records, necessitating splitting the worksheet into two, one for the outward segment and the other for the homeward one, and uploading them to the Web map as separate layers.

Still, the shortness of the voyage made the Attelante a good candidate to explore the capabilities of GoogleMaps for the type of data used in this particular project. Other vessels and voyages would have required splitting into more than two layers. Besides, I had not previously done a Web map of the Attelante voyage.

Seasonality of the Slave Trade Voyages, 1751-1795

A temporal analysis in ArcMap of the slaving voyages in the database yields the following temporal vizualization. Forty-six of the forty-eight voyages are Dutch and two British, most of them transporting enslaved people across the Atlantic from Africa to Suriname and the Caribbean. Forty-four of the voyages also appear in the Trans-Atlantic Slave Trade Database. For further details on the voyages, see this post.

I achieved the visualization by changing the year of each voyage to a single year, setting both the time interval and time window to one week, and creating a new point feature in the South Atlantic labeled with the month field. The daily-position symbols are triangles, with a week’s worth showing at a time. Their colors represent the direction of the voyage segment, based on their destination fields: green for northbound, orange for southbound, and purple for westbound. That compresses the voyages, which took place from 1751 through 1795, into a single annual period and allows analysis of variation by week and month.

The Trans-Atlantic Slave Trade Database contains many more voyages than this GIS and therefore reveals a great deal of variation in seasonal patterns and why it occurred, as explained in this essay.

Seasonality of the Catalan Triangular Trade

Seasonal variation in supply and demand for tasajo must affected the timing of the triangular Catalan trade with the Rio de la Plata and Cuba. Cuban demand for tasajo increased during the cane harvest from January through June, as plantation owners bought additional enslaved Africans for the grueling work, which killed many. Meanwhile, tasajo production took place during the nine months of October through June, the southern hemisphere’s spring, summer, and fall. During the winter, in contrast, the cattle weighed less and the air temperature was too cold to dry tasajo. To maximize the price differential between the purchase of the tasajo and its sale, then, voyages departed from the Río de la Plata during the season of tasajo production in time to arrive during the season of peak demand in Cuba.

The export statistics for the port of Buenos Aires confirm that seasonal rhythm (after fig. 6.12 in my book Black Ranching Frontiers, Yale,2012). Shipments of tasajo through the port of Buenos Aires began as soon as tasajo became available in the spring (October and, even, September) to arrive in Cuba in November and December in anticipation of the beginning of harvest season. That volume of shipments continued until about May, with those departing at that time still able to arrive in Cuba before the end of the cane harvest. July and August marked the nadir of tasajo shipments northward due to a lack of both supply and demand.

A temporal analysis run in ArcMap, shared using the video below, clearly shows the seasonal variation in the triangular trade, even with only fourteen voyages. I achieved the visualization by changing the year of each voyage to a single year, setting both the time interval and time window to one week, and creating a new point feature in the South Atlantic labeled with the month field. The daily-position symbols are triangles, with a week’s worth showing at a time. Their colors represent the direction of the voyage segment, based on their destination fields: green for northbound, orange for southbound, yellow for eastbound, and purple for the single westbound segment.

The visualization shows that in September, as cattle began to fatten, temperatures started to increase, and tasajo production kicked off, shipments began to leave the Rio de la Plata. They arrived in Cuba by November or December in anticipation of the cane harvest that would begin after Christmas. The shipments continued until tasajo production and the cane harvest both wrapped up in June. Focusing on the Caribbean during July and August reveals the lack of northbound (green) segments those two months.

Higher Resolution Acanica Videos

Uploading the videos to YouTube, also free, allows sharing them at higher resolutions, up to 1080p if your connection will allow it. Set the resolution with the gear icon once you start the video. See the previous post for details about the animations.

Animated Acancia-Hurricane Interactions

Despite the lack of support for temporal GIS in the free, non-subscription version of ArcGIS Online, ArcMap does support the creation of videos that can be uploaded to free blogs like this one or to video sites like YouTube. To create a video, users zoom to the desired area of the GIS, such as the North Atlantic, and then open the Time Slider window to set controls such as time extent, interval, and window. They then use the Animation toolbar to select Create Time Animation. The Animation Controls and Animation Manager, available through the same toolbar, allow adjustment of variables like the length of the animation in real time and playback time. Export Animation opens a dialog box to set the video format, choose compression options, and save the video.

I exported such a video focused on the North Atlantic with the route and daily position symbols for the Acancia and the HURDAT layers turned on using August 1 through September 5, 1893 as the time extent and 1 day as the time interval and window. The result was AcanciaAug1893.avi, which shows the vessel's day-by-day progress along its route line with hurricanes tracking nearby. Since each 24-hour time window has two noontime position markers, the current and previous day's noontime positions are visible on each frame, allowing users to visualize the daily progress of the Acancia along its route. Similarly, since each hurricane has several positions per day in HURDAT, at 6 hour intervals, the time settings allow users to visualize the evolution of the hurricane tracks through time and space even without the equivalent of a hurricane route line.

AcanciaAug1893.avi was 950 MB in size, however, far in excess of the 100 MB upload limit for this blog. I therefore opened it in Windows Live Movie Maker, included in the most recent versions of MS Windows and available from Microsoft as a free download, to reduce the file size. After adding some title frames, I saved it as AcanciaAug1893.wmv, only 8.3 MB, and uploaded it to this blog.

The result allows readers to view an animated version of the crossing of the North Atlantic by the Acancia in August of 1893 that is fairly identical to what they would see using the time slider in ArcMap. The resolution is lower and the user cannot pan, zoom, or control the time animation as fully as in ArcMap, but the video nonetheless visualizes vessel-hurricane interactions. As the Acancia passed north of Bermuda into the mid Atlantic with its cargo of aguardiente, a tropical storm formed in its wake and tracked northward. To the south, several hurricanes arced westward from the Cape Verdes toward the Caribbean, Florida, and the Atlantic Seaboard. As the Acancia turned southward toward its destination, Montevideo, it sailed directly across that cordon of hurricanes, and on August 26th at noon came within 50 miles (80 km) of the eye of a category 2. After an eastward excursion from its southward course, the Acancia continued southward toward Montevideo.

Since users have much more limited control to pan, zoom, and so on than if ArcMap could be time enabled on ArcGIS Online, I created another video that focuses more closely on the mid-Atlantic encounter between the hurricane, the Acancia, and its crew in late August 1893.

In future posts, I will explore other ways to share the temporal aspects of the GIS on the Web.

The Acancia meets Hurricane 7 of 1893

To more closely examine encounters of particular vessels and hurricanes would benefit greatly with being able to run the voyage and hurricane layers through time, as I can with my ArcMap version of the GIS. While I cannot upload time enabled files to the free version of ArcGIS Online, NOAA has published HURDAT on ArcGIS Online as a map service and as a Web map. The map service is served from the following URL: http://maps4.arcgisonline.com/ArcGIS/rest/services/A-16/NOAA_Hurricane_Tracks-Temporal/MapServer, and anyone can therefore add it as a layer to their Web map using the free ArcGIS Online viewer and display the data through time using the temporal function.

That is what I have done experimentally in the below Web map, reducing the voyages to that of the Acancia in order to simplfy the map during the trial. To add such layers, select Add Data, Add Layer from the Web, and then type the URL for the service into the dialog box. After being added to the Web map, the time control, or "time slider,"  for the NOAA layer appears at the bottom of the map frame. In the version embedded in this blog, select the time button and the time slider will appear as a separate window.

View Larger Map

The problem is that it does not work well because of the enormous amount of data. The HURDAT layer in my ArcMap GIS, which covers the North Atlantic for 1851-1900, has 9366 records. The NOAA map service includes Pacific and Atlantic storms, 1851-2011. When running the time sequence, the rendering takes so long that blobs of hurricane tracks, indistinguishable from one another, jerk across the screen.

I tried filtering the layer to reduce the amount of data being displayed, which is also possible in ArcGIS Online for layers from a map service. I selected each sub-layer in turn, selected filter, and set the Hurricane Season field to 1893, which was the year the Acancia was crossing the North Atlantic in August. Then I set the controls for the time slider to display only 1893 and do it in 365 equal intervals to get a daily interval. Even at an extent, or "time window," of one year, however, the time function does not work well enough to use because the symbols take so long to render that they remain essentially static.

Even the data in the pop-up boxes for the hurricane symbols take a long time to arrive from the server compared to those for the Acancia. Nonetheless, the pop-ups do reveal that the Acancia, shortly after turning southward, had a near encounter with a category 2 hurricane in mid Atlantic on August 26, 1893.

In the next post, I will try another strategy to visualize discrete hurricane-vessel encounters.

The Tasajo Voyages and Hurricane Season

One of the aspects of Atlantic commodity networks like the one that involved Catalan vessels and tasajo that interests me are their seasonal patterns. In what months did the various legs, or segments, of the voyage typically occur? What did seasonal timing have to do with the production schedules of the commodities, such as the months in which various crops were harvested? What did seasonal timing have to do with sailing hazards, such as the hurricanes that characteristically occur in the North Atlantic between early June and late November, with mid August through mid October marking the peak of hurricane season?

Because I am using the free, non-subscription version of ArcGIS Online, at least so far, I cannot implement the time function of my ArcMap version as a Web map. Nor can I do queries and filters with the Web map like I can on my computer in ArcMap. But another way to analyse such patterns in time and space involves applying symbols to represent months or seasons instead of cargoes.

To do that with the objective of visualizing patterns relative hurricane season, I saved the Web map from the previous post (Catalan Tasajo Voyages, 1837-1900) as Catalan Tasajo Voyages: Seasonality. Then I applied new symbols based on the Month field instead of the Cargoes field. For peak hurricane season (roughly August through October, or Months 8-10) I applied red. For the tail ends of hurricane season (June, July, and November, or Months 6, 7, and 11) I applied yellow. And for the rest of the year (Months 12 and 1-5, or December through May) I applied blue.

View Larger Map

The overall pattern indicates that most voyages crossed the North Atlantic during low hurricane season (December-May) or during the tails of the hurricane season (June-July and November). Still, out of fourteen voyages, five have red position symbols in the northeastern quadrant of the during peak hurricane season. By zooming in on the northeastern quadrant of the Atlantic and using the pop-ups to get more information on those five voyages, they become distinct. Two, the Romantico in 1864 and the Prudente in 1884, Left Montevideo bound for Cuba with tasajo in August and September respectively, entered the zone of hurricanes in October, and reached Cuba in November, at the tail end of hurricane season. Two others left Cuba bound for Europe during hurricane season: another voyage of the Romantico, in 1863, with a cargo of sugar and the Pepe in 1851. The Pepe crossed the North Atlantic in late October and November, during the tail of the hurricane season. The Romantico left imprudently in late July and crossed during the August peak of hurricane season. And one, the Acancia, left Cuba bound for Montevideo with a cargo of aguardiente in early August of 1893, sailing first eastward and then southward across the hurricane zone during the peak of the season.

The survival of the logbooks testifies to the lack of a fatal encounter with a hurricane on any of those imprudent voyages, but adding a hurricane layer to the Web map to examine any close encounters between vessel and hurricane will nonetheless be the task of the next post.

Catalan Tasajo Voyages, 1837-1900

This Web map shows the routes of fourteen voyages by twelve Catalan vessels from the mid to late nineteenth century. For the most part, they carried manufactured goods, wine, and other Catalan products from Barcelona to Buenos Aires or Montevideo. There they loaded tasajo, a type of salt beef used as food for enslaved Africans on sugar plantations in Cuba. Once in Cuba, they either loaded sugar or aguardiente, a type of cane alcohol, and returned to Barcelona; or they proceeded in ballast for ports in the US South, like Charleston, and loaded cotton for the textile mills of Barcelona. The GIS includes several variations on that basic trade pattern. See my 2010 article on The Hispanic Atlantic's Tasajo Trail in the Latin American Research Review for more details.

For method and data, see the first four posts of May 2013. The main difference with the previously posted Web map of Dutch slaving voyages is that I did not delete as many of the fields in the Catalan layers of the GIS before uploading them. That slowed the upload process a bit, even causing timeouts on a few occasions because I am working from my home office on a relatively slow connection at the moment) but it transferred more of the data in the GIS to the Web map. The fields I did delete were related to weather observations, included in the GIS because of all the weather data in the CLIWOC database but mainly empty fields regarding the Catalan voyages because when I transcribed those logbooks I had limited time, did not need the detailed weather observations for the research project I was working on, and so focused on the latitude, longitude, and cargo data.

I will use this Web map as the basis for others in the next several posts.

View Larger Map

The project GIS and website are © 2013 by Andrew Sluyter but open source and licensed through the Creative Commons as attribution-noncommercial 3.0, which allows others to use the data and programming to produce non-commercial derivative products. I collected the data in four archives in Spain while funded by a 2009-10 Louisiana Board of Regents ATLAS grant: Arxiu Històric Municipal del Masnou, Arxiu Històric Municipal de Sitges, Biblioteca de Catalunya, and Museu Marítim De Barcelona.I processed the data and built the GIS while a 2012-13 Digital Innovation Fellow of the American Council of Learned Societies. As with the other Web maps on this blog, anyone is free to use it as a basis for new, non-commercial, scholarly and creative products as long as each such GIS, website, and/or publication contains a statement that acknowledges Andrew Sluyter as this GIS's creator.

Websites with Info on Vessels

These are some useful Websites with information and images about specific sailing vessels:

1. Maritiem Digitaal: www.maritiemdigitaal.nl
2. Sailing Ships: www.sailing-ships.oktett.net
3. The Ships List: www.theshipslist.com
4. Three Decks - Warships in the Age of Sail: www.threedecks.org

Web GIS

Web GIS

To create an initial implementation of the Web GIS for the project, I used the free, non-subscription version of ArcGIS Online, not only because my institution does not have a subscription to ArcGIS Online but because I want to make this project as useful as possible to people with limited resources. Anyone with a broadband connection and a computer can now use the Web GIS published in the second post to this blog as the basis for their own Web GIS by modifying it, adding to it, deleting parts of it and saving their creation to their own free account on ArcGIS Online.

The first step in publishing the GIS on ArcGIS Online involved converting the feature class files, both line and point, to a format that the free version of that software accepts. In order to reduce the work involved in what was a test, being my first experience with web mapping, I copied the GIS and renamed it Slave_Voyages_WebGIS. Since I had used relative paths, the copy worked perfectly. I then used a select by attribute query to find all the voyages that carried enslaved Africans and deleted all the layers for other voyages as well as the hurricane layer because the slave trade had ended by the time the hurricane record begins in 1851. Then, to reduce the size of the files in the Web GIS and ensure it ran smoothly, I used the Delete Field tool (in Data Management Tools, then Fields) to delete all fields except VesselID, Flag, DayOfWeek, Latitude, Longitude, TSTDURL, Cargo, and DATE from the attribute tables of the point Feature Class files. Then I used the Feature Class to Shapefile tool (in Conversion Tools, then To Shape file) to export all the point and line Feature Class files from the geodatabase to a new folder. Opening that folder reveals that each feature class exports as several files, including the essential file.shp, file.prj. file.dbf, and file.shx files. Each includes essential information such as the projection, spatial reference, point or line locations, and attributes. Selecting all the files associated with a point or line feature class, right clicking, and selecting Send to Zipped Folder creates a compressed folder that the free ArcGIS Online accepts as an upload, as long as it does not contain more than 1,000 features.

Next, I started a new map from within my free ArcGIS Online account by selecting the Map link, then the New Map button, and then the Basemap button and adding the same Terrain base map I used in ArcMap. I then saved the new map to My Content with the name Dutch and British Atlantic Slave Trade Voyages, 1751-1795. Once that GIS had been saved to the ArcGIS cloud, I was able to add all the point and line layers for the 48 slaving voyages by selecting the Add Content button, selecting Add Layer from File, navigating to the folder of shape files on my computer, and uploading each in turn.

Those shape files do not contain the symbology and other features, such as the visibility range, of the layers in the GIS from which they were exported, necessitating a lot of work to reestablish appropriate symbols to indicate cargoes and so on within ArcGIS Online. That is accomplished laboriously but straightforwardly through the Contents panel of the  Web Map and a menu that appears when the arrow to the right of each layer is selected.

The popup box associated with clicking on each feature, line or point, is particularly useful in a Web GIS because it can include URL links to other Websites. Most basically, I configured the popup boxes for the point symbols for a particular voyage to include the URL for the search window in TSTD for that same voyage, allowing users to immediately connect the data and representations of one project to those of the other. I also ended up importing the Ports_and_Landmarks shape file to identify places relevant to the slave trade, from landmarks like Cape Mount, to ports like Paramaribo, to archives like the Zeeuws Archief in Middelburg that hold the logbook data on which the GIS is based. Each popup can contain a relevant image and “Get more info.” link to, for example, the Website of the archive or a Wikipedia entry on the place. I also added popup boxes to some of the lines that represent the routes of the vessels, in this case linking to the shipping news sections of online newspaper archives so that clicking on the line representing a specific voyage brings up an image of the newspaper that contains a link to the reference in its shipping news section for that specific voyage.

I also wanted to represent one of the patterns that became clear in the ArcMap version of the GIS, namely the pattern of deaths of enslaved Africans during the Middle Passage and the disposal overboard of their corpses. The Dutch logbooks recorded the number of deaths on each day of the crossing, making it possible to visualize that horrific aspect of the slave trade. To do so, I used the Merge tool in ArcMap (in Data Management Tools, then General) to generate a single layer that combined all the individual voyages but eliminated all fields except for VesselID, Cargo, and Occurrences, the last of which contained the number of enslaved who died on each day at each position, from a low of 0 to a high of 5. That created a new point Feature Class File with 8,889 records, or rows in the attribute table. I used various tools (Add Field, Find & Replace, Field Calculator, and Sort) available through the Table Options button in the top-left corner of the attribute table window or through right clicking the column heading to create a new short-integer field called SlaveDeaths and populate it with the number of slaves who died at each latitude, longitude on a particular day. I then eliminated all records with zero deaths and ended up with a layer with only 389 records that I could export, compress, and upload to ArcGIS Online in the same way as the other layers. Once added to the Web map, I assigned different diameter point symbols to each number of deaths. The pattern is striking enough with that representation, but I also wanted to visualize it as a continuous density surface.

To represent the pattern of deaths as a density surface, I first ran the Kernel Density Tool (in Spatial Analyst Tools, then Density) in ArcMap. It calculates the density of occurrences, deaths in this case, around each data point, the vessel positions in this case, and creates a floating point raster layer as its output, in other words, a layer that consists of square cells that each have a value such as 0.349759 or 2.897347. The problem was that the free version of ArcGIS Online does not allow you to upload raster layers. The raster layers must already exist within ArcGIS Online, for example, the base maps, which are raster layers. I used three additional tools to work around that limitation. First I used the Raster Calculator Tool (in Spatial Analyst Tools, then Map Algebra) to multiply all the raster values by 10,000, which changed the output from deaths per square kilometer to deaths per 10,000 square kilometers but would stop the next tool from truncating most of the significant figures. That next tool was the Int Tool (in Spatial Analyst Tools, then Math), used to convert a floating point raster file to an integer raster file by truncating all of the cell values. And the last tool was the Raster to Polygon Tool (in Conversion Tools, then From Raster), which converts integer rasters, but not floating point rasters, to a series of lines that define polygons. Once that polygon layer had been created, I exported it as a shape file, compressed the results into a zipped folder, uploaded it to ArcGIS Online, and applied colors to each polygon to represent different values of deaths per 10,000 km².

The free version of ArcGIS Online does not support uploading temporal layers, so the voyages cannot be run as a temporal sequence in the Web GIS. But future posts will explore solutions to that issue.

In the meantime, here is a prior temporal animation of the CLIWOC database for purposes other than historical climatology; here is another; and here is a nice static representation as well as one that shows seasonal patterns.

The Project GIS

 The Project GIS

To create the GIS, I used the ArcMap and ArcCatalog components of ArcGIS 10, for which my institution has a site license. I have been using GIS on and off since the late 1980s. Over the past two decades, GIS has gone from typing in commands to selecting them from pull-down menus, from relatively little functionality to high functionality, and from static dot-matrix printouts to high quality, dynamic output on the web. In parallel, the data sources have gone from having to convert them to digital format yourself, to a limited range becoming available on floppy disk, to many becoming available on the web in digital form. The latest innovation, and the one that prompted me to get into a relatively large GIS project for the first time, has been the addition of a temporal function to ArcGIS, finally making it quite easy to display changes in spatial patterns over time. Its productivity has not kept up with its impressive functionality, necessitating a lot a mouse clicks and keystrokes to accomplish many tasks that could be much better integrated, and its temporal functions remain limited compared to its spatial ones. But ArcMap has reached the level of utility that makes it essential to my research and instructional programs and, in fact, for anyone who does historical research.

The first stage of turning the prior work with the databases into a GIS involved setting it up and importing the data. I will be a bit more step-by-step about this stage because I assume those interested in the digital humanities have less experience with ArcGIS than with Excel.

For the initial setup of the GIS, I created a folder named ACLS_GIS to keep the project in, started ArcMap, used the File menu to start a New GIS, and saved it in the folder as ACLS_GIS.mxd. Then I went to the Customize menu and selected ArcMap Options to make sure that “Make relative paths the default” was selected, which makes it easier to move the project to another folder or computer. At the same time, you can check around the other tabs of the ArcMap Options dialog box to set other options to your personal preferences. Then I used the Windows menu to open the Catalog window, navigated to the folder ACLS_GIS, right clicked in it, selected New, selected File Geodatabase (not Personal Geodatabase, which is more limited), and named the resulting file ACLS_Geodatabase.gdb. That database, indicated by a special icon, has a specific format used by ArcGIS, will contain many of the files needed for the GIS, and will keep track of them and their relationships. The last step of the setup involved going to the File menu, selecting Add Data, selecting Add Basemap, and selecting one of the base maps that appear, everything from a basic gray outline to imagery from the Bing Maps Aerial view. The base map is the underlying map on top of which the data will be represented, and it can be changed for one of the other base maps at any time. I selected the Terrain base map, which includes four sub-layers named Reference, Borders and Places, Shaded Relief, and Terrain. For reference in the next step, the base maps use the Web Mercator projection of the WGS 1984 geoid, also known as the World Mercator; its projection reference file is named WGS_1984_Web_Mercator_Auxiliary_Sphere, and its datum reference is named D_WGS_1984 (more on this in the next step).

The next step involved adding the data. To add the worksheets in the Excel workbook named ACLS_Voyages_Data.xlsx to the GIS, I used the Folder Connection button (a folder icon with + on it) at the top of the Catalog Window to navigate to the folder that contained the workbook and establish a link to the ACLS_GIS folder and its ACLS_Geodatabase. I then opened each of the worksheets in turn (for example, Middelburgs_Welvaren_A$), right clicked it, and selected Create Feature Class From XY Table to open the dialog box that adds the data to the GIS. In the dialog box, I selected the longitude column as the X coordinates, the latitude column as the Y coordinates, and GCS_WGS_1984 as the Geographic Coordinate System of the Input XYs. Next, under Advanced Geometry Options, I selected Use Map Spatial Reference, which was the previously mentioned WGS 1984 Web Mercator. Those selections told the GIS the geographic reference of the data being added and the way in should project them onto a flat map. Without that information, the GIS has no idea if all those latitude and longitude numbers in the worksheet are in degrees, meters, kilometers, or something else; and it has no idea how to project those positions, which were taken from the curved surface of the Earth, onto a flat map. Then, still in the dialog box, I navigated to the ACLS_Geodatabase and saved the resulting Feature Class Files with the name of the vessel (for example, Middelburgs_Welvaren_A). Opening the ACLS_Geodatabase shows all of those files with an icon that indicates they are point features. After creating point feature class files for each of the voyages, I did the same for the Ports_and_Landmarks worksheet. (Note that if ArcMap does not recognize an Excel worksheet, saving it in an older version of Excel typically resolves the issue, in other words, as file.xls instead of file.xlsx.)

The next step results in all those data actually appearing on top of the base map, a gratifying moment in any GIS project. I used ArcMap’s Windows menu to open the Table of Contents window, selected the List by Drawing Order button at the top of that window (on the far left), went to the open ACLS_Geodatabase in the Catalog Window, and dragged each of the Feature Class Files to the Table of Contents under layers, in alphabetical order and on top of the base map. The order of the layers in the Table of Contents represents the order of the layers on the map, with all the voyages therefore drawing on top of the base map. All the voyages then appeared as a series of linear point patterns, the overall pattern emphasizing the triangular routes of the slave trade in the North Atlantic. Checking or unchecking on any of the layers in the Table of Contents turns that layer on an off. Dragging them rearranges their order in the Table of Contents and on the map. Right clicking any of the layers and selecting Open Attribute Table opens a table that shows all the cargo, date, latitude, longitude, and other fields associated with each of the features, or daily positions, in that layer. Selecting a particular day or range of days in the attribute table, selects its point symbol on the map and vice versa.

Once all the data have been imported into the GIS and represented as point symbols, the next stage involves the use of various tools to alter the data layers and derive entirely new types of layers from them. Those tools are accessible in ArcMap by selecting the Geoprocessing menu and then ArcToolbox or by starting ArcCatalog.

The first tool I used was the Convert Time Field tool to get the dates of each position into a standard ArcMap format to use the temporal function. I opened the tool (in the ArcToolbox under Data Management, then Fields), navigated to and opened the ACLS_Geodatabase, selected each of the voyage Feature Class Files in turn, selected their date field, which was in a yyyymmdd format, and converted them to the ArcMap, standard time format. Opening the attribute table revealed that the tool has added a new field, DATE, to the attribute table and populated it with dates in the correct format while leaving the original yyyyddmm date field intact.

Next, I used the Points to Line tool to create lines to underlay the points that defined each voyage, which I believed would better define the route of each voyage by connecting its daily positions and increase the symbolization, labeling, and other visualization opportunities. I opened the tool (under Data Management, then Features), navigated to and opened the ACLS_Geodatabase, selected each of the voyage point Feature Class Files in turn, selected the field that contained the voyage name as the Line Field, and saved the resulting line Feature Class files in the ACLS_Geodatabase, naming them for the voyage with _line appended (for example, Middelburgs_Welvaren_A_line). I then dragged each of the line Feature Class Files to the Table of Contents, placing them immediately underneath their respective point layers. A check of each line layer’s attribute table reveals that it contains a field with the voyage name, which can later be used to label the line. Then I selected Start Editing from the Editor menu and used the split tool to cut each line at the beginning and end of the segments of the voyage, deleting the superfluous segments: for example, the segments that cut overland across West Africa from the coast lf Liberia to the coast of Ghana because of the missing days and positions while the vessel followed the coast trading for slaves. This changed the attribute table for each line by adding a record for each new segment. Finally, in the Table of Contents, I selected both the point and line layers for each voyage, right clicked, selected Group from the menu, and combined them as sub-layers of a single layer, naming it after the vessel and voyage (for example, Middelburgs_Welvaren_A).

I then customized the way in which the GIS represented each voyage by right clicking each layer and selecting Properties. The dozen tabs of that dialog box allow you to set everything from how the temporal display works, to the style and color of the symbols, to the labels, to the scales (zoom levels) at which the layer is visible. To set the time function for the point layers, for example, I selected the Time tab, enabled time, selected the DATE field, and set the time interval at 1 day. To label each point or line segment, I used the Label tab, selected label features, selected the date field for the points layers and the voyage name layer for the line layers, and set the scale range at which the labels would come into and out of view. To connect to TSTD, I used the HTML Popup tab to select the TSTDURL field to open a search results window for a particular voyage in that database’s website whenever someone selected one of the point symbols for that same voyage in the GIS.

The most time intensive task turned out to be setting the symbols that would represent each daily position. To do that, I used the Symbology tab in the Properties dialog box. Since I wanted each daily position to reflect the cargo, I selected Categories, Unique Values, and the Cargoes field in the attribute table. I then assigned standard colors to each type of cargo: gray for unknown, brown for tasajo, and so on. In addition to the standard symbols included in ArcMap, users can create their own symbols and save groups of symbols under unique headings so the process becomes somewhat more productive.

By that point, the GIS had much of the data and functionality I had planned. With all layers visible, it displayed all the voyage segments as lines overlain with the daily position points, each with a unique symbol that indicated the cargo. At broad scales all the voyages blended together and large-scale patterns were evident, such as the triangular trade between Europe, Africa, the Americas, and back. Zooming in revealed individual voyages, their lines labels by name and their points labeled by date. Queries to search for particular attributes (Selection menu, then Select by Attributes) could be run to select all voyages that carried, for example, enslaved Africans. And, when time was enabled by selecting Open Time Slider Window (the clock icon on the main menu bar), and the GIS run through its temporal extent, the point symbols associated with each day appeared and disappeared in sequence. Selecting Time Slider Options from the Time Slider Window menu bar allowed changes to the length of the period represented, the speed at which it ran through that period, and so on. Such functions were enhanced by running the Add Attribute Index tool (under Data Management, then Indexes) for each of the Feature Class Files so that ArcMap could quickly locate records.

The final major task to ready the GIS for use involved correcting the positions added in Excel to represent missing days, latitudes, and/or longitudes as well as check the integrity of each of the other daily positions for each voyage. I accomplished that objective by advancing through each voyage, day by day, position by position, with its attributes table open and editing turned on. Positions that had been added for missing days with last known latitudes and longitudes, for example, where selected in the attribute table and dragged to interpolated positions between known ones. Position makers that appeared too close to shore were moved off shore on the bases of the attribute table information, which sometimes named a landmark, its bearing, and its distance. Position markers that had been given in the logbooks but appeared erroneous relative the preceding and following days were also corrected at this point, for example, when a logbook reported a positive latitude as a negative one or vice versa, as sometime happened near the equator.

Once all the voyage data were complete and correct as far as the available data allowed, the hurricane Excel worksheet was converted to a points Feature Class File and added as a layer in the Table of Contents. I used the ArcToolbox to create a DATE field, time enabled the layer, and applied symbols to differentiate among categories of storms.  Even reduced to 1851-1900, the attribute table had 9366 records.

By this point the GIS was highly functional. It made it possible to query intercepts in time and space between hurricanes and vessels. It revealed the pattern of deaths and disposals overboard on Dutch slaving voyages, the captains of which have left us a daily record of such deaths in the logbooks. And therefore the time had arrived to begin implementing it, or at least parts of it, as a Web GIS.

Selected References

1. Bodenhamer D. J., J. Corrigan, and T. M. Harris, eds., The Spatial Humanities: GIS and the Future of Humanities Scholarship (Bloomington: Indiana University Press, 2010).
2. Gregory, I. N., and P. S. Ell, Historical GIS: Technologies, Methodologies and Scholarship (Cambridge: Cambridge University Press, 2008).
3. Hunter, Richard, Methodologies for Reconstructing a Pastoral Landscape: Land Grants in Sixteenth-Century New Spain, Historical Methods 43 (2010): 1-13
4. Knowles, Anne Kelly, ed., Past Time, Past Place: GIS for History (Redlands, California: ESRI Press, 2002).

The Project Database

The Project Database

 

After that initial processing of CLIWOC, TSTD, HURDAT, and my database of Catalan voyages, I developed the project database as an Excel 2010 workbook named ACLS_Voyages_Data.xlsx. It contains 73 voyages by 41 vessels between 1752 and 1900 sailing between Europe, Africa, the Caribbean, South America, North America, and Asia carrying cargos of coffee, sugar, spices, gold, and many other products as well as enslaved Africans. The daily position and cargo data for 15 of the vessels making 17 of the voyages came from my database of Catalan voyages while that for the other 26 vessels and 56 voyages came from CLIWOC. Of the 41 vessels, 27 made a single voyage; 4 made 2 voyages; 4 made 3 voyages, 4 made 4 voyages, and 2 made five voyages. Vessels making multiple voyages were designated by different letters for each voyage: for example, Middelburgs_Welvaren_A versus Middelburgs_Welvaren_B.

The following table shows the breakdown by nationality.

Nationality	Number of Vessels	Number of Voyages	Slaving Voyages	Slaving Voyages Also in TSTD
British	4	4	2	2
Dutch	22	52	46	42
Spanish	15	17	0	0
TOTAL	41	73	48	44

Four of the Dutch slaving voyages do not appear in TSTD because they did not carry slaves across the Atlantic. The Zephyr was a naval brig that sailed from the Netherlands directly to Suriname; while on the Berbice River in Guyana in February 1764, the Zephyr took aboard 72 slaves from the Sieben Provinzen, a vessel name that does appear in TSTD but not for any voyages during the 1760s. Two of the three voyages of the Mercurius do not appear in TSTD because after taking aboard some slaves in Africa, the captain traded them and returned directly to the Netherlands with cargoes of ivory and gold. Similarly, one of the two voyages of the Granadier initially took aboard enslaved Africans but returned directly to the Netherlands with a cargo of ivory, wood, and wax.

Two of the British and two of the Spanish voyages did not carry cargoes at all because they were voyages of scientific exploration. I chose to include them because of their potential interest. The British ones were James Cook’s first two voyages: in the Endeavor, 1768-1771, and the Resolution, 1772-1775. The Spanish ones concerned the Alessandro Malaspina expedition of 1789-1794, involving two ships named the Descubierta and the Atrevida.

Another of the Spanish voyages was by the Astuto from Peru to Spain in 1778 with a cargo of coinage, jewels, gold, silver, copper, tin, cacao, cacao hulls, chocolate, and sugar. During the voyage, ten people kept separate logbooks and thereby generated ten separate records of their position that I chose to include because of the potential to evaluate the relative precision of their estimates.

Before importing the database into the GIS, I used Excel’s standard tools such as fill, find and replace, and sort to modify it in various ways because of the much higher productivity possible with Excel than trying to make the modifications once in ArcGIS. I standardized spellings of vessel names, captain’s names, place names, and so on, typically to conform modern usage. I added underscores to join words, for example, changing Drie Gezusters to Drie_Gezusters to avoid problems with ArcMap recognizing field names. Because CLIWOC had data on the number of crew members, tonnage, armaments, and other interesting information but presented it in a single field called OtherRemarks, I added separate fields for CrewSize, Tonnage, Armaments, and so on. And I added a field named DayOfWeek and populated it using the fill series tool and the calendar calculator at http://www.timeanddate.com.

Once all of those the changes had been made, most productively done in a single worksheet, I saved each voyage as a separate worksheet within the workbook because the free version of ArcGIS Online will only accept files of limited size, up to a maximum of 1,000 features. Since each voyage included several hundred features, each one representing a daily position on a particular date with associated attributes such as cargo, I would not be able to add them to ArcGIS Online using the free, non-subscription version as a single file.

A remaining issue involved the lack of vessel positions for some vessels while they were in port. I used the Geodata table in CLIWOC21.mdb to create another worksheet called Ports_and_Landmarks to locate the sometimes obscure places noted in the logbooks. The Geodata table provided an initial list of places and their latitudes and longitudes for that worksheet. I then added more places to it, especially regarding the Catalan voyages and the places along the coasts of West Africa and the Gulf of Guinea. David Eltis and David Richardson, An Atlas of the Transatlantic Slave Trade (New Haven: Yale University Press, 2010) proved helpful in locating obscure place references along the African coast. I used GoogleEarth to determine their latitudes and longitudes in +/- decimal degrees and copied/pasted them into the Ports_and_Landmarks worksheet. That worksheet then allowed me to populate the empty latitude and longitude cells for vessels in specific ports with the latitudes, longitudes for those ports.

A final remaining issue to ready the database before importing it into ArcGIS concerned missing days, latitudes, and longitudes. These occurred in three types of instance.

The first type of instance concerned entirely missing segments of the voyage. For example, many of the slaving voyages made their first African landfall along the coast of West Africa between present-day Senegal and Liberia, followed the coast southward and eastward while trading for enslaved Africans, and then departed for the Americas from a port in the Gulf of Guinea, such as Elmina in present-day Ghana. The logbooks generally did not record latitude and longitude positions while following the coast because sailors navigate by coastal landmarks when they can. Often there are no entries at all; sometimes a general description of the itinerary is given as a single entry that encompasses weeks or months of coastal sailing and trading; and in a few logbooks, landmarks and anchorages are noted for particular dates. For entirely missing segments, I simply ended with the last record of the previous segment, such as from Fort Rammekens, Netherlands to Cape Mount on the Liberian coast, and resumed with the first record of the next segment, such as from Elmina, Ghana to Paramaribo, Suriname. For segments that gave landmarks on particular days and places that the vessel anchored for, sometimes, weeks at a time, I added those days and used the latitudes and longitudes of those landmarks, anchorages, and ports in my Ports_and_Landmarks worksheet to add the missing positions for that segment.

The second instance concerned single days that were clearly missing from particular voyage segments. For example, if a day was missing in mid Atlantic from a segment of a voyage from the Gulf of Guinea to Suriname, I simply added that day and used the last known latitude and longitude as the position. In cases that the day was present but either the latitude, longitude, or both were missing, I again used the last known latitude, longitude, or both. Once the voyages were mapping in the GIS, I knew it would be possible to use the interactive editing tool to select those position markers and drag them into what amounted to a best estimate, interpolation of the missing position. In other words, the missing day, latitude and longitude would be dragged to a position intermediate to the position of the day before and the day after.

The third instance concerned days and positions, often multiple, at the ends of segments. A common instance of this occurred when vessels approached Europe on the homeward segment of a voyage. Sometimes the last entry in the logbook recorded sighting The Lizard, a lighthouse on Lizard Point that marked the Atlantic end of the English Channel. In other logbooks, The Lizard also marked the last latitude, longitude entry but other types of entries followed. Sometimes a single entry gave the date of arrival in a port, such as Fort Rammekens. Other logbooks included multiple entries that gave sightings of key landmarks as the vessel progressed up the Channel, such as Portland, the Isle of Wight, and Dover. For the first type, I ended the segment at that point, for example, at the Atlantic end of the English Channel. For the second type, I added the days between the end of the latitude, longitude entries to arrival in a specific port on a particular day and used the latitudes, longitudes of that port, of landmarks given, and of either the last known or next known position. Again, once the voyages were mapping in the GIS, I knew it would be possible to use the interactive editing tool to select those position markers and drag them into what amounted to a best estimate, interpolation of the missing position.

(The next post will deal with importing the database into the GIS.)

Methods and Data

As promised, after some initial posts introducing the project and displaying some results, the next few posts will concern data and methods.

PROJECT METHODS AND DATA

Some of the methodological choices I have made for this project derive from the terms of the Digital Innovation Fellowship from the American Council of Learned Societies: for example, the choice to disseminate the products as broadly and freely as possible via the Web and to use the project as an opportunity to encourage and teach others how to undertake other such digital humanities innovations. Other choices relate to funding limitations, such as the choice to use this free blog to disseminate the results because the fellowship mainly provides salary replacement for one academic year rather than research funds to acquire expensive equipment such as a server or pay for ongoing costs such as annual fees. Some of the choices also align with personal values, like the one to use as much free software as possible, such as the ArcGIS Online map viewer, in order to allow anyone with a broadband connection to use and modify the GIS. And yet other choices emerged from the research process itself: for example, my fellowship proposal focused on Atlantic commodity networks in the nineteenth century, but the scope of the project soon expanded to all oceans and the seventeenth century through the nineteenth because once I actually began I realized that producing a worldwide GIS for the 1600s through the 1800s would require only a little more effort and avoid conceptually and technically problematic compromises such as excluding voyages that entered the Atlantic from the Indian Ocean or that began in 1799 but ended in 1801.

Software and Hardware

The principal software packages I used were either free web applications or ones for which LSU had existing site licenses. The licensed ones included Microsoft Excel, Word, and Access 2010; Adobe Photoshop and Illustrator CS6; ArcGIS Server; and ArcMap and ArcCatalog 10. The free ones included the non-subscription version of ArcGIS Online and GoogleBlogger, Maps, Earth, Maps Engine, Sites, and Translate. The free version of ArcGIS Online does not allow the addition of time-enable layers, raster layers, or vector layers with more than 1,000 features. While problematic in some ways, of course, the use of the free version of ArcGIS Online does align with my personal values to allow anyone with broadband connection to use and modify the GIS. To implement all the functions of the GIS on the web, however, required the use a server running Arc Server maintained through the Computer Aided Design and Geographic Information Systems Laboratory (CADGIS) at LSU. An alternative would be a subscription to ArcGIS Online, which allows the same functionality but through cloud computing. LSU, however, does not yet subscribe to ArcGIS Online.

Most of those software packages have extensive help documentation. In cases where that documentation fails to answer questions, a well phrased search of the Web usually yields helpful posts from people who have previously had a similar issue. The following discussions of data sources and methods, therefore, remain relatively general. For step-by-step instructions, see the help documentation for the databases and the specific software packages.

I used a PC and laptop, both with Core i7 processors, to carry out the research.

Data Sources

The data I used are all freely available and, for the most part, broadly accessible.

CLIWOC

The main source of data was the Climatological Database for the World's Oceans, 1750-1850 (CLIWOC), a project funded by the European Union and carried out by a consortium at the Universidad Complutense de Madrid, the University of Sunderland and the University of East Anglia in the UK, the Royal Netherlands Meteorological Institute, and ANIGLA and CONICET in Argentina. The CLIWOC database is freely available for research by anyone and can be downloaded from the project’s website: the “main aim of the project is to produce and make freely available for the scientific community the world’s first daily oceanic climatological database” (http://www.ucm.es/info/cliwoc/intro.htm, accessed December 1, 2012).

The CLIWOC database contains the noon weather observations from 1,674 logbooks that record 4,942 voyages by Spanish, English, French, and Dutch vessels with 975 distinct names, some pertaining to several vessels with the same name, dating to 1750 through 1854. Each weather observation is thereby located by its latitude and longitude on a specific date at noon, allowing climatologists to extend the observational record back in time to well before the establishment of widespread weather stations and climate satellites. CLIWOC 1.0 was released on January 23, 2004 and went through three subsequent versions (1.5, 2.0, and 2.1), each adding records and correcting errors. The last version, CLIWOC 2.1, released on September 25, 2007 and the project has now effectively ended except for continued dissemination of the database. The CLIWOC website also contains an extensive list of references that describe the project, some of them referenced below.

I downloaded CLIWOC 2.1 as a Microsoft Access 2003 database file from the CLIWOC website on November 3, 2011. CLIWOC21_2002-3.mdb contains thirteen tables, with the one named CLIWOC21 containing the bulk of the relevant data in 287,114 rows. The 142 column headings, or fields, are abbreviated but understandable with reference to the detailed explanatory materials on the website and include many variables besides the noontime weather observations and the latitude, longitude: for example, vessel name, type, and nationality; names of the logbook keepers such as the captain and other officers; the prime meridian used; cargoes; landmarks; and origin and destination. The only field not decipherable from the materials on the website related to abbreviations for ownership of the Dutch vessels, some of which (such as WIC for the West India Company) were familiar to me while others were not. An e-mail to Frits Koek at the Royal Netherlands Meteorological Institute, however, rapidly came back with the missing information.

Before using the data in a GIS that suited the particular purposes of my project, I had to delete, combine, and modify many of those fields as well as add some new ones and delete some records. Some of the fields were satisfactory as they stood: for example, the latitudes and longitudes are given as +/- decimal degrees normalized to the present-day prime meridian (the Greenwich meridian) from the hundreds of prime meridians in use before the late nineteenth century. Others required modification or deletion, such as the elimination of the records for many voyages that did not contain any cargo information, achieved by opening the file in MS Access, using the filtering and search functions to create a new table that contained only vessels relevant to the project, and saving the result as CLIWOC21.mdb.

After some similar initial modifications I exported the resulting table as a MS Excel workbook and saved it as CLIWOC21.xlsx to continue modifying the database because I am much more familiar and productive with Excel than Access. I used Excel’s find-and-replace, fill, paste-special, and sort tools as well as the concatenation formula to make various modifications. They include renaming of fields to clarify their meanings; elimination of fields superfluous to my project, such as the one for sea surface temperature; translation of Dutch and Spanish into English; addition of a time field in the format yyyymmdd; and so on.

TSTD

The Trans-Atlantic Slave Trade Database (TSTD) also provided some data, especially the URLs that linked to specific records in that database for vessels carrying enslaved Africans. TSTD contains 34,947 voyages of vessels engaged in the Atlantic slave trade between 1514 and 1866. Unlike CLIWOC, users can not only download the database from the project Website, but also use the Website to search the database by vessel name, origin, destination, and many other fields; generate custom tables and graphs; and view summary graphs, tables, maps, and other representations. Also unlike the CLIWOC database, TSTD does not record vessel locations other than when in port.

TSTD is licensed under a GNU General Public License and a Creative Commons Attribution-Non-Commercial 3.0 and is thereby freely available to copy, distribute, transmit, and adapt the work for non-commercial purposes as long as the data source is attributed to the TSTD and the product is itself is open source. The database and its SPSS codebook are also available to download from the TSTD website as a comma-delimited spreadsheet, but on November 30, 2011 it contained fewer records than available through the search engine of the updated, online database.

To determine which of the CLIWOC voyages also appeared in the TSTD, I used its website search function on November 30, 2011 to create a comma-delimited spreadsheet of all vessel names, downloaded it, opened it in Excel, and named it TSTD_and_CLIWOC_matches.xlsx. The spreadsheet had 34,947 rows, each recording a voyage, with some vessels represented several times because they made more than one voyage. Each row names the vessel as well as, ideally, its captain, the year of the voyage, its origin and destination, the number of slaves aboard, the outcome of the voyage, and so on. I then imported the ShipName, VoyageFrom, VoyageTo, and Name1 (which records the captain’s name) fields from the CLIWOC21 table of CLIWOC21_2002-3.mdb into TSTD_and_CLIWOC_matches.xlsx. Excel’s conditional formatting function then identified all the CLIWOC and TSTD records that had matching vessel and captain names. Parsing the years, origins, and destinations of the matches identified identical voyages in CLIWOC and TSTD. Also, I carefuly checked the spellings of vessel names once I noticed that CLIWOC had retained the spellings of the logbooks while TSTD modernized them: for example, Drie Gezusters instead of Drie Gesusters, Enigheid instead of Eenigheijt, and so on. In the end I had a list of forty-four voyages that appeared in both TSTD and CLIWOC. Since some of the CLIWOC records involved did not have cargo data and I had therefore already eliminated them from my original iteration of CLIWOC21.xlsx, I imported the relevant fields for those records from CLIWOC21_2002-3.mdb into CLIWOC21.xlsx. I also added the data on slaves from TSTD to the cargo field of CLIWOC21.xlsx and created a new field named TRTDURL for the URL that linked to the TSTD search result.

Database of Catalan Voyages

This database includes the voyages of several Catalan vessels, 1837-1900, developed for one of my previous projects: Andrew Sluyter, Black Ranching Frontiers: African Cattle Herders of the Atlantic World, 1500-1900 (Yale University Press, 2012). It derives from my research in four archives in and around Barcelona, Spain in 2010: Arxiu Històric Municipal del Masnou, Arxiu Històric Municipal de Sitges, Biblioteca de Catalunya, and Museu Marítim De Barcelona. From logbooks preserved in those archives, I transcribed the daily noontime position, cargo, and some other data related to tonnage, crews, and captains, but generally not the weather observations, for 21 voyages. For this project I eliminated the voyages without cargo data, leaving 14 voyages by 12 different vessels between 1837 and 1900.

These data are as free as those of CLIWOC and TSTD, although not as widely available. Anyone can go to the relevant archives in Spain and ask to see the logbooks to transcribe the data. But they were not widely available in the same sense as the other databases because they previously could not be downloaded from the Web in a digital format.

HURDAT

Because one of the purposes of the project was to demonstrate how Atlantic voyages interacted with their environment, I also used the North Atlantic Hurricane Database, 1851-2011 (HURDAT). The Hurricane Research Division of the Atlantic Oceanographic and Meteorological Laboratory of the National Oceanic and Atmospheric Administration developed this database, which contains hurricane track and intensity data for the North Atlantic from 1851 through 2011. The National Hurricane Center originally developed HURDAT for 1886-1983, as described in Brian R. Jarvinen, Charles J. Neumann, and Mary A. S. Davis, “A Tropical Cyclone Data Tape for the North Atlantic Basin, 1886-1983: Contents, Limitations, and Uses,” NOAA Technical Memorandum NWS NHC 22 (Miami: National Hurricane Center, 1984). The Atlantic Hurricane Database Re-Analysis Project has since revised HURDAT to extend it back to 1851, continue it forward to the present, and revise all tracks and intensities using additional historical meteorological data, updated models of hurricane behavior, and more sophisticated computing techniques, as described in publications such as C. W. Landsea, C. Anderson, N. Charles, G. Clark, J. Dunion, J. Fernandez-Partagas, P. Hungerford, C. Neumann, and M. Zimmer, “The Atlantic Hurricane Database Re-Analysis Project: Documentation for the 1851-1910 Alterations and Additions to the HURDAT Database,” in Hurricanes and Typhoons: Past, Present and Future, R. J. Murname and K.-B. Liu, eds. (New York City: Columbia University Press, 2004), 177-221.

HURDAT is freely available for non-commercial use, with the stipulation that derivative products must acknowledge the National Oceanic and Atmospheric Administration as the data source: “We ask that a proper acknowledgement to the ‘NOAA Hurricane Research Division of AOML’ accompany the use of these data in any publications or presentations” (http://www.aoml.noaa.gov/hrd/data_sub/datapolicy.html, accessed December 1, 2012).

HURDAT has already been published on ArcGIS Online by NOAA, both as a map service and as a Web map. Details are at http://www.arcgis.com/home/item.html?id=bec3dbc25db14a848427b7f14800395b for the Web map, and at http://www.arcgis.com/home/item.html?id=d9b8b9a1b09b4038813954a1be7043cc for the web service, which is served from the following URL: http://maps4.arcgisonline.com/ArcGIS/rest/services/A-16/NOAA_Hurricane_Tracks-Temporal/MapServer. Anyone can therefore add this layer to their Web map using the free ArcGIS Online viewer and display the data through time using the temporal function. Nonetheless, that choice would not allow control of which specific hurricanes to use in my project, nor how to represent them. Therefore, I chose to download and modify the HURDAT database myself so that I could add hurricanes that I chose and represent them in ways that supported the goals of the project.

On November 30, 2012 I downloaded the file named easytoread-spreadsheet2012-may.xls through the link Excel spreadsheet derived from HURDAT. I used Excel to modify the downloaded file and save it as HURDAT.xlsx for this project. The spreadsheet represents each storm as a series of rows, with each recording the storm position in decimal degrees latitude, longitude as well as its intensity on a particular day at a particular time. The field names are as follows: Month, Day, Hour, Latitude, Longitude, Direction, Speed, Wind, Pressure, and Type. Time is expressed in hours UTC (Coordinated Universal Time), equivalent to GMT (Greenwich Mean Time), meaning the time at the prime meridian.  The time interval between positions is 6 hours: O, 6, 12, and 18 UTC. Positions are given as latitude and longitude expressed in decimal degrees North (N) and West (W). Direction is given as an azimuth, such as 270 degrees. Storm speed and wind speed are both given in miles per hour and kilometers per hour, in separate columns. Pressure is given in millibars but extremely infrequently before the mid twentieth century. Type refers to the intensity category on the widely used Saffir-Simpson Scale.

To prepare the database for the GIS, I used Excel’s find-and-replace, paste-special, and sort tools to make various modifications. I first deleted all records after 1900 so that the worksheet ran from Storm 1 of 1851 through Storm 7 of 1900 (alphabetical naming of storms did not begin until 1950). Second, I added two fields: the first, Name, allowed me to add the storm name to each row, for example, 5 of 1857; the second, Year, added the year of the storm to each row. I also deleted the pressure field; converted the N and W designations of the latitudes and longitudes to + and - preceding the decimal degrees; and converted the month names to numbers, 1 through 12.

Other Databases

Other potential databases were surveyed and explored but not used for various reasons.

For example, The International Comprehensive Ocean-Atmosphere Data Set (ICOADS) is a historical climate database of global marine surface climate observations spanning 1662 to 2008, but its available outputs are gridded at a resolution of 2 degrees latitude by 2 degrees longitude, appropriate as input for historical climate models but not for this project.

Likewise, an inventory of Catalan logbooks by historical climatologists provides data on general itinerary and cargo but not daily position data. The Grup de Climatologia and Grup d’Anàlisi de Situacions Meteorològiques Adverses, Universitat de Barcelona has inventoried 579 logbooks archived in seventeen maritime and other museums in and near Barcelona and begun to assess their potential for reconstructing wind direction and sea condition for the nineteenth-century Atlantic. One of the group, Mariano Barriendos Vallvé, kindly provided me with a copy of that database as a MS Access file (Diarisnavegacio.mdb). That database made it possible to locate relevant logbooks in archives and go to Spain to transcribe them for my database of Catalan voyages.

The Hurricane Research Division of the Atlantic Oceanographic and Meteorological Laboratory of the National Oceanic and Atmospheric Administration also makes available a database similar to HURDAT but for the Eastern and Central North Pacific. I did not use it for this project because it begins only in the mid twentieth century. Its development is described in Mary A. S. Davis, Gail M. Brown, and Preston Leftwich “A Tropical Cyclone Data Tape for the Eastern and Central North Pacific Basins, 1949-1983: Contents, Limitations, and Uses,” NOAA Technical Memorandum NWS NHC 25 (Miami: National Hurricane Center, 1984).

Selected References

1. Elsner, James B., and A. Birol Kara, Hurricanes of the North Atlantic: Climate and Society (Oxford: Oxford University Press, 1999).
2. Eltis, David, and David Richardson, eds., Extending the Frontiers: Essays on the New Transatlantic Slave Trade Database (New Haven: Yale University Press, 2008). 
3. García-Herrera, R., G. P. Können, D. Wheeler, M. R. Prieto, P. D. Jones, and F. B. Koek, CLIWOC: A Climatological Database for the World's Oceans 1750-1854, Climatic Change 73 (2005), 1-12.
4. García-Herrera, R., G. P. Können, D. Wheeler, M. R. Prieto, P. D. Jones, and F. B. Koek, Ship Logbooks Help Analyze Pre-instrumental Climate, EOS 87, no. 18 (2006): 173-180.
5. Können, G. P., and F. B. Koek,. Description of the CLIWOC Database, Climatic Change 73 (2005): 117-130.
6. Prohom Durán, Marc J., and Mariano Barriendos Vallvé, Los diarios de navegación Catalanes: una nueve fuente de datos climáticos sobre los océanos (siglos XVIII a XX), in El Clima Entre el Mar y la Montaña, Juan Carlos García Codrón, Concha Diego Liaño, Pablo Fernández de Arróyabe Hernáez, Carolina Garmendia Pedraja, and Domingo Fernando Rasilla Alvarez, eds., (Santander: Universidad de Cantabria, 2004), pp. 519-28.
7. Prohom Durán, Marc J., El uso de los diarios de navegación como instrumento de reconstrucción climática: la marina catalana del siglo XIX, Investigaciones Geográficas 28 (2002): 89-104.