Transition to Website

1. As the initial phase of the project comes to an end, I will begin constructing a Website to publish the results. The purpose of this blog has been to share results in an ongoing manner. The Website will publish them thematically. The Website is called Atlantic Networks Project and I will work on it incrementally over the next month.

The Slave Trade Map Using ArcGIS Server

Uploading the GIS to one of the CADGIS servers running ArcGIS Server 10.1 and then linking that "map service" to a Webmap on ArcGIS Online has several advantages over uploading the layers directly to ArcGIS Online. The only hitch was that I had to upgrade from ArcMap 10 to 10.1 because the server was running ArcGIS Server 10.1, and I ran into compatibility issues. After doing that, I was able to upload map services to the machine running ArcGIS Server directly from ArcMap 10.1 using the Share as Service command under the File menu.

Advantageously, layers upload together rather then individually without the length limitations involved in uploading directly to ArcGIS Online. Moreover, they retain their symbolization. They do not use up the limited storage allocated to a non-subscription ArcGIS Online account. And, critically for historical researchers, they retain their temporal functionality. When any of the time-enabled layers are turned on in the Webmap on ArcGIS Online, a time slider appears at the bottom of the map.

The biggest disadvantage is cost, of course. Servers and ArcGIS Server are expensive compared to using only a non-subscription ArcGIS Online account.

I constructed the Webmap for this post with my non-subscription ArcGIS Online account by using three of the layers from the non-temporal version of the same map posted at the end of April: (1) the number of deaths of enslaved Africans, (2) the density of those deaths, and (3) the place markers with their pop-ups. To those layers, I added three map services by using the Add Data button, selecting Search for Layers in a GIS Server, and typing in the address of the CADGIS server on which I had previously created the map services by uploading layers from ArcMap: (1) the voyage route lines, (2) the daily position points without being time enabled, (3) and a time-enabled version of the daily position points. The data for those layers remains on the server but is rendered as needed on the Webmap, similarly to the way the ArcGIS Online basemaps work.

I then embedded the Webmap below using the Share menu in ArcGIS Online. When I did that, I had turned on only the time-enabled layers for vessel daily noontime positions, set the time extent to 1751-1795, and the moving time window to 1 month. Lines of up to 31 points therefore snake around the Atlantic, one point for each sailing day in the month. Clicking the clock symbol in the top menu bar turns the time function on and off. Zooming in reveals the place markers and their pop-ups, date labels on the position symbols, vessel name labels on the route lines, and so on.

To access more functionality, such as turning layers on or off, users can click the link that appears below the map, which opens it in ArcGIS Online. There, users can see the list of layers and turn them on or off in order to, for example, display the sublayer of a single voyage of particular interest, set the time extent for the period of that vessel's voyage, and reduce the length of the time window. Users can also turn on the layers that show the number of deaths of enslaved Africans at a particular location on a specific voyage as well as the density of deaths across the breadth of the Middle Passage. For the layer called All Positions and Cargoes, which is not time enabled, clicking on points or lines brings up pop-ups with vessel information and links to the Trans-Atlantic Slave Trade Database records for those vessels, similarly to the non-temporal version of the same map posted at the end of April. The Middelburgs Welvaren's pop-up even has its image, taken from its logbook.

Note that sometimes the CADGIS server is down or slow to respond, so please be patient, try reloading the page, come back another day.... I am working on funding to acquire a new server and so on.

View Larger Map

The Slaving Voyages KML in GoogleMaps

GoogleMaps can also be used directly, without using Map Engine Lite to upload a spreadsheet for each layer. To create a map of the slaving voyages for the second half of the eighteenth century, go to GoogleMaps, select My Places, and then Create Map. Next, either upload the compressed KML file (KMZ)  generated with GoogleMaps (see previous post) or enter the URL for its location on GoogleSites. It, in theory, is then rendered over the default GoogleMaps content such as Wikipedia and photo links. The base map can also be changed to the  satellite or GoogleEarth view within GoogleMaps, but the time slider is not enabled.

In practice, the KML file uploads but does not render the voyages all at once or in chronological order. The voyages display in twenty-five separate "layers." By opening the map in GoogleMaps by clicking the link below the embeded version, users can advance through the layers with the index at the foot of the panel that displays daily position symbols, dates, and TSTD links.

View The Slaving Voyages KML in GoogleMaps in a larger map

Opening the Web map with GoogleMaps also allows users to save it to their own My Places and modify it in various ways such as adding the commonly used place markers or pushpins as well as lines and polygons or descriptive attributes in pop ups.

Time Enabled Slaving Voyages on GoogleEarth

GoogleEarth provides another free option for publishing Web maps, one that provides an alternative to videos for sharing the temporal function of ArcMap. GoogleEarth 7 can be downloaded gratis here. It is an application that runs on and saves your data to your computer, not the cloud.

I created the below temporal Web map of the British and Dutch slaving voyages by converting the ArcMap GIS file (MXD file) into a KMZ file (a compressed KML file) using the Map to KML Tool found with the other Conversion Tools. GoogleEarth and other Web mapping applications use KML (Keyhole Markup Language) to render point, line, and other types of features. That KMZ file therefore includes not only the features and their attributes but the way I symbolized and otherwise configured them as layers in ArcMap.

Next, I opened the KMZ file with GoogleEarth and configured the time slider settings and symbols to suit GoogleEarth, a rather different type of display than ArcMap. Then I saved the reconfigured layer as one of My Places in GoogleEarth as well as, by right clicking the layer folder, as a new KMZ file. I uploaded that KMZ file to a website that I created on GoogleSites, for free yet again, storing it in a page created with the File Cabinet Template and setting its permissions to allow public access. Then I went back to Google Earth and opened the Embed KML Gadget, configuring the various options such as dimensions and entering the URL for the KMX file stored on my GoogleSite.

The result is a fully functional temporal GIS of the British and Dutch slaving voyages of 1751-1795, first shared using ArcGIS Online. Selecting any of the daily position symbols opens a pop-up box with a link to the Web page for that voyage in the Trans-Atlantic Slave Trade Database. By adjusting the various controls of the time slider, users can view the entire period (1751-1795) or constrain it to a single decade, adjust how many months or years of symbols are visible at once, and move along the temporal sequence one step at a time or as a continous animation.

The control of the time slider, especially over the speed at which the sequence runs, is limited compared to using it in GoogleEarth. The KMZ file can, however, be downloaded and added to GoogleEarth running on a computer or added using the Add Network Link. Doing that will also allow users to access the individual layers for each voyage, the daily position points, and all the attributes as well as to resymbolize points, remove and add data, such as place markers for ports, and finally to save their work as a new KMZ file.

The free versions of GoogleEarth, ArcGIS Online, and GoogleMaps Engine Lite, of course, cannot do nearly as much as the ones you pay for: GoogleEarth Pro is $399 for an annual license; a paid subscription to ArcGIS Online is at $2,500 per year for the minimum of 5 users; and GoogleMaps Engine is "expensive."

The Attelante on GoogleMaps

Another free Web mapping application is GoogleMaps Engine Lite. It allows rapid creation of a basic online GIS, with layers and attribute tables. One way of adding data layers is to upload an Excel worksheet, as long as it has no more than 100 records/rows or 50 columns/fields. Maps Engine Lite also has limited symbolization and other functions, compared even to the free, non-subscription version of ArcGIS Online I have been posting with so far. It does allow use of the GoogleMaps Satellite view, however, and its oblique, 45 degree perspective. Simply click the drop down menu beside Base Map in the legend and select satellite view; it will switch to oblique when zoomed in.

The Attelante was a Dutch Brig that sailed for New York City with a cargo of gin (jenever), copper, and white lead in July 1815, arrived in September, and returned to the port of Schiedam with a cargo of tobacco in December 1815. It actually arrived in November but could not get into the harbor due to thick ice on the river, Schiedam being a port on the Nieuwe Maas River and relatively far inland. Although Schiedam is not named as the departure port on the outbound voyage, it likely was because it was the return port and one of the principal producers of gin in the Netherlands.

Despite the brevity of the voyage, it still requires 127 records, necessitating splitting the worksheet into two, one for the outward segment and the other for the homeward one, and uploading them to the Web map as separate layers.

Still, the shortness of the voyage made the Attelante a good candidate to explore the capabilities of GoogleMaps for the type of data used in this particular project. Other vessels and voyages would have required splitting into more than two layers. Besides, I had not previously done a Web map of the Attelante voyage.

Seasonality of the Slave Trade Voyages, 1751-1795

A temporal analysis in ArcMap of the slaving voyages in the database yields the following temporal vizualization. Forty-six of the forty-eight voyages are Dutch and two British, most of them transporting enslaved people across the Atlantic from Africa to Suriname and the Caribbean. Forty-four of the voyages also appear in the Trans-Atlantic Slave Trade Database. For further details on the voyages, see this post.

I achieved the visualization by changing the year of each voyage to a single year, setting both the time interval and time window to one week, and creating a new point feature in the South Atlantic labeled with the month field. The daily-position symbols are triangles, with a week’s worth showing at a time. Their colors represent the direction of the voyage segment, based on their destination fields: green for northbound, orange for southbound, and purple for westbound. That compresses the voyages, which took place from 1751 through 1795, into a single annual period and allows analysis of variation by week and month.

The Trans-Atlantic Slave Trade Database contains many more voyages than this GIS and therefore reveals a great deal of variation in seasonal patterns and why it occurred, as explained in this essay.

Seasonality of the Catalan Triangular Trade

Seasonal variation in supply and demand for tasajo must affected the timing of the triangular Catalan trade with the Rio de la Plata and Cuba. Cuban demand for tasajo increased during the cane harvest from January through June, as plantation owners bought additional enslaved Africans for the grueling work, which killed many. Meanwhile, tasajo production took place during the nine months of October through June, the southern hemisphere’s spring, summer, and fall. During the winter, in contrast, the cattle weighed less and the air temperature was too cold to dry tasajo. To maximize the price differential between the purchase of the tasajo and its sale, then, voyages departed from the Río de la Plata during the season of tasajo production in time to arrive during the season of peak demand in Cuba.

The export statistics for the port of Buenos Aires confirm that seasonal rhythm (after fig. 6.12 in my book Black Ranching Frontiers, Yale,2012). Shipments of tasajo through the port of Buenos Aires began as soon as tasajo became available in the spring (October and, even, September) to arrive in Cuba in November and December in anticipation of the beginning of harvest season. That volume of shipments continued until about May, with those departing at that time still able to arrive in Cuba before the end of the cane harvest. July and August marked the nadir of tasajo shipments northward due to a lack of both supply and demand.

A temporal analysis run in ArcMap, shared using the video below, clearly shows the seasonal variation in the triangular trade, even with only fourteen voyages. I achieved the visualization by changing the year of each voyage to a single year, setting both the time interval and time window to one week, and creating a new point feature in the South Atlantic labeled with the month field. The daily-position symbols are triangles, with a week’s worth showing at a time. Their colors represent the direction of the voyage segment, based on their destination fields: green for northbound, orange for southbound, yellow for eastbound, and purple for the single westbound segment.

The visualization shows that in September, as cattle began to fatten, temperatures started to increase, and tasajo production kicked off, shipments began to leave the Rio de la Plata. They arrived in Cuba by November or December in anticipation of the cane harvest that would begin after Christmas. The shipments continued until tasajo production and the cane harvest both wrapped up in June. Focusing on the Caribbean during July and August reveals the lack of northbound (green) segments those two months.

Higher Resolution Acanica Videos

Uploading the videos to YouTube, also free, allows sharing them at higher resolutions, up to 1080p if your connection will allow it. Set the resolution with the gear icon once you start the video. See the previous post for details about the animations.

Animated Acancia-Hurricane Interactions

Despite the lack of support for temporal GIS in the free, non-subscription version of ArcGIS Online, ArcMap does support the creation of videos that can be uploaded to free blogs like this one or to video sites like YouTube. To create a video, users zoom to the desired area of the GIS, such as the North Atlantic, and then open the Time Slider window to set controls such as time extent, interval, and window. They then use the Animation toolbar to select Create Time Animation. The Animation Controls and Animation Manager, available through the same toolbar, allow adjustment of variables like the length of the animation in real time and playback time. Export Animation opens a dialog box to set the video format, choose compression options, and save the video.

I exported such a video focused on the North Atlantic with the route and daily position symbols for the Acancia and the HURDAT layers turned on using August 1 through September 5, 1893 as the time extent and 1 day as the time interval and window. The result was AcanciaAug1893.avi, which shows the vessel's day-by-day progress along its route line with hurricanes tracking nearby. Since each 24-hour time window has two noontime position markers, the current and previous day's noontime positions are visible on each frame, allowing users to visualize the daily progress of the Acancia along its route. Similarly, since each hurricane has several positions per day in HURDAT, at 6 hour intervals, the time settings allow users to visualize the evolution of the hurricane tracks through time and space even without the equivalent of a hurricane route line.

AcanciaAug1893.avi was 950 MB in size, however, far in excess of the 100 MB upload limit for this blog. I therefore opened it in Windows Live Movie Maker, included in the most recent versions of MS Windows and available from Microsoft as a free download, to reduce the file size. After adding some title frames, I saved it as AcanciaAug1893.wmv, only 8.3 MB, and uploaded it to this blog.

The result allows readers to view an animated version of the crossing of the North Atlantic by the Acancia in August of 1893 that is fairly identical to what they would see using the time slider in ArcMap. The resolution is lower and the user cannot pan, zoom, or control the time animation as fully as in ArcMap, but the video nonetheless visualizes vessel-hurricane interactions. As the Acancia passed north of Bermuda into the mid Atlantic with its cargo of aguardiente, a tropical storm formed in its wake and tracked northward. To the south, several hurricanes arced westward from the Cape Verdes toward the Caribbean, Florida, and the Atlantic Seaboard. As the Acancia turned southward toward its destination, Montevideo, it sailed directly across that cordon of hurricanes, and on August 26th at noon came within 50 miles (80 km) of the eye of a category 2. After an eastward excursion from its southward course, the Acancia continued southward toward Montevideo.

Since users have much more limited control to pan, zoom, and so on than if ArcMap could be time enabled on ArcGIS Online, I created another video that focuses more closely on the mid-Atlantic encounter between the hurricane, the Acancia, and its crew in late August 1893.

In future posts, I will explore other ways to share the temporal aspects of the GIS on the Web.

The Acancia meets Hurricane 7 of 1893

To more closely examine encounters of particular vessels and hurricanes would benefit greatly with being able to run the voyage and hurricane layers through time, as I can with my ArcMap version of the GIS. While I cannot upload time enabled files to the free version of ArcGIS Online, NOAA has published HURDAT on ArcGIS Online as a map service and as a Web map. The map service is served from the following URL: http://maps4.arcgisonline.com/ArcGIS/rest/services/A-16/NOAA_Hurricane_Tracks-Temporal/MapServer, and anyone can therefore add it as a layer to their Web map using the free ArcGIS Online viewer and display the data through time using the temporal function.

That is what I have done experimentally in the below Web map, reducing the voyages to that of the Acancia in order to simplfy the map during the trial. To add such layers, select Add Data, Add Layer from the Web, and then type the URL for the service into the dialog box. After being added to the Web map, the time control, or "time slider,"  for the NOAA layer appears at the bottom of the map frame. In the version embedded in this blog, select the time button and the time slider will appear as a separate window.

View Larger Map

The problem is that it does not work well because of the enormous amount of data. The HURDAT layer in my ArcMap GIS, which covers the North Atlantic for 1851-1900, has 9366 records. The NOAA map service includes Pacific and Atlantic storms, 1851-2011. When running the time sequence, the rendering takes so long that blobs of hurricane tracks, indistinguishable from one another, jerk across the screen.

I tried filtering the layer to reduce the amount of data being displayed, which is also possible in ArcGIS Online for layers from a map service. I selected each sub-layer in turn, selected filter, and set the Hurricane Season field to 1893, which was the year the Acancia was crossing the North Atlantic in August. Then I set the controls for the time slider to display only 1893 and do it in 365 equal intervals to get a daily interval. Even at an extent, or "time window," of one year, however, the time function does not work well enough to use because the symbols take so long to render that they remain essentially static.

Even the data in the pop-up boxes for the hurricane symbols take a long time to arrive from the server compared to those for the Acancia. Nonetheless, the pop-ups do reveal that the Acancia, shortly after turning southward, had a near encounter with a category 2 hurricane in mid Atlantic on August 26, 1893.

In the next post, I will try another strategy to visualize discrete hurricane-vessel encounters.

The Tasajo Voyages and Hurricane Season

One of the aspects of Atlantic commodity networks like the one that involved Catalan vessels and tasajo that interests me are their seasonal patterns. In what months did the various legs, or segments, of the voyage typically occur? What did seasonal timing have to do with the production schedules of the commodities, such as the months in which various crops were harvested? What did seasonal timing have to do with sailing hazards, such as the hurricanes that characteristically occur in the North Atlantic between early June and late November, with mid August through mid October marking the peak of hurricane season?

Because I am using the free, non-subscription version of ArcGIS Online, at least so far, I cannot implement the time function of my ArcMap version as a Web map. Nor can I do queries and filters with the Web map like I can on my computer in ArcMap. But another way to analyse such patterns in time and space involves applying symbols to represent months or seasons instead of cargoes.

To do that with the objective of visualizing patterns relative hurricane season, I saved the Web map from the previous post (Catalan Tasajo Voyages, 1837-1900) as Catalan Tasajo Voyages: Seasonality. Then I applied new symbols based on the Month field instead of the Cargoes field. For peak hurricane season (roughly August through October, or Months 8-10) I applied red. For the tail ends of hurricane season (June, July, and November, or Months 6, 7, and 11) I applied yellow. And for the rest of the year (Months 12 and 1-5, or December through May) I applied blue.

View Larger Map

The overall pattern indicates that most voyages crossed the North Atlantic during low hurricane season (December-May) or during the tails of the hurricane season (June-July and November). Still, out of fourteen voyages, five have red position symbols in the northeastern quadrant of the during peak hurricane season. By zooming in on the northeastern quadrant of the Atlantic and using the pop-ups to get more information on those five voyages, they become distinct. Two, the Romantico in 1864 and the Prudente in 1884, Left Montevideo bound for Cuba with tasajo in August and September respectively, entered the zone of hurricanes in October, and reached Cuba in November, at the tail end of hurricane season. Two others left Cuba bound for Europe during hurricane season: another voyage of the Romantico, in 1863, with a cargo of sugar and the Pepe in 1851. The Pepe crossed the North Atlantic in late October and November, during the tail of the hurricane season. The Romantico left imprudently in late July and crossed during the August peak of hurricane season. And one, the Acancia, left Cuba bound for Montevideo with a cargo of aguardiente in early August of 1893, sailing first eastward and then southward across the hurricane zone during the peak of the season.

The survival of the logbooks testifies to the lack of a fatal encounter with a hurricane on any of those imprudent voyages, but adding a hurricane layer to the Web map to examine any close encounters between vessel and hurricane will nonetheless be the task of the next post.

Catalan Tasajo Voyages, 1837-1900

This Web map shows the routes of fourteen voyages by twelve Catalan vessels from the mid to late nineteenth century. For the most part, they carried manufactured goods, wine, and other Catalan products from Barcelona to Buenos Aires or Montevideo. There they loaded tasajo, a type of salt beef used as food for enslaved Africans on sugar plantations in Cuba. Once in Cuba, they either loaded sugar or aguardiente, a type of cane alcohol, and returned to Barcelona; or they proceeded in ballast for ports in the US South, like Charleston, and loaded cotton for the textile mills of Barcelona. The GIS includes several variations on that basic trade pattern. See my 2010 article on The Hispanic Atlantic's Tasajo Trail in the Latin American Research Review for more details.

For method and data, see the first four posts of May 2013. The main difference with the previously posted Web map of Dutch slaving voyages is that I did not delete as many of the fields in the Catalan layers of the GIS before uploading them. That slowed the upload process a bit, even causing timeouts on a few occasions because I am working from my home office on a relatively slow connection at the moment) but it transferred more of the data in the GIS to the Web map. The fields I did delete were related to weather observations, included in the GIS because of all the weather data in the CLIWOC database but mainly empty fields regarding the Catalan voyages because when I transcribed those logbooks I had limited time, did not need the detailed weather observations for the research project I was working on, and so focused on the latitude, longitude, and cargo data.

I will use this Web map as the basis for others in the next several posts.

View Larger Map

The project GIS and website are © 2013 by Andrew Sluyter but open source and licensed through the Creative Commons as attribution-noncommercial 3.0, which allows others to use the data and programming to produce non-commercial derivative products. I collected the data in four archives in Spain while funded by a 2009-10 Louisiana Board of Regents ATLAS grant: Arxiu Històric Municipal del Masnou, Arxiu Històric Municipal de Sitges, Biblioteca de Catalunya, and Museu Marítim De Barcelona.I processed the data and built the GIS while a 2012-13 Digital Innovation Fellow of the American Council of Learned Societies. As with the other Web maps on this blog, anyone is free to use it as a basis for new, non-commercial, scholarly and creative products as long as each such GIS, website, and/or publication contains a statement that acknowledges Andrew Sluyter as this GIS's creator.

Websites with Info on Vessels

These are some useful Websites with information and images about specific sailing vessels:

1. Maritiem Digitaal: www.maritiemdigitaal.nl
2. Sailing Ships: www.sailing-ships.oktett.net
3. The Ships List: www.theshipslist.com
4. Three Decks - Warships in the Age of Sail: www.threedecks.org

Web GIS

Web GIS

To create an initial implementation of the Web GIS for the project, I used the free, non-subscription version of ArcGIS Online, not only because my institution does not have a subscription to ArcGIS Online but because I want to make this project as useful as possible to people with limited resources. Anyone with a broadband connection and a computer can now use the Web GIS published in the second post to this blog as the basis for their own Web GIS by modifying it, adding to it, deleting parts of it and saving their creation to their own free account on ArcGIS Online.

The first step in publishing the GIS on ArcGIS Online involved converting the feature class files, both line and point, to a format that the free version of that software accepts. In order to reduce the work involved in what was a test, being my first experience with web mapping, I copied the GIS and renamed it Slave_Voyages_WebGIS. Since I had used relative paths, the copy worked perfectly. I then used a select by attribute query to find all the voyages that carried enslaved Africans and deleted all the layers for other voyages as well as the hurricane layer because the slave trade had ended by the time the hurricane record begins in 1851. Then, to reduce the size of the files in the Web GIS and ensure it ran smoothly, I used the Delete Field tool (in Data Management Tools, then Fields) to delete all fields except VesselID, Flag, DayOfWeek, Latitude, Longitude, TSTDURL, Cargo, and DATE from the attribute tables of the point Feature Class files. Then I used the Feature Class to Shapefile tool (in Conversion Tools, then To Shape file) to export all the point and line Feature Class files from the geodatabase to a new folder. Opening that folder reveals that each feature class exports as several files, including the essential file.shp, file.prj. file.dbf, and file.shx files. Each includes essential information such as the projection, spatial reference, point or line locations, and attributes. Selecting all the files associated with a point or line feature class, right clicking, and selecting Send to Zipped Folder creates a compressed folder that the free ArcGIS Online accepts as an upload, as long as it does not contain more than 1,000 features.

Next, I started a new map from within my free ArcGIS Online account by selecting the Map link, then the New Map button, and then the Basemap button and adding the same Terrain base map I used in ArcMap. I then saved the new map to My Content with the name Dutch and British Atlantic Slave Trade Voyages, 1751-1795. Once that GIS had been saved to the ArcGIS cloud, I was able to add all the point and line layers for the 48 slaving voyages by selecting the Add Content button, selecting Add Layer from File, navigating to the folder of shape files on my computer, and uploading each in turn.

Those shape files do not contain the symbology and other features, such as the visibility range, of the layers in the GIS from which they were exported, necessitating a lot of work to reestablish appropriate symbols to indicate cargoes and so on within ArcGIS Online. That is accomplished laboriously but straightforwardly through the Contents panel of the  Web Map and a menu that appears when the arrow to the right of each layer is selected.

The popup box associated with clicking on each feature, line or point, is particularly useful in a Web GIS because it can include URL links to other Websites. Most basically, I configured the popup boxes for the point symbols for a particular voyage to include the URL for the search window in TSTD for that same voyage, allowing users to immediately connect the data and representations of one project to those of the other. I also ended up importing the Ports_and_Landmarks shape file to identify places relevant to the slave trade, from landmarks like Cape Mount, to ports like Paramaribo, to archives like the Zeeuws Archief in Middelburg that hold the logbook data on which the GIS is based. Each popup can contain a relevant image and “Get more info.” link to, for example, the Website of the archive or a Wikipedia entry on the place. I also added popup boxes to some of the lines that represent the routes of the vessels, in this case linking to the shipping news sections of online newspaper archives so that clicking on the line representing a specific voyage brings up an image of the newspaper that contains a link to the reference in its shipping news section for that specific voyage.

I also wanted to represent one of the patterns that became clear in the ArcMap version of the GIS, namely the pattern of deaths of enslaved Africans during the Middle Passage and the disposal overboard of their corpses. The Dutch logbooks recorded the number of deaths on each day of the crossing, making it possible to visualize that horrific aspect of the slave trade. To do so, I used the Merge tool in ArcMap (in Data Management Tools, then General) to generate a single layer that combined all the individual voyages but eliminated all fields except for VesselID, Cargo, and Occurrences, the last of which contained the number of enslaved who died on each day at each position, from a low of 0 to a high of 5. That created a new point Feature Class File with 8,889 records, or rows in the attribute table. I used various tools (Add Field, Find & Replace, Field Calculator, and Sort) available through the Table Options button in the top-left corner of the attribute table window or through right clicking the column heading to create a new short-integer field called SlaveDeaths and populate it with the number of slaves who died at each latitude, longitude on a particular day. I then eliminated all records with zero deaths and ended up with a layer with only 389 records that I could export, compress, and upload to ArcGIS Online in the same way as the other layers. Once added to the Web map, I assigned different diameter point symbols to each number of deaths. The pattern is striking enough with that representation, but I also wanted to visualize it as a continuous density surface.

To represent the pattern of deaths as a density surface, I first ran the Kernel Density Tool (in Spatial Analyst Tools, then Density) in ArcMap. It calculates the density of occurrences, deaths in this case, around each data point, the vessel positions in this case, and creates a floating point raster layer as its output, in other words, a layer that consists of square cells that each have a value such as 0.349759 or 2.897347. The problem was that the free version of ArcGIS Online does not allow you to upload raster layers. The raster layers must already exist within ArcGIS Online, for example, the base maps, which are raster layers. I used three additional tools to work around that limitation. First I used the Raster Calculator Tool (in Spatial Analyst Tools, then Map Algebra) to multiply all the raster values by 10,000, which changed the output from deaths per square kilometer to deaths per 10,000 square kilometers but would stop the next tool from truncating most of the significant figures. That next tool was the Int Tool (in Spatial Analyst Tools, then Math), used to convert a floating point raster file to an integer raster file by truncating all of the cell values. And the last tool was the Raster to Polygon Tool (in Conversion Tools, then From Raster), which converts integer rasters, but not floating point rasters, to a series of lines that define polygons. Once that polygon layer had been created, I exported it as a shape file, compressed the results into a zipped folder, uploaded it to ArcGIS Online, and applied colors to each polygon to represent different values of deaths per 10,000 km².

The free version of ArcGIS Online does not support uploading temporal layers, so the voyages cannot be run as a temporal sequence in the Web GIS. But future posts will explore solutions to that issue.

In the meantime, here is a prior temporal animation of the CLIWOC database for purposes other than historical climatology; here is another; and here is a nice static representation as well as one that shows seasonal patterns.

The Project GIS

 The Project GIS

To create the GIS, I used the ArcMap and ArcCatalog components of ArcGIS 10, for which my institution has a site license. I have been using GIS on and off since the late 1980s. Over the past two decades, GIS has gone from typing in commands to selecting them from pull-down menus, from relatively little functionality to high functionality, and from static dot-matrix printouts to high quality, dynamic output on the web. In parallel, the data sources have gone from having to convert them to digital format yourself, to a limited range becoming available on floppy disk, to many becoming available on the web in digital form. The latest innovation, and the one that prompted me to get into a relatively large GIS project for the first time, has been the addition of a temporal function to ArcGIS, finally making it quite easy to display changes in spatial patterns over time. Its productivity has not kept up with its impressive functionality, necessitating a lot a mouse clicks and keystrokes to accomplish many tasks that could be much better integrated, and its temporal functions remain limited compared to its spatial ones. But ArcMap has reached the level of utility that makes it essential to my research and instructional programs and, in fact, for anyone who does historical research.

The first stage of turning the prior work with the databases into a GIS involved setting it up and importing the data. I will be a bit more step-by-step about this stage because I assume those interested in the digital humanities have less experience with ArcGIS than with Excel.

For the initial setup of the GIS, I created a folder named ACLS_GIS to keep the project in, started ArcMap, used the File menu to start a New GIS, and saved it in the folder as ACLS_GIS.mxd. Then I went to the Customize menu and selected ArcMap Options to make sure that “Make relative paths the default” was selected, which makes it easier to move the project to another folder or computer. At the same time, you can check around the other tabs of the ArcMap Options dialog box to set other options to your personal preferences. Then I used the Windows menu to open the Catalog window, navigated to the folder ACLS_GIS, right clicked in it, selected New, selected File Geodatabase (not Personal Geodatabase, which is more limited), and named the resulting file ACLS_Geodatabase.gdb. That database, indicated by a special icon, has a specific format used by ArcGIS, will contain many of the files needed for the GIS, and will keep track of them and their relationships. The last step of the setup involved going to the File menu, selecting Add Data, selecting Add Basemap, and selecting one of the base maps that appear, everything from a basic gray outline to imagery from the Bing Maps Aerial view. The base map is the underlying map on top of which the data will be represented, and it can be changed for one of the other base maps at any time. I selected the Terrain base map, which includes four sub-layers named Reference, Borders and Places, Shaded Relief, and Terrain. For reference in the next step, the base maps use the Web Mercator projection of the WGS 1984 geoid, also known as the World Mercator; its projection reference file is named WGS_1984_Web_Mercator_Auxiliary_Sphere, and its datum reference is named D_WGS_1984 (more on this in the next step).

The next step involved adding the data. To add the worksheets in the Excel workbook named ACLS_Voyages_Data.xlsx to the GIS, I used the Folder Connection button (a folder icon with + on it) at the top of the Catalog Window to navigate to the folder that contained the workbook and establish a link to the ACLS_GIS folder and its ACLS_Geodatabase. I then opened each of the worksheets in turn (for example, Middelburgs_Welvaren_A$), right clicked it, and selected Create Feature Class From XY Table to open the dialog box that adds the data to the GIS. In the dialog box, I selected the longitude column as the X coordinates, the latitude column as the Y coordinates, and GCS_WGS_1984 as the Geographic Coordinate System of the Input XYs. Next, under Advanced Geometry Options, I selected Use Map Spatial Reference, which was the previously mentioned WGS 1984 Web Mercator. Those selections told the GIS the geographic reference of the data being added and the way in should project them onto a flat map. Without that information, the GIS has no idea if all those latitude and longitude numbers in the worksheet are in degrees, meters, kilometers, or something else; and it has no idea how to project those positions, which were taken from the curved surface of the Earth, onto a flat map. Then, still in the dialog box, I navigated to the ACLS_Geodatabase and saved the resulting Feature Class Files with the name of the vessel (for example, Middelburgs_Welvaren_A). Opening the ACLS_Geodatabase shows all of those files with an icon that indicates they are point features. After creating point feature class files for each of the voyages, I did the same for the Ports_and_Landmarks worksheet. (Note that if ArcMap does not recognize an Excel worksheet, saving it in an older version of Excel typically resolves the issue, in other words, as file.xls instead of file.xlsx.)

The next step results in all those data actually appearing on top of the base map, a gratifying moment in any GIS project. I used ArcMap’s Windows menu to open the Table of Contents window, selected the List by Drawing Order button at the top of that window (on the far left), went to the open ACLS_Geodatabase in the Catalog Window, and dragged each of the Feature Class Files to the Table of Contents under layers, in alphabetical order and on top of the base map. The order of the layers in the Table of Contents represents the order of the layers on the map, with all the voyages therefore drawing on top of the base map. All the voyages then appeared as a series of linear point patterns, the overall pattern emphasizing the triangular routes of the slave trade in the North Atlantic. Checking or unchecking on any of the layers in the Table of Contents turns that layer on an off. Dragging them rearranges their order in the Table of Contents and on the map. Right clicking any of the layers and selecting Open Attribute Table opens a table that shows all the cargo, date, latitude, longitude, and other fields associated with each of the features, or daily positions, in that layer. Selecting a particular day or range of days in the attribute table, selects its point symbol on the map and vice versa.

Once all the data have been imported into the GIS and represented as point symbols, the next stage involves the use of various tools to alter the data layers and derive entirely new types of layers from them. Those tools are accessible in ArcMap by selecting the Geoprocessing menu and then ArcToolbox or by starting ArcCatalog.

The first tool I used was the Convert Time Field tool to get the dates of each position into a standard ArcMap format to use the temporal function. I opened the tool (in the ArcToolbox under Data Management, then Fields), navigated to and opened the ACLS_Geodatabase, selected each of the voyage Feature Class Files in turn, selected their date field, which was in a yyyymmdd format, and converted them to the ArcMap, standard time format. Opening the attribute table revealed that the tool has added a new field, DATE, to the attribute table and populated it with dates in the correct format while leaving the original yyyyddmm date field intact.

Next, I used the Points to Line tool to create lines to underlay the points that defined each voyage, which I believed would better define the route of each voyage by connecting its daily positions and increase the symbolization, labeling, and other visualization opportunities. I opened the tool (under Data Management, then Features), navigated to and opened the ACLS_Geodatabase, selected each of the voyage point Feature Class Files in turn, selected the field that contained the voyage name as the Line Field, and saved the resulting line Feature Class files in the ACLS_Geodatabase, naming them for the voyage with _line appended (for example, Middelburgs_Welvaren_A_line). I then dragged each of the line Feature Class Files to the Table of Contents, placing them immediately underneath their respective point layers. A check of each line layer’s attribute table reveals that it contains a field with the voyage name, which can later be used to label the line. Then I selected Start Editing from the Editor menu and used the split tool to cut each line at the beginning and end of the segments of the voyage, deleting the superfluous segments: for example, the segments that cut overland across West Africa from the coast lf Liberia to the coast of Ghana because of the missing days and positions while the vessel followed the coast trading for slaves. This changed the attribute table for each line by adding a record for each new segment. Finally, in the Table of Contents, I selected both the point and line layers for each voyage, right clicked, selected Group from the menu, and combined them as sub-layers of a single layer, naming it after the vessel and voyage (for example, Middelburgs_Welvaren_A).

I then customized the way in which the GIS represented each voyage by right clicking each layer and selecting Properties. The dozen tabs of that dialog box allow you to set everything from how the temporal display works, to the style and color of the symbols, to the labels, to the scales (zoom levels) at which the layer is visible. To set the time function for the point layers, for example, I selected the Time tab, enabled time, selected the DATE field, and set the time interval at 1 day. To label each point or line segment, I used the Label tab, selected label features, selected the date field for the points layers and the voyage name layer for the line layers, and set the scale range at which the labels would come into and out of view. To connect to TSTD, I used the HTML Popup tab to select the TSTDURL field to open a search results window for a particular voyage in that database’s website whenever someone selected one of the point symbols for that same voyage in the GIS.

The most time intensive task turned out to be setting the symbols that would represent each daily position. To do that, I used the Symbology tab in the Properties dialog box. Since I wanted each daily position to reflect the cargo, I selected Categories, Unique Values, and the Cargoes field in the attribute table. I then assigned standard colors to each type of cargo: gray for unknown, brown for tasajo, and so on. In addition to the standard symbols included in ArcMap, users can create their own symbols and save groups of symbols under unique headings so the process becomes somewhat more productive.

By that point, the GIS had much of the data and functionality I had planned. With all layers visible, it displayed all the voyage segments as lines overlain with the daily position points, each with a unique symbol that indicated the cargo. At broad scales all the voyages blended together and large-scale patterns were evident, such as the triangular trade between Europe, Africa, the Americas, and back. Zooming in revealed individual voyages, their lines labels by name and their points labeled by date. Queries to search for particular attributes (Selection menu, then Select by Attributes) could be run to select all voyages that carried, for example, enslaved Africans. And, when time was enabled by selecting Open Time Slider Window (the clock icon on the main menu bar), and the GIS run through its temporal extent, the point symbols associated with each day appeared and disappeared in sequence. Selecting Time Slider Options from the Time Slider Window menu bar allowed changes to the length of the period represented, the speed at which it ran through that period, and so on. Such functions were enhanced by running the Add Attribute Index tool (under Data Management, then Indexes) for each of the Feature Class Files so that ArcMap could quickly locate records.

The final major task to ready the GIS for use involved correcting the positions added in Excel to represent missing days, latitudes, and/or longitudes as well as check the integrity of each of the other daily positions for each voyage. I accomplished that objective by advancing through each voyage, day by day, position by position, with its attributes table open and editing turned on. Positions that had been added for missing days with last known latitudes and longitudes, for example, where selected in the attribute table and dragged to interpolated positions between known ones. Position makers that appeared too close to shore were moved off shore on the bases of the attribute table information, which sometimes named a landmark, its bearing, and its distance. Position markers that had been given in the logbooks but appeared erroneous relative the preceding and following days were also corrected at this point, for example, when a logbook reported a positive latitude as a negative one or vice versa, as sometime happened near the equator.

Once all the voyage data were complete and correct as far as the available data allowed, the hurricane Excel worksheet was converted to a points Feature Class File and added as a layer in the Table of Contents. I used the ArcToolbox to create a DATE field, time enabled the layer, and applied symbols to differentiate among categories of storms.  Even reduced to 1851-1900, the attribute table had 9366 records.

By this point the GIS was highly functional. It made it possible to query intercepts in time and space between hurricanes and vessels. It revealed the pattern of deaths and disposals overboard on Dutch slaving voyages, the captains of which have left us a daily record of such deaths in the logbooks. And therefore the time had arrived to begin implementing it, or at least parts of it, as a Web GIS.

Selected References

1. Bodenhamer D. J., J. Corrigan, and T. M. Harris, eds., The Spatial Humanities: GIS and the Future of Humanities Scholarship (Bloomington: Indiana University Press, 2010).
2. Gregory, I. N., and P. S. Ell, Historical GIS: Technologies, Methodologies and Scholarship (Cambridge: Cambridge University Press, 2008).
3. Hunter, Richard, Methodologies for Reconstructing a Pastoral Landscape: Land Grants in Sixteenth-Century New Spain, Historical Methods 43 (2010): 1-13
4. Knowles, Anne Kelly, ed., Past Time, Past Place: GIS for History (Redlands, California: ESRI Press, 2002).

The Project Database

The Project Database

 

After that initial processing of CLIWOC, TSTD, HURDAT, and my database of Catalan voyages, I developed the project database as an Excel 2010 workbook named ACLS_Voyages_Data.xlsx. It contains 73 voyages by 41 vessels between 1752 and 1900 sailing between Europe, Africa, the Caribbean, South America, North America, and Asia carrying cargos of coffee, sugar, spices, gold, and many other products as well as enslaved Africans. The daily position and cargo data for 15 of the vessels making 17 of the voyages came from my database of Catalan voyages while that for the other 26 vessels and 56 voyages came from CLIWOC. Of the 41 vessels, 27 made a single voyage; 4 made 2 voyages; 4 made 3 voyages, 4 made 4 voyages, and 2 made five voyages. Vessels making multiple voyages were designated by different letters for each voyage: for example, Middelburgs_Welvaren_A versus Middelburgs_Welvaren_B.

The following table shows the breakdown by nationality.

Nationality	Number of Vessels	Number of Voyages	Slaving Voyages	Slaving Voyages Also in TSTD
British	4	4	2	2
Dutch	22	52	46	42
Spanish	15	17	0	0
TOTAL	41	73	48	44

Four of the Dutch slaving voyages do not appear in TSTD because they did not carry slaves across the Atlantic. The Zephyr was a naval brig that sailed from the Netherlands directly to Suriname; while on the Berbice River in Guyana in February 1764, the Zephyr took aboard 72 slaves from the Sieben Provinzen, a vessel name that does appear in TSTD but not for any voyages during the 1760s. Two of the three voyages of the Mercurius do not appear in TSTD because after taking aboard some slaves in Africa, the captain traded them and returned directly to the Netherlands with cargoes of ivory and gold. Similarly, one of the two voyages of the Granadier initially took aboard enslaved Africans but returned directly to the Netherlands with a cargo of ivory, wood, and wax.

Two of the British and two of the Spanish voyages did not carry cargoes at all because they were voyages of scientific exploration. I chose to include them because of their potential interest. The British ones were James Cook’s first two voyages: in the Endeavor, 1768-1771, and the Resolution, 1772-1775. The Spanish ones concerned the Alessandro Malaspina expedition of 1789-1794, involving two ships named the Descubierta and the Atrevida.

Another of the Spanish voyages was by the Astuto from Peru to Spain in 1778 with a cargo of coinage, jewels, gold, silver, copper, tin, cacao, cacao hulls, chocolate, and sugar. During the voyage, ten people kept separate logbooks and thereby generated ten separate records of their position that I chose to include because of the potential to evaluate the relative precision of their estimates.

Before importing the database into the GIS, I used Excel’s standard tools such as fill, find and replace, and sort to modify it in various ways because of the much higher productivity possible with Excel than trying to make the modifications once in ArcGIS. I standardized spellings of vessel names, captain’s names, place names, and so on, typically to conform modern usage. I added underscores to join words, for example, changing Drie Gezusters to Drie_Gezusters to avoid problems with ArcMap recognizing field names. Because CLIWOC had data on the number of crew members, tonnage, armaments, and other interesting information but presented it in a single field called OtherRemarks, I added separate fields for CrewSize, Tonnage, Armaments, and so on. And I added a field named DayOfWeek and populated it using the fill series tool and the calendar calculator at http://www.timeanddate.com.

Once all of those the changes had been made, most productively done in a single worksheet, I saved each voyage as a separate worksheet within the workbook because the free version of ArcGIS Online will only accept files of limited size, up to a maximum of 1,000 features. Since each voyage included several hundred features, each one representing a daily position on a particular date with associated attributes such as cargo, I would not be able to add them to ArcGIS Online using the free, non-subscription version as a single file.

A remaining issue involved the lack of vessel positions for some vessels while they were in port. I used the Geodata table in CLIWOC21.mdb to create another worksheet called Ports_and_Landmarks to locate the sometimes obscure places noted in the logbooks. The Geodata table provided an initial list of places and their latitudes and longitudes for that worksheet. I then added more places to it, especially regarding the Catalan voyages and the places along the coasts of West Africa and the Gulf of Guinea. David Eltis and David Richardson, An Atlas of the Transatlantic Slave Trade (New Haven: Yale University Press, 2010) proved helpful in locating obscure place references along the African coast. I used GoogleEarth to determine their latitudes and longitudes in +/- decimal degrees and copied/pasted them into the Ports_and_Landmarks worksheet. That worksheet then allowed me to populate the empty latitude and longitude cells for vessels in specific ports with the latitudes, longitudes for those ports.

A final remaining issue to ready the database before importing it into ArcGIS concerned missing days, latitudes, and longitudes. These occurred in three types of instance.

The first type of instance concerned entirely missing segments of the voyage. For example, many of the slaving voyages made their first African landfall along the coast of West Africa between present-day Senegal and Liberia, followed the coast southward and eastward while trading for enslaved Africans, and then departed for the Americas from a port in the Gulf of Guinea, such as Elmina in present-day Ghana. The logbooks generally did not record latitude and longitude positions while following the coast because sailors navigate by coastal landmarks when they can. Often there are no entries at all; sometimes a general description of the itinerary is given as a single entry that encompasses weeks or months of coastal sailing and trading; and in a few logbooks, landmarks and anchorages are noted for particular dates. For entirely missing segments, I simply ended with the last record of the previous segment, such as from Fort Rammekens, Netherlands to Cape Mount on the Liberian coast, and resumed with the first record of the next segment, such as from Elmina, Ghana to Paramaribo, Suriname. For segments that gave landmarks on particular days and places that the vessel anchored for, sometimes, weeks at a time, I added those days and used the latitudes and longitudes of those landmarks, anchorages, and ports in my Ports_and_Landmarks worksheet to add the missing positions for that segment.

The second instance concerned single days that were clearly missing from particular voyage segments. For example, if a day was missing in mid Atlantic from a segment of a voyage from the Gulf of Guinea to Suriname, I simply added that day and used the last known latitude and longitude as the position. In cases that the day was present but either the latitude, longitude, or both were missing, I again used the last known latitude, longitude, or both. Once the voyages were mapping in the GIS, I knew it would be possible to use the interactive editing tool to select those position markers and drag them into what amounted to a best estimate, interpolation of the missing position. In other words, the missing day, latitude and longitude would be dragged to a position intermediate to the position of the day before and the day after.

The third instance concerned days and positions, often multiple, at the ends of segments. A common instance of this occurred when vessels approached Europe on the homeward segment of a voyage. Sometimes the last entry in the logbook recorded sighting The Lizard, a lighthouse on Lizard Point that marked the Atlantic end of the English Channel. In other logbooks, The Lizard also marked the last latitude, longitude entry but other types of entries followed. Sometimes a single entry gave the date of arrival in a port, such as Fort Rammekens. Other logbooks included multiple entries that gave sightings of key landmarks as the vessel progressed up the Channel, such as Portland, the Isle of Wight, and Dover. For the first type, I ended the segment at that point, for example, at the Atlantic end of the English Channel. For the second type, I added the days between the end of the latitude, longitude entries to arrival in a specific port on a particular day and used the latitudes, longitudes of that port, of landmarks given, and of either the last known or next known position. Again, once the voyages were mapping in the GIS, I knew it would be possible to use the interactive editing tool to select those position markers and drag them into what amounted to a best estimate, interpolation of the missing position.

(The next post will deal with importing the database into the GIS.)