User:Shereth/2009 Mapping Project

The 2009 Mapping Project is a successor to the unofficial project embarked upon by myself in conjunction with my bot, User:ArkyBot, which itself was an extension of the unofficial (but extraordinarily useful) project started by User:Ixnayonthetimmay. At its heart, the goal of the project was to replace the ugly old raster "dot-on" maps being used in the majority of United States place articles with shiny new SVG vector maps, complete with corporate boundaries.

The first maps were created by hand and that took a lot of time, as there are many thousands of locations in the country. I created a script that used data files from the United States Census Bureau to largely automate the process, and as a result many thousands of US place articles are using maps created as a part of that project. In that regards, the project was largely a success.

However, there were some flaws with the maps. For one, they used data from 2000 and were already outdated in many cases. Secondly, deficiencies in the data meant that shorelines were not always accurately rendered and reflected political boundaries better than physical boundaries.

The 2009 project aims to correct these deficiencies by updating the data with 2009 boundaries, as well as overlaying accurate shorelines on the maps to make them look more as they are expected.

Information about the maps generated

The generated maps are in the SVG (Scaleable Vector Graphics) format suitable for use on Wikimedia sites. The maps are projected using the Lambert Conformal Conic projection, the projection used by the United States Geological Survey for their state-level maps. While vector maps are scaleable, the "native" scale of these maps is approximately 1 pixel = 200 meters. All maps (except for Alaska and Hawaii) use the same standard parallels and thus are on the same projection. All maps are based on the 2009 TIGER/Line® Shapefiles provided by the United States Census Bureau.

Rough project outline

• Procure complete 2009 data files
• Produce raw SVG files for each state. These files will include :

• Incorporated places and CDPs
• Native American/Alaskan Native/Native Hawaiian areas (reservations)
• County boundaries
• Shoreline data

• Shoreline data includes estuaries, bays, inlets, and other bodies of water directly connected to the sea
• Shorelines for the Great Lakes and Lake Saint Clair
• May incorporate the Great Salt Lake, Lake Okeechobee, and Lake Pontchartrain, depending on availablity of data

• Generate county-specific SVG files using the raw state files
• Manually prepare each county-specific file for individual place map generation
• Generate place-specific SVG maps
• Upload maps to Wikimedia Commons

State by state implementation

A quick survey of a few highly visible cities in each state was performed to determine the current status quo of the mapping by state; the results follow.

• Standard SVG maps showing incorporated places and CDPs by county are found in 24 states. There should be no problems implementing the new batch in the following states : Alabama, Arizona, Arkasnas, California, Colorado, Florida, Georgia, Hawaii, Idaho, Indiana, Iowa, Kansas, Maryland, Minnesota, Mississippi, Missouri, Montana, Nebrasca, Nevada, New Mexico, North Dakota, Oregon, South Dakota, Texas and Washington. An early variant showing townships was used in Michigan; see below for the township issue.

• Raster (PNG) versions of the same type maps can be found in Louisiana. There should be no issues converting these to the new format.

• Ohio, South Carolina and Utah have some PNG maps and some dot-on maps. I do not anticipate any problems from converting these states.

• Oklahoma, Kentucky and Wyoming have the old dot-on style maps, but again, should not be problematic for a conversion.

• Raster maps are used in Connecticut, Maine, New Hampshire, New York, Pennsylvania, Vermont, West Virginia and Wisconsin that show townships. Refer to the discussion below.

• Delaware, Illinois and Tennessee use the pushpin locator map. I do not know if the members of the relevant projects have specific reasons for migrating to this style map, and we might want to confer with them prior to doing thier maps.

The following are special cases :

• Alaska - this state presents unique geography types (boroughs versus counties), is geographically expansive, and the incorporated places therein tend to be geographically expansive as well. We may want to consider alternatives for this situation.

• Massachusetts - existing maps of this state do not break it up by county, and it may be preferable to consult with the relevant wikiproject to see if they prefer to maintain this style.

• New Jersey - there are a number of unique raster maps that have been devised for New Jersey, and last time a suggestion was made to standardize them it went over like a lead balloon. I intend on skipping this state unless someone wants to engage the relevant wikiproject and see if they have had a change of heart since then.

• Rhode Island - similar to Massachusetts, the existing maps do not break it up by county.

• Virginia - the unique situation in Virgina with relation to cities and counties means we may want to consider it carefully.

Minor civil divisions/townships

There are 12 states that contain minor civil divisions, such as townships, towns or the like, that function on a level similar to incorporated cities. These states are Connecticut, Maine, Massachusetts, Michigan, Minnesota, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, Vermont, and Wisconsin. While the Census does not include them with incorproated places, it might be advisable to include them in the maps. To do so from my end is (relatively) trivial and I already have an option in the script to do so.

There are 17 states that contain minor civil divisions with varying levels of functionality and generally do not function as independent local governments. These states are Arkansas, Illinois, Indiana, Iowa, Kansas, Louisiana, Maryland, Mississippi, Missouri, Nebraska, North Carolina, North Dakota, Ohio, South Daokta, Tennessee, Virginia and West Virginia. I believe these to be of minimal value in the context of maps of incorporated places and see no point in adding them, but this may, again, be an issue to confer with the individual state wikiprojects for more input.

The remainder of the states do not have any minor civil divisions to worry about.

Current Issues

• Bounding boxes in the files appear to have some kind of discrepancy that is causing clipping at the edges of the state maps. This problem should be easily fixed with the next round of processing
• Data for Great Salt Lake is fragmented and not all readily idenitfiable; this problem must be clarified if the boundaries for this lake are to be included in the relevant Utah maps.

Questions to consider

• Do we want the boundaries for Indian Reservations to display on the map? Do we want them to display in blank white shapes like the CDPs, or do we want to display them in a different shade? If so, what shade?
• Some states (particularly in the northeast) have a different model of incorporated places that introduces a new relationship between cities, towns and townships.

• The states that use an alternate method need to be identified prior to processing them.
• A decision needs to be made as to what information ought to be displayed on the map.
• In general, state wikiprojects might be consulted to get a feel for which states might not even want these maps

• Kansas City, Missouri has significant territory in mulitple counties; a custom map should be generated for this city. A list of other cities with significant holdings in multiple counties should be compiled to combat this very issue. Cities with some territory in more than one county but with a majority in a single county should not have this problem.

Image comparisons

Old version using 2000 data, no water shown

New version using 2009 data, water shown - this image is raw output from the scrpt, does not include any post-processing

Much better, am most grateful for your efforts to create a better image -- thanks for giving me a look, please let me know if I can help you. ----83d40m (talk) 02:21, 6 January 2010 (UTC)

Any projection yet for how long before getting the new image in the Sarasota article? ----83d40m (talk) 11:22, 16 August 2010 (UTC)