One of the first things a new Woosmap user does to get started and before embedding the Woosmap WebApp in his site is to import his location data using Woosmap API. To fulfil this requirement, we provide you through this guide a python sample to help you dealing with your locations as a CSV Spreadsheet.
Parse and process your CSV File
Sample CSV File
|NOM DU MUSEE||VILLE||SITWEB||latitude||longitude|
|Musée du Louvre||PARIS||http://www.louvre.fr||48.858905||2.340991|
|Musée de la Poterie||BETSCHDORF||http://www.betschdorf.fr||48.900331||7.914441|
|Musée des Maisons Comtoises||NANCRAY||http://www.maisons-comtoises.org||47.243044||6.189533|
|Musée du Vieux Nîmes||NIMES||http://www.nimes.fr/||43.838287||4.359585|
Cleaning CSV Data Using CSVKIT command line tool
This CSV File contains some null geometry values (not well geocoded) and columns that we’re not interested in. To deal with it, we need to clean this dataset. csvkit is a suite of command-line tools for converting to and working with CSV. In our case, especially csvgrep and csvcut are very helpful to extract the desired data. You could work with a graphical user interface app, like Apple Numbers, Microsoft Excel or Google Sheet but csvkit lets you work with your data more efficiently.
Retain only all non-zero geocoded data (where latitude field is not empty).
Then extract the desired columns.
CSV Python Module
Because Woosmap API only accepts JSON as input, the next step is to transform each row of your file to Woosmap elements. The following python script is based on the native csv module. It makes it easier to deal with csv formatted file, especially when working with data exported from spreadsheets and databases into text files.
We preferred to iterate over the data rows using
csv.DictReader instead of classic reader mainly because it takes advantage of the header row with the column names and gives you the opportunity to grab it cell value by its name.
There is no well-defined standard for comma-separated value files, there are plenty of ways one CSV file can differ from another, yet contains exactly the same data. Many tools, which can import or export tabular data allow the user to indicate the field delimiter, quote character, line terminator, and other characteristics of the file.
All these parameters are grouped together conveniently into a
When creating a
csv.DictReader object, the programmer can specify a subclass of the Dialect class as the dialect parameter.
In our CSV file the fields are separated by commas and some values are double quoted.
Work with Woosmap API
The one step further is to transform each row element to a location object that follows the required structured format. The mandatory fields are a
storeId (as String), a
name and a
location geometry object (a couple of Latitude and Longitude).
Below is the method used to transform each row of our CSV to a Woosmap location element:
We are now able to open the file and iterate over the data to build an array of all the Woosmap location elements before POST them to Woosmap API:
Import using HTTP POST Method
The Woosmap Data API is a RESTful API with endpoint
https://api.woosmap.com/stores. It takes a mandatory parameter : your private key. Therefore the url you call should look like this:
The API provides 4 HTTP methods for interacting with resources but in our case we are interested in creating new resources so we’ll use the
Rolling batch import
If you work with more than 1000 locations, we recommend you to set a rolling import every X elements.
To do this, define a numerical constant in your code (
100 for example) and execute your
POST request each time this counter value is reached.
As you could see in the sample hosted on GitHub, the code allows you to update your stores using
PUT http verb and
DELETE the whole resource before re-creating it.
Don’t forget that the update method is based on unique identifier of your data so be careful to keep the same identifier.
Explore the data
Below, you can explore the museums of France previously imported using the Woosmap WebApp. Click on the image to display it.
- Woosmap API documentation and official Woosmap WebSite
- Python Sample
- CSV Manipulation with csvkit utilities
- The resulting WebApp