Lesson 8: Web APIs and Mashups

Lesson 8: Web APIs and Mashups

Introduction

Nice job, hotshot! The WatzThis app is starting to have some fancy features.

Frank, the new head of business development (or "Biz Dev," as we say), is less impressed than the rest of your colleagues. He's desperately looking for a way to justify his salary. He wants to squeeze some money out of this app.

He schedules an "all-hands-on-deck" meeting for 8:00 on Monday. (There had better be doughnuts . . . or at least bagels . . . but doughnuts would be better.)

You arrive right on time, of course. He arrives 10 minutes late with his complicated caffeinated beverage and no doughnuts. You can already tell this is going to be a tough week.

He announces that the brainstorming should commence immediately on new features that could bring in some money, or at least give him something new, exciting, and "Web 2.0" to work with when he pitches the app to investors.

The ideas start flying:

We could have a feature where people can post things for other people to read. But the number of characters will be limited to something less than 141 characters and more than 139!
How about if our users can read and post reviews of businesses?
Post pictures to WatzThis, and other people can view and comment on them!
How about a site where you can connect with your friends and see what they're up to?

Every idea, it seems, is something that's already been done. You're starving and thinking about those chocolate and peanut butter candy bars in your desk, and how much you wish you had them here. Then it hits you:

WatzThis could combine its new geolocation feature with an interactive map. By knowing exactly where you are, you might be able to use the power of the Internet to help you figure out "WatzThis?"

Everyone agrees that this is a great idea. You quickly dash off with an "Okay! I'm on it!"—narrowly avoiding a heavy-handed congratulatory slap on the back from Frank.

Back at your desk, you stare at the monitor and wonder how you're going to deliver this exciting new feature. You know nothing about drawing maps. Finding a user's latitude and longitude with HTML's Geolocation Application Programming Interface was easy. But actually putting those coordinates on a map seems overwhelming.

Once again, the Internet comes to your rescue!

After a little searching, you find that many of your favorite websites—including Twitter, Google, Foursquare, and hundreds of others—have made parts of their data and functionality publicly available to any programmer who wants to use them.

Whether out of pure marketing genius or a simple desire to make the world a better place (but let's face it, it's probably the former), the result is that you can get access to data and functionality that you never would have been able to create yourself. The format that companies are increasingly using to share their data with other websites is called Web API.

In this lesson, I'll show you how to use Web APIs to access third-party data. In Chapter 2, we'll cover what Web APIs are and what goes into them. In Chapter 3, we'll discuss how you can combine data from different Web APIs into a mashup.

Let's get started!

Using Web APIs

Web Application Programming Interfaces allow developers to make parts of their apps' functionality or data available to other apps. I know what you're thinking: Why would someone who has invested time and money developing an app want to share their data and functionality with others for free?

Welcome to Web App Marketing 101, my friend. A Web business might allow outside apps access to its data or functionality for many reasons.

  1. Exposure. Millions of Web businesses compete for the eyeballs of Web surfers. For most of these businesses, it's a struggle to acquire new customers . . . or to acquire them faster and keep them longer than the competition. A Web API that other businesses find valuable enough to use puts your message in front of more users.
  2. Functionality. If your app has a Web API that makes it easy to integrate with other Web apps, potential customers will see this as another reason to choose you over the competition.
  3. Advertising. Providing free data to users in exchange for users viewing messages from advertisers is the oldest business model on the Web. Publishing a Web API is one way to reach a larger audience.

Now that you have a better idea why a business would create a Web API, let's look at the building blocks of Web APIs. After that, we'll take a look at some examples of APIs and how you can use them.

REST, XML, and JSON: Three Keys to Creating Interfaces

A great number of Web APIs rely on three simple technologies: REST, XML, and JSON.

REST stands for Representational State Transfer. It's a flexible way to expose functionality of a Web app by using meaningful URLs. The term Representational State Transfer is a fancy way of saying, "Use ordinary Web links that represent what you're asking for to get what you want."

A Web API that uses REST is a RESTful Web API.

For example, Geonames.org provides a variety of Web APIs. One of these tells you thecurrent time based on your latitude and longitude. To use this API, you simply access it usingthe following format (where the ?? represents variables).

http://api.geonames.org/timezone?lat=??&lng=??&username=??

If you find out your current location (using the modification we made to WatzThis in Lesson7, for example), you can find your current time zone by plugging the latitude and longitudeinto the appropriate places in the URL. For example:

http://api.geonames.org/timezone?lat=47.01&lng=10.2&username=demo

If you type or paste this URL into a browser, the Geonames.org Web API will return some data.

A RESTful Web API usually returns data in an XML or JSON format. We'll get to JSON in aminute; let me give you the details on XML first.

XML (Extensible Markup Language) is a markup language, like HTML, that's easy to read. Ithas stricter syntax rules than HTML, however, which also makes it useful forcommunications between computers, where having precise data matters.

The main syntax differences between XML and HTML are:

  1. XML is case sensitive. As you learned in Lesson 6, most computer languages are casesensitive. In other words, they don't recognize "T" and "t" as being the same.
  2. Every XML element must be closed. Unlike HTML, XML doesn't permit openingtags without matching closing tags or a closing slash in an empty element.

Here's a sample of XML data that you get when you use the Geonames.org URL listed above:

  <geonames>
    <timezone tzversion="tzdata2015g">>
      <countryCode>AT><countryCode>
      <countryName>Austria><countryName>
      <lat>47.01><lat>
      <lng>10.2><lng>
      <timezoneId>Europe/Vienna><timezoneId>
      <dstOffset>2.0><dstOffset>
      <gmtOffset>1.0><gmtOffset>
      <rawOffset>1.0><rawOffset>
      <time>2015-12-07 23:04><time>
      <sunrise>2015-12-07 07:50><sunrise>
      <sunset>2015-12-07 16:30><sunset>
    <timezone>
  <geonames>

Notice that the basic structure of the document resembles HTML in that it uses tags to create elements that describe data.

However, here's the important difference between XML and HTML: XML has no predefined elements. As long as you stick to the syntax rules and some additional restrictions on which characters are valid (no spaces in element names, for example), you can use pretty much anything as an element name. So the following are all perfectly okay XML elements:

<taco></taco>

<salsa></salsa>

<cerveza></cerveza>

You can put content between the beginning and ending tags, just as you would in HTML, and the tags provide more information about the purpose of the content.

Extensible Markup Language means that you can "extend" the language by creating your own tags. You don't have to wait for some other person or organization to invent and approve the tags you need right now.

Take the example of the <taco> element I created earlier. If you wanted to describe your taco, here's one way you might do it with XML:

<taco>
  <tortilla type="corn">
    <filling>
      <bean type="pinto">2 oz</bean>
      <produce>cabbage</produce>
      <produce>onion</produce>
      <produce>radish</produce>
      <salsa color="red">2 Tbsp</salsa>
      <cheese type="cotija">1 Tbsp</cheese>
    </filling>
  </tortilla>
</taco>

Notice that this markup code is readable to you and me, and that it provides almost all the information that you need to create the perfect vegetarian taco, along with the proper nesting of elements (the filling goes inside the tortilla, for example). None of the elements are predefined—I just made them up as I needed them.

Even though you're free to create your own XML elements as you wish (and this is exactly what most people do when working with Web APIs), there are sets of predefined XML elements, called vocabularies, which experts have created for different purposes. For example, a standardized XML recipe vocabulary might specify the elements you should use to write a recipe.

In the next lesson, we'll look at one popular XML vocabulary, SVG, for creating graphics on the Web.

JSON (JavaScript Object Notation) is a newer and lighter-weight data-exchange language than XML. It's becoming increasingly popular for Web apps because it's easy to use inside JavaScript code. Why is it so easy to use? Because it actually is JavaScript code.

JSON packages data as JavaScript objects with the following format:

{
name:value,
name:value,
name:value
}

Here's what the XML example above looks like when you convert it to JSON:

{
  "geonames": {
    "timezone": {
      "-tzversion": "tzdata2015g",
      "countryCode": "AT",
      "countryName": "Austria",
      "lat": "47.01",
      "lng": "10.2",
      "timezoneId": "Europe/Vienna",
      "dstOffset": "2.0",
      "gmtOffset": "1.0",
      "rawOffset": "1.0",
      "time": "2015-12-07 23:04",
      "sunrise": "2015-12-07 07:50",
      "sunset": "2015-12-07 16:30"
    }
  }
}

Notice that the JSON representation of this data uses less than half of the number ofcharacters (and therefore less storage space).

Once JSON data is inside a running JavaScript program, you can access the data inside it using dot notation. For example, here's how you can access different parts of the above geonames data:

geonames.timezone.time
geonames.timezone.lat
geonames.timezone.lng

Many Web APIs provide their output data in XML and JSON formats. The New York Times is a great example. Once you sign up as a developer at developer.nytimes.com, you can access articles, best sellers, event listings, movie reviews, and more by using their RESTful Web APIs.

For example, the following URL selects movie reviews using the keyword "airplane":

http://api.nytimes.com/svc/movies/v2/reviews/search.json?query=airplane&api-key={your-API-key}

To retrieve the same data as XML, just change the extension after "search":

http://api.nytimes.com/svc/movies/v2/reviews/search.xml?query=airplane&api-key={your-API-key}

Why don't these URLs work? Notice that each one contains a parameter called "api-key." Many Web APIs require you to register as a developer and obtain a unique ID, called an API key, to access their service.

Including the API key helps the app owners make sure that you've agreed with their terms of service and gives the provider of the Web service some information about who's using their API. In most cases, requesting the API key is a simple process and takes only a few minutes, as you'll see in this lesson's assignment.

Here's a sample of the data that the above request returns (assuming that you provide a valid API key):

{
"status":"OK",
"copyright":"Copyright (c) 2012 The New York Times Company. All Rights Reserved.",
"num_results":6,
"results":[
{
"nyt_movie_id":1306,
"display_title":"Airplane!",
"sort_name":"Airplane!",
"mpaa_rating":"PG",
"critics_pick":1,
"thousand_best":"1",
"byline":"Janet Maslin",
"headline":"",
"capsule_review":"Hilarious spoof of airport movies. Love the automatic pilot.",
"summary_short":"",
"publication_date":"1980-07-02",
"opening_date":"1980-07-02",
"dvd_release_date":"2000-10-24",
"date_updated":"2010-08-25 14:47:08",
"seo_name":"Airplane-",
"link":{
"type":"article",
"url":"http:\/\/movies.nytimes.com\/movie\/review?res=EE05E7DF1738E762BC4A53DFB166838B699EDE",
"suggested_link_text":"Read the New York Times Review of Airplane!"
}
}

Just as you can click a link or paste the URL for a RESTful Web API into a browser address bar, Web apps that programmers write can request data through the same URL. Then, when the Web app on the other end responds with XML or JSON code, the Web app can convert the response into a useful format and do things with it (such as show it to the user or use it in its own calculations).

Now that you know a little bit about Web APIs, let's explore the answer to this question: How do people combine their data (or functionality, or both) to make mashups? Please move ahead to Chapter 3.

Mashing Up Services

The availability of public Web APIs has created infinite new possibilities for combining the functionality and data of different websites, so you can create things that never existed previously. The name for these combinations of data or functionality is mashup.

In the same way that a baker combines flour, water, and yeast to make something much greater than the individual parts, you can combine different websites and Web applications to make something more useful than the parts.

For example, gasbuddy.com is a mashup that combines data about current gas prices with map and location data from Google. It shows you a map of gas prices in any location.

An interactive map of gas prices

Other examples of mashups include:

  • Moviegram (moviegr.am) combines data from YouTube, RottenTomatoes, and TrailerAddict to help you find movie trailers.
  • Klout.com combines data it gathers about you from various places on the Web to come up with a number that represents how influential you are online.
  • Music-Map (http://www.music-map.com/) uses data from several different sources to create maps ofmusical artists and how they relate to each other. The result is a cool interactive chart:

Music-Map chart of relationships between bands

There are three basic types of mashups:

  1. Business mashups combine a business's internal data with an external Web service. For example, a Web business that rates restaurants may combine its rating data with Google Maps to show the location of the rated restaurants, and with data from Facebook and Foursquare to show which of your friends has been to the business. The distinguishing feature of a business mashup is that it starts with data that the business owns.
  2. Consumer mashups combine two types of data from publicly available sources to create something that didn't previously exist or to help you look at the data in a different way. For example, Wikipediavision combines a map of the world with recent updates to Wikipedia, so you can see who's changing which articles.

    A map showing who is updating Wikipedia at the moment. Wikipediavision

  3. Data mashups combine similar data from multiple sources. An example of a data mashup is a comparison-shopping site, where you can search for a particular item, airplane tickets, hotel rates, and find out prices from multiple different sources.

You can create mashups by writing JavaScript code to retrieve, combine, and display the results of the combination. Or, you can create custom mashups by combining standard components in a mashupcomposition tool, such as Ifttt.com or huginn (https://github.com/cantino/huginn).

In the next chapter, we're going to create a business mashup inside Watzthis.com. That is, we're going to combine internal data from WatzThis (geolocation data) with external functionality from Google (specifically, the Google Maps API).

Using Google Maps API to Map WatzThis Locations

In Lesson 7, you saw how to use the geolocation API to find out the latitude and longitude of a user's smartphone. The latest release of the WatzThis app has a button that reveals the current location when you press it. Now it's time to use a Web API to translate that location into something that's easily understandable—a map, for example.

To do this, we'll pass the longitude and latitude from the device into the Google Maps Web API. It's surprisingly easy, and the results are impressive.

Some of the techniques I'll use in this example may go a little bit beyond the JavaScript that I showed you in Lesson 6. Don't worry about that. You can learn much here from just running the code and tinkering with it a little.

I've included some comments in the code to help you understand what's going on. Remember that HTML comments use <!-- and --> and that JavaScript comments use // at the beginning of the line.

Here's a modified version of the complete WatzThis app. This version opens up a Google Map and shows your current location when you press the GET LOC button. To try this out, you can create a new HTML document, or you can just copy and paste this code over the code in your copy of the WatzThis.com index.html file. Remember, unless you post the file to a Web server, this code won't work correctly in Chrome. Try using Safari or Firefox to preview the app locally.

<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="initial-scale=1.0, user-scalable=no" />
<title>Map Me!</title>
<link rel="stylesheet" type="text/css" href="watzthis.css">


<!-- the following script tag brings in the Google Maps API functions -->
<script src="http://maps.google.com/maps/api/js?sensor=true"></script>
<script>


//initialize variables for color buttons and auto mode
var light = new Array(); 
var t; 
var color=0; 
var flipping=0; 
var speed;  
light[0] = 'black'; 
light[1] = 'white'; 
light[2] = "red"; 
light[3] = "blue"; 
light[4] = "green"; 
light[5] = "orange";


function flip(whichway){
   //changes the screen color when called
   document.body.style.backgroundColor = light[whichway];    
   stopFlip(); 
}      


function autoFlip() {
   //changes the screen color automatically at an interval set by the global speed variable
   document.body.style.backgroundColor = light[color];
   if (color < light.length-1) {         
     color++;     
   } else {         
     color=0;     
   }
   t = setTimeout("autoFlip()",speed); 
}
function doAutoFlip(changespeed){
   //start the auto-flipping if it's not already going. 
   if (!flipping){         
     flipping=1;
     speed=changespeed;
     autoFlip();     
   }    
}
function stopFlip() {
    //stop the auto-flipping
   clearTimeout(t);     
   flipping=0; 
}


function $(id){
   //gets an element by the id passed to it.
   return document.getElementById(id);
}


//initialize map variables;
var you = {};
var map = {};


function getLoc() {
if (navigator.geolocation){
   //if the browser supports geolocation, get current location and display on a map.
       var gps = navigator.geolocation;
       gps.getCurrentPosition(function(position){
               var latLng = new google.maps.LatLng(position.coords.latitude,position.coords.longitude);
               var opts = {zoom:6, center:latLng, mapTypeId: google.maps.MapTypeId.ROADMAP};
               map = new google.maps.Map($("map_canvas"), opts);
               you = new google.maps.Marker({
                       position: latLng,
                       map: map,
                       title: "There you are!"
               });
               var infowindow = new google.maps.InfoWindow({
               map: map,
               position: latLng,
               content: 'Location found using HTML5.'
               });
       });
       
} else {
   //if the browser doesn't support geolocation, display an alert saying so.
   alert("Your browser doesn't support geolocation."); 
}
}


</script>
</head>
<body style="margin: 0px; padding: 0px;">
<div id="main">
<h1>WatzThis?</h1>


<div id="container"> 
<div id="directions">
<ol>
 <li>Use the top row of buttons to turn the flashlight on or off, or to make the light red.</li>
 <li>Use the 2nd row of buttons to turn on and off your automatic flashing color light dance party.</li>
</ol>
</div>


<div id="flashlight">
<input type="button" id="OFF" class="bigButton" value="OFF" onclick="flip(0);"> 
<input type="button" id="ON" class="bigButton" value="ON" onclick="flip(1);"> 
<input type="button" id="RED" class="bigButton" value="RED" onclick="flip(2);">     
</div> 
<div id="danceparty"> 
<input type="button" id="AUTO" class="bigButton" value="AUTO" onclick="doAutoFlip(500);"> 
<input type="button" id="STOP" class="bigButton" value="STOP" onclick="stopFlip();"> 
</div>
<div id="geolocation"> 
<input type="button" id="latlong" class="bigButton" value="GET LOC" onclick="getLoc();"> 
</div>
</div>
<div id="map_canvas"
	style="width: 300px; height: 300px; float: left; border: 1px solid black;">
</div>


</body>
</html>


Here's what our app looks like in iOS:

Mapping WatzThis.com location

Notice that all the regular functionality that you're used to having with Google Maps is available—including zooming, satellite view, and (if you zoom in close enough) street-level view.

This example shows off the primary benefit of using Web APIs such as Google Maps in your apps. They allow you to do amazing things that you would otherwise have no idea how to do . . . and to do them while writing a minimal amount of code yourself.

Summary

In this lesson, you learned how to use Web APIs and mashups to reduce the amount of work you need to do while increasing the quality of your product. What could be better?

We began this lesson by exploring why people create Web APIs, and you saw the fundamental languages people use to enable them (REST, XML, and JSON). Then you found out that there are three types of mashups: business, consumer, and data; and you saw some examples of mashups. Finally, you saw some real JavaScript code that that combines geolocation code from mobile devices with the Google Maps API to do something useful.

Has learning about mashups given you new ideas for apps? I hope so. It's now possible to create something impressive in very little time and without much expertise.

In the next lesson, you're going to learn about HTML5 technologies that are enabling new types of dynamic graphics and animation on desktop and mobile browsers. Until then, good luck on the quiz, and be sure to check out the assignment and Supplementary Material!

Supplementary Material

ProgrammableWeb - APIs, Mashups and the Web as Platform

Google Developers

IFTTT helps your apps and devices work together - IFTTT

A Beginner’s Guide To jQuery-Based JSON API Clients — Smashing Magazine

GitHub - huginn huginn Create agents that monitor and act on your behalf Your agents are standing by!

API Gallery - NYT Developers Network

Example Domain

Undisturbed REST A Guide to Designing the Perfect API MuleSoft

How to Use the NYTimes Developer API

APL - google search

APL example - google search

How do I use an APL - google search

Lesson 8 FAQs

Q: Why do people call them "mashups"?

A: The term "mashup" comes from the world of electronic music, where a mashup is a remix, or a combination, of two different songs (and sometimes videos). For example, you might want to check out "Queen vs. Outkast—Hey We Will Rock You Ya" on YouTube.

Q: Is the Google Maps API free to use for anyone?

A: Google makes its Maps API available to anyone to use in any app or website, as long as the site is free to use and doesn't violate any of the other terms of use. For example, Google wouldn't let you use the Maps API to violate anyone's privacy.

Lesson 8 Assignment

The New York Times makes its online content available to developersthrough a variety of public Web APIs. In order to control how often an application requestsdata, and to make sure that users abide by their terms of use, the Times requires developers toobtain a free API key for each Web API.

Follow these steps to get your own New York Times developer API keys:

Go to the New York Times developer site at http://developer.nytimes.com and clickRequest an API key.

Fill out the required fields. Be sure to enter an email address that you have access to. If youdon’t have a website address, you can use www.example.com

Select the Article Search API from the list of Web APIs in the drop down menu.

Click Create API Key to submit the form. After you submit the form you will receive an email with your new API key. Navigate back to the New York Times developer site at http://developer.nytimes.com.On the developer site, scroll down and you will find a list of the available API’s. Follow thesesteps to try out your API key:

Click on the Article Search API.
Click Try it out
Copy the API key that you received in your email into the field titled apikey
You can specify a search query to narrow your results by entering a search term into the input field titled q under the Parameters title.
Click View Results.

If your query returns any results, you should see data appear in the area to the right of thesearch form, and sample JavaScript code beneath the apikey input area. The data to the rightcontains articles that match your query and different pieces of information about those articlesin the JSON format. If you want to learn how to interpret the API request or the API results,you can click on View Documentation from the API console and then select Show details.

In order to run the sample JavaScript code, you’ll need to include the jQuery library into yourHTML document. You can also paste the sample JavaScript code into a script element in oneof the jQuery documents that you created in Lesson 6. The results will appear in the JavaScriptConsole as a JavaScript object. Click on the arrows next the Object to expand the data andview the results.