Using JSON to Exchange Data

Get the Math, Better Explained eBook and turn Huh? to Aha!

Javascript Object Notation (JSON) is a fancy name for a simple idea: A web page can download data stored as javascript variables. It’s caused a buzz in the tech world because JSON is much easier to load, read and manipulate compared to XML. Also, parsing large XML files can be a large performance hit – JSON gives you an object, packaged up and ready to go!

Remember the Script Tag?

In javascript you can import any script using notation like this:

    <script src="http://www.mysite.com/myscript.js"></script>

The browser will pull down the script and run it inside the current page. This is how embeddable widgets like Adsense and InstaCalc work.

The neat thing is that any variables and functions defined inside the script are available to the page. If myscript.js includes a function showMeTheMoney(), your page can call it (and, presumably, get shown the money).

This is all fine and dandy, and been known for a while. A cool use is to provide dynamic data to a page with encoded Javascript variables. If the variables were “objects” and we passed them using their special “notation”, we could call the system JavaScript Object Notation. (Technically, JSON just covers the “object” part; executable functions are dynamic javascript).

In Javascript, you can use objects like this:

var fruit = {}; // create new obj
fruit.name = "apple"; // set properties
fruit.color = "red";

// that is old fashioned! Try my new notation:

var fancyFruit = {"name":"pear", "color": "greenish"};

Javascript has a nice notation where you can define objects using "key":"value" pairs. You can define arrays and functions this way too, so your objects are more like a class, in fact.

So why is this useful again?

So far, passing scripts around is old news. But an idea comes along: what if you load a dynamically generated script, which has data inside of it? Now you can access the data as javascript variables. Here’s what I mean:

<script src="http://www.weathersite.com/latestweather.js"></script>
Today's weather is:
<script>
document.write(Weather.today);
</script>

See what’s happening here? We load the javascript (which could be updated every hour by the server), and access the Weather object it defines. Presumably, Weather.today is a string containing today’s weather. Realistically, we could have Weather[zipcode] or whatever format the site defines.

Cool, huh?

But Why Not XML?

Ah — some programmers will say “Verily, can you not download an XML file, and then process and parse it into Javascript variables?”

Sure, just like you can run a marathon or floss your teeth. Many want to, some try to, not everybody does.

XML is fine for certain things, but it can be quite cumbersome. JSON is great because you can include data in an easy, painless process. There’s no parsing step – you are getting your variables “for free” by just including the javascript file.

Also, JSON lets the site include functions, which may be used to process the data or handle other tasks.

A Few Details (As Always)

There are various ways to use JSON (see links for source of code sample):

  • Include a script tag directly (as above). This means the code inside the script is run immediately as the browser encounters it.

    <script src="http://www.mysite.com/mydata.js"></script>
    
  • Dynamically load the script tag. You can add a javascript tag to the head of a document (from codehouse.com):

    function dhtmlLoadScript(url)
    {
       var e = document.createElement("script");
       e.src = url;
       e.type="text/javascript";
       document.getElementsByTagName("head")[0].appendChild(e);
    }
    dhtmlLoadScript("http://www.mysite.com/mydata.js");
    
  • Use an AJAX library like jQuery to access the information:

    $.getJSON("mydata.js", function(json){
      // access data with json.fieldname
    });
    

Note: The first two methods can load files from any domain; an AJAX request can normally only access files on the same domain. For a cross-domain AJAX request, you may need to use a “callback=?” parameter, like this:

http://api.flickr.com/services/feeds/photos_public.gne?tags=cat&#038;tagmode=any&#038;format=json&#038;jsoncallback=?

This tells jQuery to create a temporary callback function to process the data, and tells the endpoint to wrap the JSON response in that callback function (example using jsoncallback=test).

Security, Security, Security

I’m not an expert in Web Security, but I know enough to realize it’s tricky and that there will be things I miss. Even the big boys like Google can get caught in bugs sometimes.

The safest approach is to only store public data with JSON. If you store private data in JSON (or XML for that matter), there are techniques where malicious sites can impersonate logged-on users. Yes, there are complicated workarounds for the security problems (like double-submitting cookies). But if you’re starting out (like me), start off using JSON for public info you don’t mind having disclosed (like weather reports).

Callbacks

The JSON data may not be a raw object; it could be an object passed to a callback function, such as:

specialFunction( {"name" : "Bob", "email": "bob@example.com"} );

In this case, the script will run a function called specialFunction that you’ve defined, passing it an anonymous object with a “name” and “email” property. Your specialFunction can then do wild things with this information.

This technique is the same at heart – this is how Google originally passed your Gmail contacts. Sometimes you can specify the name of your callback function in the URL you use to access the JSON data – it depends on the data provider.

Real-life Example: Currency Conversion in InstaCalc

I recently added currency conversion to the InstaCalc Online Calculator, which was a good lesson in importing data:

  • Find your data source. The Federal Bank of New York has an XML file with currency conversion data. That’s good enough for me!

  • Convert data to JSON format. You can convert XML to JSON using XSLT transformations.

  • Install XALAN so you can do XSL transformations using Java. A typical transformation looks like this:

    java org.apache.xalan.xslt.Process -classpath "xalan.jar" -IN in.xml -XSL in.xslt > out.xml
    

Be careful for Java classpath issues, I had trouble and had to include the classpath to xalan.jar manually.

  • Convert XML attributes to elements. Attributes can be difficult to access in the various XML to JSON scripts, so make them elements. Use this XSL to convert attributes to elements:

   <?xml version="1.0" encoding="ISO-8859-1"?>
    <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    version="1.0">
   <xsl:output indent="yes" />
   <xsl:strip-space elements="*" />
   <xsl:template match="*">
     <xsl:copy>
       <xsl:if test="@*">
         <xsl:for-each select="@*">
           <xsl:element name="{name()}">
             <xsl:value-of select="." />
           </xsl:element>
         </xsl:for-each>
       </xsl:if>
       <xsl:apply-templates />
     </xsl:copy>
   </xsl:template>
 </xsl:stylesheet>

Then process the XML file using XALAN:

java org.apache.xalan.xslt.Process -classpath "xalan.jar" -IN in.xml -XSL attributes-to-elements.xslt > no-attributes.xml

Finally, convert the XML to JSON using this XSL transformation. There are several XML 2 JSON stylesheets out there if you search.

Run your XML through XALAN to create the final JSON data.

java org.apache.xalan.xslt.Process -classpath "xalan.jar" -IN no-attributes.xml -XSL xml2json.xslt > currencies.json

Clean up your data. You have the raw JSON data as an object, but you may wish to wrap it in a function call. You can use a quick perl script to wrap myFunction( ) around the data above so your file does a callback when executed.

Tada! Now you have JSON data ready to access using one of the techniques above (see the currency JSON data). Inside InstaCalc, I have a static reference to the file:

  <script src="http://instacalc.com/data/currencies.json"><script>

Because the bank has some namespace information, I access variables like this inside the callback function: data["frbny:DataSet"];. Of course, the details of how you access your JSON data will change given the format of your XML and the exact stylesheet you used. Play around. If you look at the raw JSON file you can see the field names you need to use.

Bonus round: Updating the JSON. If your data changes often (like currency data does), you can put the above steps into a script and run it on a schedule. The next time your webpage is loaded, it will get the new JSON data.

Parting Thoughts

JSON is a really easy way to exchange data – just think of it as including extra javascript files in your program. Read more at json.org, learn it, and love it. Good luck.

Other Posts In This Series

  1. Starting Ruby on Rails: What I Wish I Knew
  2. Intermediate Rails: Understanding Models, Views and Controllers
  3. A Simple, Comprehensive Overview of Javascript
  4. Using JSON to Exchange Data
  5. How To Make a Bookmarklet For Your Web Application
Kalid Azad loves sharing Aha! moments. BetterExplained is dedicated to learning with intuition, not memorization, and is honored to serve 250k readers monthly.

Enjoy this article? Try the site guide or join the newsletter:
Math, Better Explained is a highly-regarded Amazon bestseller. This 12-part book explains math essentials in a friendly, intuitive manner.

"If 6 stars were an option I'd give 6 stars." -- read more reviews

24 Comments

  1. For prototype users,

    
     var e = new Element('script',
    	   		{
    	   			'src': url,
    	   			'type' : 'text/javascript'
    	}
    );
     $$("head")[0].appendChild(e);
    

    Not sure about when browsers finish loading and add the files into the dom tho, there could be significant delay before objects are loaded / added using this script method…

  2. Just a side note for these dynamic loaders: You do not need the type attribute because it is ignored when the src tag is used. The server is then relied on to give the correct mime type.

  3. @coogan: I’m not quite sure what you mean. JSON basically a programmer-friendly way to represent data.

    @Greg: Thanks for the info! I wasn’t aware of that.

  4. You don’t need to quote the name if it has no spaces.. Eg:

    var fancyFruit = {name:”pear”, color:”greenish”};

  5. “You don’t need to quote the name if it has no spaces..”

    Steve,

    Even-though in JavaScript it is unnecessary to quote object variable names, it is part of the language spec of JSON to ALWAYS do so, so in order to be compatible with all of the JSON parsers of the various languages, it is a better idea to go ahead and quote all object variable names in JSON data.

  6. Nice, but a bit late. I’ve ben using such a technique for quite a long time. Actualy full 10 years long – that’s a decade.

    But I never thought about giving it a name. It was simply a workaround to pass new content to div’s and I’ never managed to keep it as clean routine.

    Hope you do. :)

  7. We do a lot of json and XML stuff, it is wrong to claim that JSOn is a lot faster, the performance is often similar, and the size advantage isn’t that signifcant either

  8. @anonymous: Good points, it does depend on the data set. But in general JSON tends to be leaner as there’s less punctuation, no need for closing tags, etc. Also, it’s faster in javascript contexts since you don’t need to convert the XML to a javascript object (JSON can just be sent in directly).

  9. Normally, some teachers want to analyse the term paper titles creative writing ability of their students, however not all good students can to write correctly because of lack of knowledge or other reasons. Hence, a comparison essay service should help to create the literature term paper very fast.

  10. SARAH22xl, you’re a nutcase spamming robot. go eat some nuts (and bolts).

    In other news, good article. Oh, and to the guy who says he’s been doing something similar for 10 years and hopes you remember a name or whatever he was talking about, man, what drugs are you on and what are you talking about?

    Have a nice day.

  11. LINE GRAPH

    function graph(){
    // prepare chart data

    var sampleData = [

    { Day:'8 - 9', Keith:20},

    { Day:'9 - 10', Keith:50},

    { Day:'10 - 11', Keith:70},

    { Day:'11 - 12', Keith:30},

    { Day:'12 - 1', Keith:60},

    { Day:'1 - 2', Keith:10},

    { Day:'2 - 3', Keith:20}

    ];

    // prepare jqxChart settings

    var settings = {

    title: “Front End Graph”,

    description: “Time spent on various intervals”,

    padding: { left: 5, top: 5, right: 5, bottom: 5 },

    titlePadding: { left: 90, top: 0, right: 0, bottom: 10 },

    source: sampleData,

    categoryAxis:

    {

    dataField: ‘Day’,

    showGridLines: false

    },

    colorScheme: ‘scheme01′,

    seriesGroups:

    [

    {

    type: 'line',

    valueAxis:

    {

    minValue: 0,

    maxValue: 100,

    unitInterval: 10,

    description: 'Time in minutes'

    },

    series: [

    { dataField: 'Keith', displayText: 'Keith'},

    { dataField: 'Erica', displayText: 'Erica'},

    { dataField: 'George', displayText: 'George'}

    ]

    }

    ]

    };
    // select the chartContainer DIV element and render the chart.

    $(‘#chartContainer’).jqxChart(settings);

    }

Your feedback is welcome -- leave a reply!

Your email address will not be published.

LaTeX: $$e=mc^2$$