Inspired by Dave Thomas’ “A First Erlang” post I decided to use Erlang to retrieve stock quotes from Google’s Finance API.  I wrote a simple Erlang program that let me explore third party JSON libraries and Erlang’s http library.

In Erlang, the -module directive defines an Erlang module, which is how code is organized in Erlang, the -export directive tells Erlang which functions in this module to expose. My Erlang module is called quote, and I’m storing it in the file quote.erl. I’m also exposing a single function get_stock_quote which accepts a single parameter.

-module(quote).
-export([get_stock_quote/1]).

Next I’ll define an Erlang macro called BASE_URL which contains the base URL of the Google Finance API. The function get_google_url builds the full URL by appending the symbol to the base URL.

-define(BASE_URL, "http://www.google.com/finance/info?client=ig&q=".
get_google_url(Symbol) ->
  ?BASE_URL ++ Symbol.

After retrieving a stock quote for MSFT in our browser I noticed that the data  returned from Google Finance is a JSON(ish) object surrounded by some extraneous text which has to be removed before we can do anything else.  Later I’ll use an external library to convert the string into a JSON object and extract the price and date.

get_stock_quote(Symbol) ->
  %% Don't know why I need the following line
  %% mentioned in inets documentation
  inets:start(),
  URL = get_google_url(Symbol),
  { ok, {_Status, _Headers, Body }} = http:request(URL),
  PureData = lists:subtract(lists:subtract(Body, "// [ "), "] ").

The code to make http request is simple, http:request() returns the HTTP status, headers and body but the call was throwing an exception on my machine; after reading the http documentation included with Erlang I learned that a call inets:start() has to be made before making a http request.  The variable PureData contains the string which will transform into our JSON object, I had to use the lists:subtract function to remove extra characters from the beginning and end of the string.

After some searching (and cursing) I found a third party library, json_parser, on Process One’s Comprehensive Erlang Archive Network. I downloaded a copy of the development version, renamed the source file to json_parser.erl, compiled json_praser and referenced it from quotes.erl with the -import(json_parser) import directive.

The documentation for json_parser is fairly sparse—the dvm_parser function in the library returns a tuple with more data then I need and I’m only interested in the actual JSON data which I’ parsed into the RealData variable and passed onto the parse_json_tuple function which extracts the fields I’m interested in.

{_,{_,RealData},_} = json_parser:dvm_parser(list_to_binary(PureData)),
parse_json_tuple(list_to_tuple(RealData)).

Finally, I parse the data in the RealData tuple and returned the CurrentPrice and Quote time:

parse_json_tuple(RealData) ->
  %% this is ugly and needs to be refactored
  {_,_,_,_,{_,CurrentPrice},_,{_,CurrentTime},_,_,_} = RealData,
  {CurrentPrice, CurrentTime}.

To use the code, you have to compile the quote.erl file by calling the  c(quote) function in the Erlang Shell, then call the get_stock_quote function with the a stock symbol: quote:get_stock_quote(“MSFT”) from the Erlang shell.

This isn’t the best use of Erlang, but I wanted to ease into Erlang before exploring the more complicated recursive and distributed functionality, later I’ll refactor the code to be more Erlang-y and modify the program to retrieve quotes in parallel using Erlang’s  concurrency magic.

*Edit*I Just discovered that Google Finance returns a slightly different dataset when the market is closed, I’ll update the parse_json_tuple function with the changes soon.

Full Listing

   1:  -module(quote).

   2:  -export([get_stock_quote/1]).

   3:  -import(json_parser).

   4:  

   5:  -define(BASE_URL, "http://www.google.com/finance/info?client=ig&q=".

   6:  

   7:  get_google_url(Symbol) ->

   8:      ?BASE_URL ++ Symbol.

   9:  

  10:  parse_json_tuple(RealData) ->

  11:      %% this is ugly and needs to be refactored

  12:     {_,_,_,_,{_,CurrentPrice},_,{_,CurrentTime},_,_,_} = RealData,

  13:      {CurrentPrice, CurrentTime}.

  14:  

  15:  get_stock_quote(Symbol) ->

  16:      %% Don't know why I need the following line

  17:      %% mentioned in inets documentation

  18:      inets:start(),

  19:      URL =  get_google_url(Symbol),

  20:      { ok, {_Status, _Headers, Body }} = http:request(URL),

  21:      PureData = lists:subtract(lists:subtract(Body, "// [ "), "] "),

  22:      {_,{_,RealData},_} = json_parser:dvm_parser(list_to_binary(PureData)),

  23:      parse_json_tuple(list_to_tuple(RealData)).

Related posts:

1. More Erlang: Adding Concurrency
2. Processes in Erlang
3. Erlang Exercise: N Processes in a Star
4. Programming Collective intelligence in Erlang