Inspired by Dave Thomas’ “A First Erlang” post I decided to use Erlang to retrieve stock quotes from Google’s Finance API. I wrote a simple Erlang program that let me explore third party JSON libraries and Erlang’s http library.
In Erlang, the -module directive defines an Erlang module, which is how code is organized in Erlang, the -export directive tells Erlang which functions in this module to expose. My Erlang module is called quote, and I’m storing it in the file quote.erl. I’m also exposing a single function get_stock_quote which accepts a single parameter.
-module(quote). -export([get_stock_quote/1]).
Next I’ll define an Erlang macro called BASE_URL which contains the base URL of the Google Finance API. The function get_google_url builds the full URL by appending the symbol to the base URL.
-define(BASE_URL, "http://www.google.com/finance/info?client=ig&q=". get_google_url(Symbol) -> ?BASE_URL ++ Symbol.
After retrieving a stock quote for MSFT in our browser I noticed that the data returned from Google Finance is a JSON(ish) object surrounded by some extraneous text which has to be removed before we can do anything else. Later I’ll use an external library to convert the string into a JSON object and extract the price and date.
get_stock_quote(Symbol) ->
%% Don't know why I need the following line
%% mentioned in inets documentation
inets:start(),
URL = get_google_url(Symbol),
{ ok, {_Status, _Headers, Body }} = http:request(URL),
PureData = lists:subtract(lists:subtract(Body, "// [ "), "] ").
The code to make http request is simple, http:request() returns the HTTP status, headers and body but the call was throwing an exception on my machine; after reading the http documentation included with Erlang I learned that a call inets:start() has to be made before making a http request. The variable PureData contains the string which will transform into our JSON object, I had to use the lists:subtract function to remove extra characters from the beginning and end of the string.
After some searching (and cursing) I found a third party library, json_parser, on Process One’s Comprehensive Erlang Archive Network. I downloaded a copy of the development version, renamed the source file to json_parser.erl, compiled json_praser and referenced it from quotes.erl with the -import(json_parser) import directive.
The documentation for json_parser is fairly sparse—the dvm_parser function in the library returns a tuple with more data then I need and I’m only interested in the actual JSON data which I’ parsed into the RealData variable and passed onto the parse_json_tuple function which extracts the fields I’m interested in.
{_,{_,RealData},_} = json_parser:dvm_parser(list_to_binary(PureData)),
parse_json_tuple(list_to_tuple(RealData)).
Finally, I parse the data in the RealData tuple and returned the CurrentPrice and Quote time:
parse_json_tuple(RealData) ->
%% this is ugly and needs to be refactored
{_,_,_,_,{_,CurrentPrice},_,{_,CurrentTime},_,_,_} = RealData,
{CurrentPrice, CurrentTime}.
To use the code, you have to compile the quote.erl file by calling the c(quote) function in the Erlang Shell, then call the get_stock_quote function with the a stock symbol: quote:get_stock_quote(“MSFT”) from the Erlang shell.
This isn’t the best use of Erlang, but I wanted to ease into Erlang before exploring the more complicated recursive and distributed functionality, later I’ll refactor the code to be more Erlang-y and modify the program to retrieve quotes in parallel using Erlang’s concurrency magic.
*Edit*I Just discovered that Google Finance returns a slightly different dataset when the market is closed, I’ll update the parse_json_tuple function with the changes soon.
Full Listing
1: -module(quote).
2: -export([get_stock_quote/1]).
3: -import(json_parser).
4:
5: -define(BASE_URL, "http://www.google.com/finance/info?client=ig&q=".
6:
7: get_google_url(Symbol) ->
8: ?BASE_URL ++ Symbol.
9:
10: parse_json_tuple(RealData) ->
11: %% this is ugly and needs to be refactored
12: {_,_,_,_,{_,CurrentPrice},_,{_,CurrentTime},_,_,_} = RealData,
13: {CurrentPrice, CurrentTime}.
14:
15: get_stock_quote(Symbol) ->
16: %% Don't know why I need the following line
17: %% mentioned in inets documentation
18: inets:start(),
19: URL = get_google_url(Symbol),
20: { ok, {_Status, _Headers, Body }} = http:request(URL),
21: PureData = lists:subtract(lists:subtract(Body, "// [ "), "] "),
22: {_,{_,RealData},_} = json_parser:dvm_parser(list_to_binary(PureData)),
23: parse_json_tuple(list_to_tuple(RealData)).
