Most programming languages implement concurrency poorly, Erlang has taken a different approach to concurrency than today’s popular programming language, as we’ve seen, it is easy to create parallel processes of execution in Erlang. Lets look more closely at how to create and communicate with these processes in Erlang by creating a small (contrived and basic) program which simulates a buyer buying stocks from a seller.
We need three Erlang primitives to work with processes: spawn, ! and receive. The built-in spawn function creates a new process executing a function and returns the new process’ process id. The ! operator sends a message to a process and processes use the receive...end function to match messages in the mailbox to patterns and execute the appropriate functions.
To send a message “BUY MSFT” to the Seller process, we use:
Seller ! {{buy, msft} , Buyer}
we include the variable Buyer which contains the process id of the buyer, this lets the seller know which process to send the acknowledgment to. Now we need to create the Seller process, we do this by calling the spawn/3 function– the first argument to spawn is the module name, followed by the function name and ending with a list of arguments:
Seller_pid = spawn(exchange, seller, [])
Both the Buyer and Seller processes can handle the message using pattern matching in the receive..end clause, Here the Seller process receives the {buy, Symbol} message, processes it, and then acknowledges the transaction by sending an “ok” to the buyer:
seller() ->
receive
{finished, _} ->
io:format("Received finished. no one is buying~n", []);
{{buy, Symbol}, Buyer_pid} ->
io:format("sold ~s to buyer~n", [Symbol]),
Buyer_pid ! ok,
seller();
end.
Now, lets put these pieces together in a program in which the Buyer process buys Stocks from the Seller and in return, the Seller process acknowledges the sale; the program terminates when the Buyer has run out of Stocks to buy.
-module(exchange).
-export([start/0, buyer/2, seller/0]).
buyer([], Seller_pid) ->
Seller_pid ! {finished, self()},
%% note there's a ";" instead of a "." at the end of the next clause
io:format("buyer finished~n", []);
buyer(List, Seller_pid) ->
[Head|Tail] = List,
Seller_pid ! {Head, self()},
receive
ok ->
io:format("buyer received seller confirmation~n", [])
end,
buyer(Tail, Seller_pid).
seller() ->
receive
{finished, _} ->
io:format("Received finished. no one is buying~n", []);
{{buy, Symbol}, Buyer_pid} ->
io:format("sold ~s to buyer~n", [Symbol]),
Buyer_pid ! ok,
seller()
%% note there's, no ";" or "," before the end
end.
start() ->
Things_to_buy = [{buy, sina}, {buy, msft}, {buy, intc}],
Seller_pid = spawn(exchange, seller, []),
spawn(exchange, buyer, [Things_to_buy, Seller_pid]).
The program illustrates the most basic ways to use Erlang processes, I’ve completely skipped over guards, error handling, timeouts and registered processes, but several detailed documents on these topics and Erlang design principles are available on there ftp site.
Erlang’s main strength is its support for concurrency– now I’ll extend the previous Erlang example to fetch the stock quotes for multiple symbols in parallel.
Because I love abstractions, I’ll use the function pmap– pmap works like map, but when the function is called it creates one parallel process to evaluate each argument in the list. You can find a copy of the code on the book’s website. Generally, it’s not advisable to use pmap when the the list is very small or very large. Ideally, the implementation of pmap should be modified to spawn n processes, but that is something that I’m not comfortable doing yet.
Full Listing
1: -module(quote).
2: -import(json_parser).
3:
4: -export([get_stock_quote/1, get_data_in_parallel/0]).
5:
6: -define(BASE_URL, "http://www.google.com/finance/info?client=ig&q=".
7:
8: symbols() ->
9: [ "MSFT", "RHIE", "INTC", "DPTR",
10: "RVSB", "BBGI", "SRDD", "DEAR",
11: "ALKS", "GOOG", "QQQQ", "AAPL",
12: "RIMM", "GEOY", "CBST", "ANGO"
13: ].
14:
15:
16: get_google_url(Symbol) ->
17: ?BASE_URL ++ Symbol.
18:
19: get_stock_quote(Symbol) ->
20: %% moved inets:start() outside this function
21: URL = get_google_url(Symbol),
22: { ok, {_Status, _Headers, Body }} = http:request(URL),
23: PureData = lists:subtract(lists:subtract(Body, "// [ "), "] "),
24: {_,{_,RealData},_} = json_parser:dvm_parser(list_to_binary(PureData)),
25: RealData.
26:
27:
28: get_data_in_parallel() ->
29: inets:start(),
30: lib_misc:pmap(fun get_stock_quote/1, symbols()).
Inspired by Dave Thomas’ “A First Erlang” post I decided to use Erlang to retrieve stock quotes from Google’s Finance API. I wrote a simple Erlang program that let me explore third party JSON libraries and Erlang’s http library.
In Erlang, the -module directive defines an Erlang module, which is how code is organized in Erlang, the -export directive tells Erlang which functions in this module to expose. My Erlang module is called quote, and I’m storing it in the file quote.erl. I’m also exposing a single function get_stock_quote which accepts a single parameter.
-module(quote).
-export([get_stock_quote/1]).
Next I’ll define an Erlang macro called BASE_URL which contains the base URL of the Google Finance API. The function get_google_url builds the full URL by appending the symbol to the base URL.
-define(BASE_URL, "http://www.google.com/finance/info?client=ig&q=".
get_google_url(Symbol) ->
?BASE_URL ++ Symbol.
After retrieving a stock quote for MSFT in our browser I noticed that the data returned from Google Finance is a JSON(ish) object surrounded by some extraneous text which has to be removed before we can do anything else. Later I’ll use an external library to convert the string into a JSON object and extract the price and date.
get_stock_quote(Symbol) ->
%% Don't know why I need the following line
%% mentioned in inets documentation
inets:start(),
URL = get_google_url(Symbol),
{ ok, {_Status, _Headers, Body }} = http:request(URL),
PureData = lists:subtract(lists:subtract(Body, "// [ "), "] ").
The code to make http request is simple, http:request() returns the HTTP status, headers and body but the call was throwing an exception on my machine; after reading the http documentation included with Erlang I learned that a call inets:start() has to be made before making a http request. The variable PureData contains the string which will transform into our JSON object, I had to use the lists:subtract function to remove extra characters from the beginning and end of the string.
After some searching (and cursing) I found a third party library, json_parser, on Process One’s Comprehensive Erlang Archive Network. I downloaded a copy of the development version, renamed the source file to json_parser.erl, compiled json_praser and referenced it from quotes.erl with the -import(json_parser) import directive.
The documentation for json_parser is fairly sparse—the dvm_parser function in the library returns a tuple with more data then I need and I’m only interested in the actual JSON data which I’ parsed into the RealData variable and passed onto the parse_json_tuple function which extracts the fields I’m interested in.
{_,{_,RealData},_} = json_parser:dvm_parser(list_to_binary(PureData)),
parse_json_tuple(list_to_tuple(RealData)).
Finally, I parse the data in the RealData tuple and returned the CurrentPrice and Quote time:
parse_json_tuple(RealData) ->
%% this is ugly and needs to be refactored
{_,_,_,_,{_,CurrentPrice},_,{_,CurrentTime},_,_,_} = RealData,
{CurrentPrice, CurrentTime}.
To use the code, you have to compile the quote.erl file by calling the c(quote) function in the Erlang Shell, then call the get_stock_quote function with the a stock symbol: quote:get_stock_quote(“MSFT”) from the Erlang shell.
This isn’t the best use of Erlang, but I wanted to ease into Erlang before exploring the more complicated recursive and distributed functionality, later I’ll refactor the code to be more Erlang-y and modify the program to retrieve quotes in parallel using Erlang’s concurrency magic.
*Edit*I Just discovered that Google Finance returns a slightly different dataset when the market is closed, I’ll update the parse_json_tuple function with the changes soon.
Full Listing
1: -module(quote).
2: -export([get_stock_quote/1]).
3: -import(json_parser).
4:
5: -define(BASE_URL, "http://www.google.com/finance/info?client=ig&q=".
6:
7: get_google_url(Symbol) ->
8: ?BASE_URL ++ Symbol.
9:
10: parse_json_tuple(RealData) ->
11: %% this is ugly and needs to be refactored
12: {_,_,_,_,{_,CurrentPrice},_,{_,CurrentTime},_,_,_} = RealData,
13: {CurrentPrice, CurrentTime}.
14:
15: get_stock_quote(Symbol) ->
16: %% Don't know why I need the following line
17: %% mentioned in inets documentation
18: inets:start(),
19: URL = get_google_url(Symbol),
20: { ok, {_Status, _Headers, Body }} = http:request(URL),
21: PureData = lists:subtract(lists:subtract(Body, "// [ "), "] "),
22: {_,{_,RealData},_} = json_parser:dvm_parser(list_to_binary(PureData)),
23: parse_json_tuple(list_to_tuple(RealData)).
