Back to Erlang-fr French area.
(Part III)
The two previous articles presented ErlyStage main structure and how clients and server are going to exchange data. This specific task is done using a codec, (as coder/decoder). Two codecs are specified in the Atlas procotol specs. Here, we are going to review and implement the Packed codec.
[type][name=][data][|endtype]
The types are listed here :
For example, the following data :
[@id=17$name=Fred +28the +2b great+29#weight=1.5(args=@1@2@3)]
means, in Erlang : [ { "id", 17 }, { "name", "Fred the great" }, { "weight", 1.5 }, { "args", [1,2,3] } ]
The whole story here will be how to translate the encoded data into an Erlang variable. The first called function is codec_packed:decode(Data), which should return the { atlas, Result } :
decode( Data ) ->
% We should have a {atlas, [...]} format, so we check here
{ atlas, [Result] } = ?MODULE:packed_parse( Data, []) ,
% ok, now returns the correct format
{ atlas, Result }.
The main decoder function is codec_packed:packed_parse( Data, []). The first argument is the current encoded data, the second being the currently decoded one (none at the beginning).
We'll use pattern matching here to guess the type of the first element. The match will be on the first character of the Data ; for example, here is the test for an integer :
%% An integer
packed_parse( "@" ++ Rest, Data ) ->
{ Rest2, Name } = get_name( Rest ),
{ Rest3, Data2 } = packed_parse( integer, Rest2, []),
%% here again we should test if Name exists (FIXME)
packed_parse( Rest3, Data ++ [ make_hash( Name, Data2 ) ] );
If the match occurs, we call a specific packed_parse ( integer, ... ), after having retrieved the name of the variable.
%% Specific parser elements
%% ...
packed_parse( integer, Data, List ) ->
get_integer( Data ) ;
%% args : 1st is current data
%% 2nd is data that has been treated
%% 3rd is current integer value
%% the algorithm is simple : get the first character,
%% test if it's an integer.
%% If yes, Current*10 + Char,
%% else return Current value (we finished the parse )
get_integer( [], Done, CurrentI ) ->
{ [], CurrentI } ;
get_integer( [ H | Tail ], Done, CurrentI ) ->
case lists:member(H, ?INTEGER_TOKENS) of
true -> get_integer( Tail, Done ++ [H], CurrentI * 10 + (H - $0)) ;
false -> { [H] ++ Tail, CurrentI }
end.
The comments here should be self explanatory. The codec_packed module implements the same functions for all necessary types. The only special functions are for lists and maps. We use recursion to get all the elements into a list, but we need to get the closing tag (for example : ")" to end a list).
Let's see how the functions work for this simple example :
[@age=20$name=Shaman]
The first match is the beginning "[" :
%% Look for a map
packed_parse( "[" ++ Rest, Data ) ->
{ Rest2, Name } = get_name( Rest ),
{ Rest3, Data2 } = packed_parse( map, Rest2, []),
packed_parse( Rest3, Data ++ [ make_hash( Name, Data2) ] );
We first isolate the possible name (which there is not for the list), and then call
%% MAPS -----
packed_parse( map, "]" ++ Data, List ) ->
{ Data, List } ;
packed_parse( map, Data, List) ->
{ Rest, Data2 } = packed_parse( Data, []),
packed_parse( map, Rest, List ++ Data2);
The first time the function is called, the first argument is @age=20$name=Shaman]. This is not the end of the map, so we call the recursive packed_parse on the Data.
The matching function will be the integer one that we saw earlier. It will return { "age", 20 } and $name=Shaman] as the Rest (data still to be decoded). The next loop will detect a string :
packed_parse( "$" ++ Rest, Data ) ->
%% We first find if there is a name for this value
{ Rest2, Name } = get_name( Rest ),
{ Rest3, Data2 } = packed_parse( string, Rest2, [] ),
packed_parse( Rest3, Data ++ [ make_hash( Name, Data2 ) ] );
The corresponding packed_parse(string, ...) will return { "name", <<"Shaman">> }, and ] as data to be treated. This will then match the packed_parse( map, "]" ... ) which will return to the caller. This way, we have created the list [ { "age", 20 }, { "name", <<"Shaman">> } ]. Ok, I cheated, my age is not 20.. ;-)
One precision here : there no type «string» in Erlang, so we decided to use the binary type instead. That won't be a problem when displaying the string, if needed : we'll just have to use the BIF « binary_to_list ».
Ok, we now have the decoding part. Let's move on to encoding...
The encoding function is codec_packed:encode(Data, Acc, Status), where Data is the data to be encoded, and Acc the current encoded data.
Here is how the guards help :
encode(Data, Acc, Status) when list(Data) ->
lists:map(fun(X)-> encode(X, Acc, Status) end, Data);
This only happens when Data is a list. In this case, we apply the encoding on each item of the list by using the lists:map function. If the data were a tuple, the function would be
encode(Data, Acc, Status) when tuple(Data) ->
{atlas, Object_desc} = Data,
case is_map(Object_desc) of
true ->
{ok, "[" ++ compose_map(Object_desc) ++ "]"};
false ->
{error, 'The first level object should be a map'}
end.
This functions will pass on compose_map, which is responsible for iterating through the Object and analyze the elements of data :
compose_map([H|T], Acc) ->
{Name, Data} = H,
{Type, Data_as_string} = analyze(Data),
compose_map(T, Acc ++ pack(Type, Name, Data_as_string)).
The analyze is done using the guards we saw earlier. This translate to a tuple { Type, Data_as_string } that will then be used to encode according to the specs.
Here's two examples of analyzes :
analyze(Data) when integer(Data) ->
{integer, integer_to_list(Data)};
analyze(Data) when float(Data) ->
[Number] = io_lib:format("~f", [Data]),
{float, Number};
Quite simple. The pack(Type, Name, Data_as_string) for same types is listed here :
pack(integer, Name, Data_as_string) ->
"@" ++ naming(Name) ++ Data_as_string;
pack(float, Name, Data_as_string) ->
"#" ++ naming(Name) ++ Data_as_string;
For example, to encode the tuple { atlas, [ { "age", 20 }, { "name", <<"Shaman">>} ] } :
Then encode can return [@age=20$name=Shaman].
Here is how the Packed coded has been implemented. In fact, it may not be very difficult to make a XML codec using the same logic. That could be a good exercise ;-)
I hope those first three articles gave you a good idea of what is going on in Worldforge and how Erlang can help implement it. My task is now to get back to its development and to continue ErlyStage. I'll write some more articles as the code advances... So, see you soon !
Until then, you can meet me on irc.worldforge.org, channel #coders :-)
Thanks to Mickaël Rémond (a.k.a Stormy) for this second reading :-)