Designing and implementing a REST API (with Webmachine)

Contents

1 Introduction and preliminaries
2 Implementation

1 Introduction and preliminaries

This post explains the implementation of a basic CRUD interface to an Erlang ETS database table. (CRUD: create, read, update, delete. See https://en.wikipedia.org/wiki/Create,_read,_update_and_delete).

For testing I use the following:

cURL -- cURL is a command line tool. And, it is scriptable. So, it is relatively easy to write scripts in a scripting language (for example, Python, Bash, Escript/Erlang, etc.) that test complex actions against a resource. You can learn more about cURL here: https://curl.haxx.se/
RESTclient -- RESTclient is an add-on for the Firefox web browser that makes it easy to make REST request. You can learn about it in Firefox by searching add-ons for "RESTclient".
HTTPRequester -- HTTPRequester is an add-on for Firefox that enables you to send REST requests.

You should consider trying each of the above. I suspect that you will find one suitable for some purposes and another for others.

Approach -- This document will follow the strategy of designing a REST API for a single resource (type). In this document, we'll learn how to implement and use a REST API that does each of the following for an example resource type:

Create an instance of the resource.
List existing instances of the resource.
Retrieve a specific instance of a resource.
Update specific attributes of a specific instance of a resource.
Delete a specific instance of a resource.

2 Implementation

The complete source files from which we'll be taking samples of code are here:

I initially generated this application with rebar3 and the Webmachine template for rebar3. See blog post Erlang Webmachine REST -- How-to and notes.

2.1 Create an instance of the resource

To create an instance, we use the POST method. Here is what that request might look like using cUrl:

curl \
    -v \
    --data "{\"type\": \"new_document\", \"body\": \"my simple body 1\"}" \
    --header "Content-Type: application/json" \
    http://crow.local:8080/docs

We need to provide Webmachine with some routing and configuration information:

-module(myapp_config).

-export([
    dispatch/0,
    web_config/0
]).

-spec dispatch() -> [webmachine_dispatcher:route()].
dispatch() ->
    lists:flatten([
        {[], myapp01_resource, [{operation, one}]},
        {["docs"], myapp01_doc_resource, [{operation, two}]},
        {["docs", recid], myapp01_doc_resource, [{operation, three}]},
        {["docs", "delete", recid], myapp01_doc_resource, [{operation, delete}]}
    ]).

web_config() ->
    {ok, App} = application:get_application(?MODULE),
    {ok, Ip} = application:get_env(App, web_ip),
    {ok, Port} = application:get_env(App, web_port),
    {ok, Tablename} = application:get_env(App, table_name),
    io:format("(web_config) Port: ~p~n", [Port]),
    io:format("(web_config) Tablename: ~p~n", [Tablename]),
    % Create an ETS table and insert the initial ID number.
    ets:new(Tablename, [set, named_table, public]),
    ets:insert(doc_table, {current_id, 0}),
    [
        {ip, Ip},
        {port, Port},
        {log_dir, "priv/log"},
        {dispatch, dispatch()}
    ].

Notes:

Function dispatch/0 sets up the routes (URI paths) that we'll use in this sample application.
Function web_config/0 (1) retrieves some configuration information, (2) creates and initializes our ETS database; and (3) passes configuration information back to Webmachine.

The port and IP address are specified in myapp/config/sys_config.erl:

[
 {'myapp',
  [
   {table_name, doc_table},
   {web_ip, "0.0.0.0"},
   {web_port, 8080}
  ]
 }
].

In order to use and handle the POST method, we need the following set-up:

-spec content_types_accepted(wrq:reqdata(), term()) ->
    {[{MediaType::string(), Handler::atom()}], wrq:reqdata(), term()}.
content_types_accepted(ReqData, State) ->
    {[{"application/json", from_json}], ReqData, State}.

%
% Tell Webmachine that we use the POST method to create an instance
% of our resource.
-spec post_is_create(wrq:reqdata(), term()) -> {boolean(), wrq:reqdata(), term()}.
post_is_create(ReqData, State) ->
    {true, ReqData, State}.

%
% If post_is_create/2 returns true, then this method is required.
% Return the current path unchanged.
-spec create_path(wrq:reqdata(), term()) -> {string(), wrq:reqdata(), term()}.
create_path(ReqData, State) ->
    Path = wrq:disp_path(ReqData),
    {Path, ReqData, State}.

%
% Tell Web machine that we also want to use the POST method.
-spec allowed_methods(wrq:reqdata(), term()) -> {string(), wrq:reqdata(), term()}.
allowed_methods(ReqData, State) ->
    Methods = ['GET', 'HEAD', 'POST', 'DELETE'],
    {Methods, ReqData, State}.

Notes:

content_types_accepted/2 tells Webmachine that we want a POST request whose payload is JSON to be handled by from_json/2.
Webmachine requires that if we want to use the POST method to create an instance of our resource, then we must implement post_is_create/2 and it must return true.
Webmachine requires that if post_is_create/2 returns true, then we must implement create_path/2. Here we just return the path unchanged.
And, we use allowed_methods/2 to tell Webmachine that we want to use the POST method. That is not the default, so we must explicitly implement it.

And, now we implement the request handler itself:

%
% Handle a POST request that submits JSON content.
% Sample request:
%     curl \
%         -v \
%         --data "{\"type\": \"new_document\", \"body\": \"my simple body 1\"}" \
%         --header "Content-Type: application/json" \
%         http://crow.local:8003/docs
%
-spec from_json(wrq:reqdata(), term()) -> {iodata(), wrq:reqdata(), term()}.
from_json(ReqData, State) ->
    Body = wrq:req_body(ReqData),
    {struct, Body1} = mochijson:decode(Body),
    Body2 = proplists:get_value("body", Body1),
    {ok, _Id} = add_doc_body(Body2),
    {true, ReqData, State}.

add_doc_body(Body) ->
    [{current_id, Id}] = ets:lookup(doc_table, current_id),
    Id1 = Id + 1,
    ets:insert(doc_table, {Id1, Body}),
    ets:insert(doc_table, {current_id, Id1}),
    {ok, Id1}.

Notes:

We extract the payload (body) from the ReqData, decode the JSON into an Erlang term, and then extract the content of the document we are going to store as a resource.
We increment the current ID to get a unique ID and then add the ID and document tuple to the ETS database.
And, finally, we increment the current ID in the ETS database.

2.2 Retrieving resources and their IDs

These operations enable us to list the IDs of existing instances of the resource and to retrieve a specific instance of a resource.

Here is what our requests will look like using cUrl:

#
# Retrieve the doc IDs as JSON.
curl \
    --data "{\"type\": \"new_document\", \"body\": \"my simple body 4\"}" \
    --header "Content-Type: application/json" \
    http://crow.local:${PORT}/docs
#
# Retrieve the doc IDs as HTML.
curl http://crow.local:${PORT}/docs

Notes:

To retrieve all document IDs, we use the URI http://crow.local:${PORT}/docs.
To retrieve the document (body) itself, we use the URI http://crow.local:${PORT}/docs/NN, where NN is a specific document ID.

In our example, we implement two functions to handle GET method requests: (1) one (to_html/2) handles requests for HTML content, and (2) the other (to_json/2) handles requests for JSON content.

We need to explicitly tells Webmachine that we want to handle requests for HTML and JSON content:

-spec content_types_provided(wrq:reqdata(), term()) ->
    {[{MediaType::string(), Handler::atom()}], wrq:reqdata(), term()}.
content_types_provided(ReqData, State) ->
    Types = [{"text/html", to_html}, {"application/json", to_json}],
    {Types, ReqData, State}.

And, here are the request handlers (and supporting functions) themselves:

-spec to_html(wrq:reqdata(), term()) -> {iodata(), wrq:reqdata(), term()}.
to_html(ReqData, State) ->
    Recid = wrq:path_info(recid, ReqData),
    Data = get_requested_data(Recid),
    Content = case Data of
        {ids, Ids} ->
          Content1 = io_lib:format("<p>Ids: ~p</p>~n", [Ids]),
          Content1;
        {body, Body} ->
          Content1 = io_lib:format("<p>Id: ~p  Body: ~p</p>~n",
                                   [Recid, Body]),
          Content1;
        not_found ->
          Content1 = io_lib:format("<p>not found</p>~n", []),
          Content1
        end,
    {Content, ReqData, State}.

%
% If the Accept header of the request specifies application/json,
% then deliver JSON content.
-spec to_json(wrq:reqdata(), term()) -> {iodata(), wrq:reqdata(), term()}.
to_json(ReqData, State) ->
    Recid = wrq:path_info(recid, ReqData),
    Data = get_requested_data(Recid),
    Content1 = case Data of
        {ids, Ids} ->
            {struct, [{ids, {array, Ids}}]};
        {body, Body} ->
            {struct, [{id, Recid}, {body, Body}]};
        not_found ->
            {struct, [{not_found, true}]}
    end,
    Content = mochijson:encode(Content1),
    {Content, ReqData, State}.

get_requested_data(Recid) ->
    Data = case Recid of
                  undefined ->
                      Ids = get_doc_id_list(),
                      {ids, Ids};
                  _ ->
                      Recid1 = list_to_integer(Recid),
                      %io:format("(get_requested_data) 2. Recid1: ~p~n", [Recid1]),
                      Result = get_doc_body(Recid1),
                      case Result of
                          not_found ->
                              not_found;
                          {ok, Body} ->
                              {body, Body}
                      end
              end,
    Data.

get_doc_id_list() ->
    Records = ets:match(doc_table, '$1'),
    %Ids = lists:map(fun (X) -> [{Id, _Body}] = X, Id end, Records),
    Ids = lists:foldl(fun (Item, Acc) ->
                              [{Id, _Body}] = Item,
                              Acc1 = case Id of
                                  current_id -> Acc;
                                  _ ->
                                      [Id | Acc]
                              end,
                              Acc1
                      end,
                      [], Records),
    Ids.

get_doc_body(Recid) ->
    Result = ets:lookup(doc_table, Recid),
    case Result of
        [] ->
            not_found;
        [{_Id, Body}] ->
            {ok, Body}
    end.

Notes:

Both to_html/2 and to_json/2 format content and return it.
We use mochijson to convert our Erlang structure to JSON text content. mochijson is included with mochiweb, which is included with Webmachine, since Webmachine is built on top of mochiweb.
In order to determine what Erlang structure I needed to create (so that I could decode it to JSON text, I:
1. Created an example of the target structure in Python;
2. Used the Python json module to dump it to a string;
3. Used mochijson:decode/1 to turn it into an Erlang structure;
4. Created that structure in my Erlang code; and
5. Used mochijson:encode/1 to produce JSON content.

2.3 Update specific attributes of a specific instance of a resource

In our simple example, we will want to replace the body associated with a record ID with a new body (text content). This seems like a use for the HTTP PUT method.

Here we'll be attempting to respond to an HTTP PUT method. Here is what such a request might look like in cUrl:

curl \
    --request PUT \
    --data "{\"type\": \"new_document\", \"body\": \"my updated body 10\"}" \
    --header "Content-Type: application/json" \
    http://crow.local:${PORT}/docs/update/3

First, we need to add a line to dispatch/0 in myapp.src/myapp_config.erl:

-spec dispatch() -> [webmachine_dispatcher:route()].
dispatch() ->
    lists:flatten([
        {[], myapp01_resource, [{operation, one}]}
        , {["docs"], myapp01_doc_resource, [{operation, two}]}
        , {["docs", recid], myapp01_doc_resource, [{operation, three}]}
        , {["docs", "delete", recid], myapp01_doc_resource, [{operation, delete}]}
        , {["docs", "update", recid], myapp01_doc_resource, [{operation, update}]}
    ]).

And, tell Webmachine that we want to handle HTTP PUT method requests. To do that we modify allowed_methods/2 in myapp.src/myapp_doc_resource.py:

%
% Tell Web machine that we also want to use the POST, PUT, and DELETE methods.
-spec allowed_methods(wrq:reqdata(), term()) -> {string(), wrq:reqdata(), term()}.
allowed_methods(ReqData, State) ->
    Methods = ['GET', 'HEAD', 'POST', 'DELETE', 'PUT'],
    {Methods, ReqData, State}.

And, finally, the methods that do the work:

%
% Handle a POST or PUT request that submits JSON content.
% Sample request:
%     curl \
%         -v \
%         --data "{\"type\": \"new_document\", \"body\": \"my simple body 1\"}" \
%         --header "Content-Type: application/json" \
%         http://crow.local:8003/docs
% Or:
%     curl \
%         --request PUT \
%         --data "{\"type\": \"new_document\", \"body\": \"my updated body 10\"}" \
%         --header "Content-Type: application/json" \
%         http://crow.local:${PORT}/docs/update/3
%
-spec from_json(wrq:reqdata(), term()) -> {iodata(), wrq:reqdata(), term()}.
from_json(ReqData, State) ->
    Method = wrq:method(ReqData),
    case Method of
        'PUT' ->
            Recid = wrq:path_info(recid, ReqData),
            Body = wrq:req_body(ReqData),
            {struct, Body1} = mochijson:decode(Body),
            Body2 = proplists:get_value("body", Body1),
            ok = update_doc_body(Recid, Body2);
        _ ->
            Body = wrq:req_body(ReqData),
            {struct, Body1} = mochijson:decode(Body),
            Body2 = proplists:get_value("body", Body1),
            {ok, _Id} = add_doc_body(Body2)
    end,
    {true, ReqData, State}.

update_doc_body(Recid, Body) ->
    Recid1 = list_to_integer(Recid),
    ets:insert(doc_table, {Recid1, Body}),
    ok.

Notes:

We modify from_json/2 so that it does something different with an HTTP PUT request from what it does with a POST request.
Since our ETS table is a set, and since the body is the only part of the record, we merely replace it by inserting a new record with the same ID.

2.4 Delete a specific instance of a resource

Here we'll be attempting to respond to an HTTP DELETE method. Here is what such a request might look like in cUrl:

curl \
    --request DELETE \
    http://crow.local:8080/docs/delete/1

Webmachine requires us to do several things in order to respond to a DELETE method.

We add a new route to our configuration file myapp/src/myapp_config.erl:

-spec dispatch() -> [webmachine_dispatcher:route()].
dispatch() ->
    lists:flatten([
        {[], myapp01_resource, [{operation, one}]},
        {["docs"], myapp01_doc_resource, [{operation, two}]},
        {["docs", recid], myapp01_doc_resource, [{operation, three}]},
        {["docs", "delete", recid], myapp01_doc_resource, [{operation, delete}]}
    ]).

Notes:

The line containing "delete" is the line that is relevant here. It is intended to tell Webmachine to respond to the URI in the cUrl DELETE request above.

We need to tell Webmachine that we want to accept DELETE requests:

%
% Tell Web machine that we also want to use the POST and DELETE methods.
-spec allowed_methods(wrq:reqdata(), term()) -> {string(), wrq:reqdata(), term()}.
allowed_methods(ReqData, State) ->
    Methods = ['GET', 'HEAD', 'POST', 'DELETE'],
    {Methods, ReqData, State}.

And, we must implement the callback function delete_resource/2 that Webmachine will call when it receives a DELETE method request:

-spec delete_resource(wrq:reqdata(), term()) -> {boolean() | term(), wrq:reqdata(), term()}.
delete_resource(ReqData, State) ->
    Recid = wrq:path_info(recid, ReqData),
    Result = delete_doc_resource(Recid),
    Status = case Result of
    ok ->
        true;
    _ ->
        false
    end,
    {Status, ReqData, State}.

delete_doc_resource(Recid) ->
    Recid1 = list_to_integer(Recid),
    ets:delete(doc_table, Recid1),
    ok.

Notes:

Don't forget to add delete_resource/2 to our export directive.
We retrieve the record ID from the URI path.
We delete the entry from our ETS table.

Designing and implementing a REST API (with Webmachine)

1 Introduction and preliminaries

2 Implementation

2.1 Create an instance of the resource

2.2 Retrieving resources and their IDs

2.3 Update specific attributes of a specific instance of a resource

2.4 Delete a specific instance of a resource

Published

Category

Tags

Contact