Creating a Elixir Phoenix CRUD Web application and database and command line operations

1.1   A few notes on the Python code

Also included in the sample code is some Python code that can be used to do Web searches from the command line. It is built on top of ddgr which makes requests to DuckDuckGo (https://duckduckgo.com/) in order to perform searches. I've built a small amount of code on top of ddgr so that I can call its functionality from Python and format the output, as JSON, to be read by the escript described below.

In case you are asking yourself: why do that? One answer is that I am exploring the possibility that there can be some benefit to be had from storing Web search results for later manipulation and analysis.

2.1   Generating the application

Here is what I ran in order to generate my Elixir/Phoenix CRUD application:

# Create a new Phoenix project.
$ mix phx.new sedb
$ cd sedb
$ mix ecto.create
# Create the Web application and the database table.
$ mix phx.gen.html \
    Searchresults Searchresult searchresults \
    searchstr:string \
    title:string \
    abstract:string \
    url:string
# Apply the migration in the database.
$ mix ecto.migrate

Notes:

• The command $ mix phx.new sedb creates our skeleton project. Do $ mix help phx.new for more information.
• The command $ mix ecto.create creates our database and DB table. Do $ mix help ecto.create for more information.

Add the following line in lib/sedb_web/router.ex at the end of the scope "/" block:

resources "/searchresults", SearchresultController

That block might now look like the following:

scope "/", SedbWeb do
  pipe_through :browser

  get "/", PageController, :index
  resources "/searchresults", SearchresultController
end

Start your application server with one (but not both) of the following:

$ mix phx.server

Or, if you want access to the Elixir interactive interpreter, use this:

$ iex -S mix phx.server

Now, in your Web browser, visit http://localhost:4000/searchresults.

You should find a Web interface that enables you to create, read (and view), update, and delete (or CRUD) records from your new database.

If you want another way to inspect your database, you can start psql, the Postgresql interactive prompt with the following:

$ psql -h localhost -U postgres sedb_dev

And, while at the psql interactive prompt, do, for example:

sedb_dev=# select id, title, url from searchresults;

2.2   Writing, building, and using an escript

Assumptions:

1. The functionality of our escript will be implemented in an Elixir module named Sedb.CLI.
2. Our `` escript`` will want access to the database that is managed by our Phoenix Web applications, and will want to get that access through code generated in our Phoenix Web application.

In order to implement an escript do the following:

1. Add the following line in your mix.exs in the list returned by the project function:

   escript: [main_module: Sedb.CLI],
   

   Your project function might, for example, now look like this:

   def project do
     [
       app: :sedb,
       version: "0.1.0",
       elixir: "~> 1.12",
       elixirc_paths: elixirc_paths(Mix.env()),
       compilers: [] ++ Mix.compilers(),
       start_permanent: Mix.env() == :prod,
       aliases: aliases(),
       deps: deps(),
       escript: [main_module: Sedb.CLI],
     ]
   end
   

2. Create a file under the lib/ directory in your project that defines module Sedb.CLI. The module should implement a function main/1. This is the code that will be executed when you run your escript.

3. Compile/build your escript by running the following command in the root directory of your application:

   $ mix escript.build
   

4. Run your script from the command line. Given our sample script (below), you could, for example, do this:

   $ ./sedb list
   

Here is an example of a module that implements an escript for our sample project:

# file: lib/add_list_search_results.ex

defmodule Sedb.CLI do
  @moduledoc """
  synopsis:
    Add or list search result records in the project database.
    Read input from stdin.
    The input is JSON.

  usage:
    $ sedb <command>
        where <command> = "add" or "list".

  examples:
    $ cat input_file.json > ./sedb add
    $ ./sedb list
  """
  def main(args) do
    case args do
      ["add"] ->
        add_records()
      ["list"] ->
        list_records()
      _ ->
        IO.puts(@moduledoc)
    end
  end

  def add_records do
    jsonstr = IO.read(:stdio, :eof)
    {:ok, jasonstruct} = Jason.decode(jsonstr)
    searchstr = jasonstruct["searchstr"]
    items = jasonstruct["items"]
    items |> Enum.each(fn item ->
      item1 = Map.put(item, "searchstr", searchstr)
      IO.puts(~s(searchstr: #{item1["searchstr"]}))
      IO.puts(~s(title: #{item1["title"]}))
      IO.puts(~s(url: #{item1["url"]}))
      IO.puts(~s(abstract: #{item1["abstract"]}))
      IO.puts("----")
      {:ok, _} = Sedb.Searchresults.create_searchresult(item1)
    end)
  end

  def list_records do
    searchresults = Sedb.Searchresults.list_searchresults
    searchresults
    |> Enum.with_index()
    |> Enum.each(fn {searchresult, idx} ->
      IO.puts("#{idx + 1}. search result -- id: #{searchresult.id}:")
      IO.puts("    searchstr: #{searchresult.searchstr}")
      IO.puts("    title: #{searchresult.title}")
      IO.puts("    abstract: #{searchresult.abstract}")
      IO.puts("    url: #{searchresult.url}")
    end)
  end

end

Notes on the above escript:

• Our Elixir escript module must define a function main/1.
• That main function dispatches to one of several functions depending on whether a single command line argument is "add" or "list".
• Access to our database is provided by Sedb.Searchresults.create_searchresult(item1) and Sedb.Searchresults.list_searchresults. You can find additional database access functions implemented in module Sedb.Searchresults, which is in file lib/sedb/searchresults.ex.
• The "Add" capability reads the results of a Web search, formatted in JSON from standard input (stdin). and feeds each of those results as an Elixir map to the Phoenix generated function that adds the result as a new record to our Postgresql database.

A sample run:

# Insert the records into the database.
$ python json_from_search.py "lamb chops" > data.json    # Step 1
$ cat data.json | ./sedb add                             # Step 2
# List/print the records in the database.
$ ./sedb list                                            # Step 3

Notes:

• Script json_from_search.py is in the included code: phoenix_crud_app.zip.
• Step 1 performs the Web search and saves the results in data.json.
• Step 2 pipes the JSON data into our escript. The "add" command line argument instructs it to add the record to the database.
• Step 3 lists the records in the database.