Drab - Access the User Interface in browser from the Server Side

The jQuery in Elixir

In the "good old times" the event handling was easy. The only thing you had to do was to create the event handler (remember TForm1.Button1Click(Sender: TObject)?) and you could do everything inside this handler: accessing data from models, manipulating user interface, sending raw Ethernet packets to the coffee machine...

Nowadays, when the most of the applications are web based, it is not so easy anymore. You need either to create a form and submit the data via POST or GET or use AJAX to send or retrieve data. Whole interface manipulation is on the client side. In practice, you need to write two applications: on the client and the server side.

Additionally, because http is stateless and a one-way protocol, it is not easy to communicate back from the server to the browser. Like, for example, during execution of the long running process, when there is a need to update status (eg. progress bar), or to ask the operator about something (like "would you like to continue?"). Of course there are workarounds for this, like polling the server every few seconds to have a status, or even auto-refreshing the page, but those are just a dirty workarounds. The solution would be to allow an access to the interface directly from the server side. And here comes Drab.

Drab is a concept of providing an access to User Interface (DOM objects) on the server side, just like in the good-old Delphi times. The examples below should give you an overview of how does it work. For more detailed instructions, please refer to the documentation.

In general, Drab treats the client browser as a database provider and gives the way to select, update, insert and delete objects or object attributes:

select(something, from: selector)
insert(something, into: selector)
delete(selector) or delete(something, from: selector)
update(something, set: new_value, on: selector)

Drab is written using Phoenix on Elixir, so all the code in examples is HTML and Elixir (no Javascript!). This page is a living demo - all examples are actually working on the Drab server. You can find a source code of this page on Github, and the source code of the Drab himself - here.

Tomek "Grych" Gryszkiewicz 2016


Drab v0.3.2 - tests, tests and tests 2017-04-14

The new release introductes automated integration tests, finally. In addition, Drab package on github contains its own Phoenix Server, makes it easier to play with it.

Drab v0.3.1 released! 2017-03-16

Introducing Drab 0.3.1, version focused on error handling and debugging. Now you can run Drab functions directly from the IEx. It is fun! Learn more here. And here you can find out how Drab presents exceptions.

Drab v0.3.0 released! 2017-03-09

New version of Drab just arrived, announcing before_ and after_handler callbacks, new Drab.Query.select(:all), better error handling and presenting errors to the user, brand new waiter functionality, and more.

Warning: Drab.Query.select API has been changed! Now there are singular and plural versions of jQuery methods, for ex. select(:val) returns value of first found element or nil, and select(:vals) returns a Map of all found objects with their name or id as a keys.

There is a new demo down here, explaining the Drab Waiter functionality.


Simple Input and Button Example

Here is the simplest example - text input and the button. Clicking the button make input text uppercase. Awesome!

On the client side we have a basic HTML with the input and the button:

<input id="text_to_uppercase" value="uppercase me">
<button drab-event="click" drab-handler="uppercase">Do it</button>

The only thing which might be considered new is the pair of drab-event and drab-handler attributes. What do this values mean? drab-event is obviously a name of the event, and drab-handler is a name of the Elixir function on the server side, in DrabPoc.PageCommander module (see the code below). Clicking on the <button> in the browser runs DrabPoc.PageCommander.uppercase/2 on the server.

defmodule DrabPoc.PageCommander do
  use Drab.Commander
  def uppercase(socket, sender) do
    t = socket |> select(:val, from: "#text_to_uppercase")
    socket |> update(:val, set: String.upcase(t), on: "#text_to_uppercase")
  end
end

How does it work? Drab.Query.select(socket, :val, from: "#text_to_uppercase") launches jQuery $("#text_to_uppercase").val() on the client side and returns it to the server. Then, the returned string is converted to uppercase using String.upcase and passed to Drab.Query.update/3 function, which - analogically to jQuery .val(value) - sets the value of the input.

There is a number of jQuery methods mapped to Drab.Query.Select - like :html, :attr, etc. They all behave exactly like their jQuery equivalent - for example, Drab.Query.select(:html, ...) returns String with html of the first found DOM object. But what if you want to get html of all DOM object found? It is still possible with Drab - you just need to use the plural version of the method (:htmls instead of :html). This is going to return a map of all DOM object found, where key is the object name or id (or special string "__undefined_[number]", if neither name or id found) and the value is the return value of the method you run.
Below is the example of using both versions of select(:val) methods. Singular one just returns String, while plural - a Map with all found DOM objects processed.

name = socket |> select(:val, from: "#name")
# "Zdzisław"
name = socket |> select(:vals, from: "input")
# %{"name" => "Zdzisław", "surname" => "Dyrma"}

All the Drab functions (callbacks, event handlers) are placed in the module called Commander. Think about it as a Controller for the living page. Commander must have a corresponding controller (because the page to command must be rendered before), in this example PageCommander corresponds to PageController.

This is a live demo. Click the button to launch Drab.Controller.uppercase/2 on the server.


Access to the Sender DOM Object Properties

The next example shows how we can retrieve the information from data attribute. We've simple set up few buttons, each with the same event function clicked_sleep_button:

<button drab-click="clicked_sleep_button" data-sleep="1">Sleep 1 second</button>
<button drab-click="clicked_sleep_button" data-sleep="2">Sleep 2 seconds</button>
<button drab-click="clicked_sleep_button" data-sleep="3">Sleep 3 seconds</button>
Did you notice drab-click attribute? This is a shorthand for drab-event and drab-handler combination. There is a number of such shorthands for most popular events, see documentation for more.

On the server side the code is not very sophisticated as well. The whole functionality is to sleep for a few seconds. But the sleep interval is given with data property in dom_sender variable. Please notice that this variable contains more interesting values to be used, like val, html or text.

def clicked_sleep_button(socket, dom_sender) do
  socket |> update(class: "btn-primary", set: "btn-danger", on: this(dom_sender))
  :timer.sleep(dom_sender["data"]["sleep"] * 1000)
  socket |> update(class: "btn-danger", set: "btn-primary", on: this(dom_sender))
end
With this example we introduce this/1 function, which allows you to operate on the DOM object in similar way as with $(this) in JS. This is a way to use one event handler for multiple objects.

Observe that when one button is "processing", you can click another one. This is because Drab requests are asynchronous on the server side. And this is a reason why buttons are disabled by default while processing - it is blocking the user from running the same action for few times. You may override this behaviour in `config.exs` with:

config :drab, disable_controls_while_processing: false

Click the button to launch sleep








Tens of Tasks Running in Parallel on the Server and Communicating Back to the Browser

In the next example we will emulate long running server Process composed of a number of Tasks, which can be run asynchronously. We want to run it in parallel, change their status when they finish the job and - when all of them are done - change the status of the whole process.

HTML side contains button and few DIVs which displays the status of the process and tasks:

Async task status: 
<span id="async_task_status" class="label label-primary">
  ready
</span>
<span class="task label label-danger" data-task_id="1">Task 1</span>
  ...
<span class="task label label-danger" data-task_id="54">Task 54</span>
<button drab-click="run_async_tasks">Start async task</button>

The whole processing is on the server side. Clicking the button runs run_async_tasks/2 function on the server side:

def run_async_tasks(socket, dom_sender) do
  socket 
    |> update(class: "label-success", set: "label-danger", on: ".task")
    |> update(:text, set: "running", on: "#async_task_status")

  tasks = Enum.map(1..54, fn(i) -> Task.async(fn -> 
    :timer.sleep(:rand.uniform(4000)) # simulate real work
    socket |> update(class: "label-danger", set: "label-success", on: ".task[data-task_id=#{i}]")
    end)
  end)
  Enum.each(tasks, fn(task) -> Task.await(task) end)
  
  socket |> update(:html,
    set: "finished",
    on: "#async_task_status")
end
The code above runs asynchronously 54 Tasks, which just wait some time (up to 4 seconds) and then communicate to the user by changing the boostrap class from label-danger to label-success. After launching the background tasks, it waits (Task.await/1) for all of the tasks to finish and then informs the user by changing the Task status to "finished".

You may think: what is the difference between this approach and running 54 AJAX requests which launches action on the controller side? Well, except of Drab natural beauty, you can't do it with AJAX. You browser limits the number of simultaneus connections.

Click the button below to simulate server-side long running process contains asynchronous tasks.

Async task status: ready

Task #01
Task #02
Task #03
Task #04
Task #05
Task #06
Task #07
Task #08
Task #09
Task #10
Task #11
Task #12
Task #13
Task #14
Task #15
Task #16
Task #17
Task #18
Task #19
Task #20
Task #21
Task #22
Task #23
Task #24
Task #25
Task #26
Task #27
Task #28
Task #29
Task #30
Task #31
Task #32
Task #33
Task #34
Task #35
Task #36
Task #37
Task #38
Task #39
Task #40
Task #41
Task #42
Task #43
Task #44
Task #45
Task #46
Task #47
Task #48
Task #49
Task #50
Task #51
Task #52
Task #53
Task #54


Long Running Process with Communication Back to Browser

Let's move to some more complicated functionality. Assume having a process which is doing some stuff on the server side and we want to report back to the user after each completed step. We will use the progress bar controlled from the server.
Client side is very simple, again. Please notice the drab-handler attribute - this time it will run perform_long_process function on the server side.

<div class="progress">
  <div class="progress-bar" role="progressbar" style="width:0%">
  </div>
</div>
<button drab-click="perform_long_process">Click me to start processing ...</button>
On the server we simulate a long processing by sleeping some random time in each step. We will run start_background_process function. Main loop runs random number of steps, and each step just sleeps for a random time. After the nap, update_bar function in launched and sets the css("width") of the progress-bar to XX% and then, the html of the progress bar node to "XX%". We keep it doing it until it reaches 100%, then we insert(class: "progress-bar-success", into: ".progress-bar"). Lets repeat this code in English: insert the class progress-bar-success into the .progress-bar selector. This is why I prefer the pipe syntax - Drab functions are more readable with it.
Notice that updates can be stacked, because all Drab setter functions return socket.
def perform_long_process(socket, dom_sender) do
  socket
    |> execute(:hide, on: this(dom_sender))
    |> insert(cancel_button(socket, self()), after: "[drab-click=perform_long_process]")
  start_background_process(socket) 
end

defp start_background_process(socket) do
  socket |> delete(class: "progress-bar-success", from: ".progress-bar")
  steps = :rand.uniform(100)
  step(socket, steps, 0)
end

defp step(socket, last_step, last_step) do
  # last step, when number of steps == current step
  update_bar(socket, last_step, last_step)
  socket |> insert(class: "progress-bar-success", into: ".progress-bar")
  case socket |> alert("Finished!", "Do you want to retry?", buttons: [ok: "Yes", cancel: "No!"]) do
    {:ok, _} -> start_background_process(socket)
    {:cancel, _} -> clean_up(socket)
  end 
end

defp step(socket, steps, i) do
  :timer.sleep(:rand.uniform(500)) # simulate real work
  update_bar(socket, steps, i)
  step(socket, steps, i + 1)
end

defp update_bar(socket, steps, i) do
  socket 
    |> update(css: "width", set: "#{i * 100 / steps}%", on: ".progress-bar")
    |> update(:html, set: "#{Float.round(i * 100 / steps, 2)}%", on: ".progress-bar")
end
Just before running the background process, the event handler (perform_long_process/2 function) hides the main button (execute(:hide, on: this(dom_sender)) and then adds the Cancel button to the DOM tree with insert("button html", after: "[drab-handler=perform_long_process])". Translating it to English again, it means insert button html after all objects having drab-handler=perform_long_process properties. This shows the Cancel button, defined here:

defp cancel_button(socket, pid) do
  """
  <button class="btn btn-danger" 
          drab-click="cancel_long_process" 
          data-pid="#{Drab.tokenize_pid(socket, pid)}">
    Cancel
  </button>
  """    
end
This function builds the simple button html to be inserted into the DOM tree. What is interesting here, is that button has its event handler (function cancel_long_process/2) and data-pid property, containing the PID of the Elixir process which runs the steps loop (in this case it is self()). This is because we need to know which process to cancel!
Tokenizing the PID prevents data tampering (and also translates Elixir PID to String, so it could be inserted into the DOM tree). Remember that Tokens are signed, but not encrypted, so do not store any sensitive data in this way.

To be able to cancel the process we need to modify step/3 to receive notifications from the outside world. In our case, in each step system checks if there is a message to cancel process. In this case it cleans the stuff up, and finish. If there is no message like this, immediatel (after 0) continues the loop.

defp step(socket, steps, i) do
  :timer.sleep(:rand.uniform(500)) # simulate real work
  update_bar(socket, steps, i)
  # wait for a cancel message
  receive do
    :cancel_processing -> 
      clean_up(socket)
  after 0 -> 
    step(socket, steps, i + 1)
  end
end

def cancel_long_process(socket, dom_sender) do
  pid = Drab.detokenize_pid(socket, dom_sender["data"]["pid"])
  send(pid, :cancel_processing)
end
In the cancel button handler, you just need to get the tokenized pid from dom_sender["data"]["pid"] and decrypt it with Drab.detokenize_pid/2 function. Then send it a gentle message asking for not to continue.

Now take a look on Drab.Modal.alert/4 - the function which shows the bootstrap modal window on the browser and waits for the user input. Please notice that nothing is updated until you close the alert box. And because you can put your own form in the alert box, this is the easiest way to get the user input. Alert boxes return not only the name of the clicked button, but as well the names and values of all inputs in the modal window.

Click a button to simulate long-running process on the server.

Server-Side Events: Display the Growing File (tail -F)

Another great example of Drab usage is to display the files, which are changing over the time, like log files. In the normal, Ajax world, you need to send a request periodically to refresh the view, and you never know, if the file changed or not. What a waste of resouces!

With Drab, you can utilize the events on the Server Side: in this case to use fsevent to trace the changes in the file. So you upgrade the UI view only in case when something really happened.

To enable the Server-Side event watcher, we are going to introduce the Drab Callbacks. Callbacks are events launched automatically, when page is loaded, when browser reconnects or disconnects from the Server. To enable it you need to specify them in the Commander:

use Drab.Commander
onconnect :connected
onconnect launches every time client connects to the server, so after first load, network crashes, server crashes etc. There is also onload callback, which runs only once and ignores disconnects.

In the handler function we subscribe to Sentix file watcher (it is a wrapper around fswatch utility):

def connected(socket) do
  clean_up(socket)

  # Sentix is already started within application supervisor
  Sentix.subscribe(:access_log)

  file_change_loop(socket, Application.get_env(:drab_poc, :watch_file))
end
Watcher sends a message each time the file change. In this case we are going to update the #log_file div to present the last few lines of the file.
defp file_change_loop(socket, file_path) do
  receive do
    {_pid, {:fswatch, :file_event}, {^file_path, _opts}} ->
      socket |> update(:text, set: last_n_lines(file_path, 10), on: "#log_file")
  end
  file_change_loop(socket, file_path)
end

There is also the disconnect callback, which runs every time browser disconnects from the server - close the browser, navigate away from the page, network issue, etc. Obviously there is no way to show it on the Demo page, as it operates when the page is not visible anymore. In the example above, you may use it to terminate the file_change_loop.

This is a realtime view of few last lines of the access.log of this page


... this pane will update when access.log change ...
      


Is Drab Quick Enough? Plus Broadcast

You may concern about Drab performance - is it quick enough to handle events in the realtime? The best to find it out is to check - lets say the keyup event.

<input drab-keyup="changed_input" data-update="#display_placeholder">
<div id="display_placeholder"></div>
def changed_input(socket, dom_sender) do
  socket |> update!(:text, 
                 set: String.upcase(dom_sender["val"]),  
                  on: dom_sender["data"]["update"])
end

Did you notice the update!/2 (with bang!) function? This one works exactly like update/2, but it broadcasts to all the browsers on which the current page is opened. Open another browser window or even another browser and observe what is going on when you type in.

Demonstration of keyup event:

In many situation you don't need to react on every single key press, and you want to launch the event when user stop typing. The good example is autocomplete - you don't want to search the database after each keypress. This is the are where debounce() option becomes handy:

<input drab-keyup="changed_input" 
       drab-options="debounce(500)" 
       data-update="#display_placeholder2">
<div id="display_placeholder">Some text here</div>
The code above is exactly the same as previous, except of debounce(500) option. The same event handler (changed_input/2) is launched, but not after every key up, but when you actually stop typing for 500 miliseconds.

The same event with debounce(500):


Drab Store and access to Plug Session

Analogically to Plug, Drab can store the values in its own session. To avoid confusion with the session, we will call it a Drab Store. You can use two functions, put_store/3 and get_store/2 to read and write the values in the Store. It works exactly the same as a "normal", Phoenix session.

  • By default, Drab Store is kept in browser Local Storage. This means it is gone when you close the browser or the tab. You may set up where to keep the data with drab_store_storage config entry.
  • Drab Store is not the Plug Session! This is a different entity. Anyway, you have an access to the Plug Session (details below).
  • Drab Store is stored on the client side and it is signed, but - as the Plug Session cookie - not ciphered.

def increase_counter(socket, _dom_sender) do
  counter = get_store(socket, :counter) || 0
  put_store(socket, :counter, counter + 1)
end

def show_counter(socket, _dom_sender) do
  counter = get_store(socket, :counter)
  socket |> alert("Counter", "Counter value: #{counter}")
  socket
end



Although Drab Store is a different entity than Plug Session (used in Controllers), there is a way to access the Session. First, you need to whitelist the keys you wan to access in access_session/1 macro (you may give it a list of atoms or a single atom). Whitelisting is due to security: it is kept in Token, on the client side, so it is signed but not encrypted.
To show an example, we've put the key :drab_test to the Plug Session while rendering this page by the Controller:

defmodule DrabPoc.PageController do
  use DrabPoc.Web, :controller
  use Drab.Controller

  def index(conn, _params) do
    conn = put_session(conn, :drab_test, "test string in Plug Session, set in the Controller")
    render conn, "index.html"
  end
end
To use it in the Commander, you need to explicitly inherit this value:
defmodule DrabPoc.PageCommander do
  use Drab.Commander

  onload :page_loaded, 
  access_session :drab_test

  def page_loaded(socket) do
    socket 
    |> update(:val, set: get_session(socket, :drab_test), on: "#show_session_test")
  end
end
Notice that there is not way to update session from Drab. Session is read-only.

Result of get_session(socket, :drab_test) in the onload handler:


Drab Waiter

Sometimes it would be good to have a possibility to ask user about something, when you are in a middle of some process, and you don't want to finish your function. The good example is a database transaction, when you run some query and want to ask user wheter commit or rollback changes. We saw the use of Drab.Modal before, but sometimes modal window is not very elegant.

So the solution for this issue is called Drab Waiter. It is a tiny little DSL which stops processing the event handler function and waits for the event from the client side. You may register multiple events as well as a timeout (by default it is waiting forever). The example below is not very sophisticated, but shows the idea.

  def waiter_example(socket, _dom_sender) do
    buttons = Phoenix.View.render_to_string(DrabPoc.PageView, "waiter_example.html", [])
    socket 
      |> delete(from: "#waiter_answer_div")
      |> insert(buttons, append: "#waiter_example_div")

    answer = waiter(socket) do
      on "#waiter_example_div button", "click", fn(sender) ->
        sender["text"]
      end
      on_timeout 5000, fn ->
        "six times nine"
      end
    end

    socket 
      |> delete(from: "#waiter_example_div")
      |> update(:text, set: "Do you realy think it is #{answer}?", on: "#waiter_answer_div")
  end
Please notice that Waiter Example button is disabled until you give an answer (or until timeout). It is because Drab re-enables it when Event Handler Function finish processing.




Handling Errors and Exceptions

Drab intercepts all exceptions from event handler function and let it die, but before it presents the error message in the logs, and, for development environment, on the page. For production, it shows just an alert with the generic error.
Please notice that an error in the specific handler function will not harm anything else: all the processing are going on normally.

def raise_error(_socket, _dom_sender) do
  map = %{x: 1, y: 2}
  # the following line will cause KeyError
  map.z
end
By default it is just an alert(), but you can easly override it by creating the template in priv/templates/drab/drab.handler_error.prod.js with your own javascript presenting the error.

There is much descriptive message in the logs. The same message is shown within the alert on development environment.

[error] Drab Handler failed with the following exception:
** (KeyError) key :z not found in: %{x: 1, y: 2}
    (drab_poc) web/commanders/page_commander.ex:183: DrabPoc.PageCommander.raise_error/2
    lib/drab.ex:178: anonymous fn/5 in Drab.handle_event/6

But what when the Handler Process is killed for some other, than the exception, reason - like being brutally murdered? The story is quite similar - we can show the error banner, but in this case there is no way to clean up. In practice the button will stay disabled.

def self_kill(_socket, _dom_sender) do
  Process.exit(self(), :kill)
end


Debugging

There is an easy way to check and debug Drab functions directly in the IEx. The only you need to do is to run your Drab-enabled phoenix server (you can get one with pre-installed Phoenix and Drab from https://github.com/grych/drab-example) with IEx: iex -S mix phoenix.server. Then, connect with the browser to http://localhost:4000 and observe the IEx console. When started, Drab will show you some important debugging information:

[debug] 
    Started Drab for /drab, handling events in DrabPoc.PageCommander
    You may debug Drab functions in IEx by copy/paste the following:
import Drab.Core; import Drab.Query; import Drab.Modal; import Drab.Waiter
socket = GenServer.call(pid("0.663.0"), :get_socket)

    Examples:
socket |> select(:htmls, from: "h4")
socket |> execjs("alert('hello from IEx!')")
socket |> alert("Title", "Sure?", buttons: [ok: "Azaliż", cancel: "Poniechaj"])

As suggested, copy those two lines into IEx and now you can remote control your browser from IEx. Very handy for learning Drab and debugging!

iex(1)> import Drab.Core; import Drab.Query; import Drab.Modal; import Drab.Waiter
Drab.Waiter
iex(2)> socket = GenServer.call(pid("0.663.0"), :get_socket)
%Phoenix.Socket{assigns: %{__action: :index,
   __controller: DrabPoc.PageController, __drab_pid: #PID<0.663.0>,
   ... 
   transport_pid: #PID<0.1548.0>}

iex(3)> socket |> select(:htmls, from: "h4")
%{"__undefined_0" => "\n  The jQuery in Elixir\n",
  "__undefined_1" => "Async task status: <span id=\"async_task_status\" class=\"label label-primary\">ready</span>",
  "__undefined_2" => "\n  ©2016 Tomek \"Grych\" Gryszkiewicz, <a href=\"mailto:grych@tg.pl\">grych@tg.pl</a>\n",
...
  "tail_dash_f" => "Server-Side Events: Display the Growing File (tail -F)",
  "tens_of_processes" => "Tens of Tasks Running in Parallel on the Server and Communicating Back to the Browser",
  "waiter" => "Drab Waiter"}

Please be aware that Drab creates its own process for every single connection, so if you open a page in multiple tabs or browsers, you will have to handle with few different Drab PIDs. Also watch out for automatic page reload after code change - this will re-launch Drab as well, it will change its PID.


©2016 Tomek "Grych" Gryszkiewicz, grych@tg.pl


Warning: this is an early beta version, a lot of things are still not implemented, some are hardcoded. Any comments, criticism, thoughts are welcome - grych@tg.pl. Your feedback is highly appreciated!


Illustrations by www.Vecteezy.com