March 2015

This month’s speakers were Damian Beresford, Sheena Bailey and Luis Faustino

Damien talked about Tracing Node on Linux with LTTng, and Glen Keane showed a live demo of the nearForm Trace Web Controller, Sheena Bailey from Test Reach discussed an example of how Node has supported their business and finally Luis Faustino who talked about real time distributed messaging with NSQ.

Damian kicked off the meetup with his presentation on “Tracing Node on Linux with LTTng”. It was really interesting to hear what LTTng had to offer as we all know Node can be a bit lacking when it comes to debugging. This presentation showed a use case of tracing and how LTTng can be used to help find and solve the problem.

What is tracing and when can it be used?

Tracing is basically taking a snapshot or “X-ray” of your running system and processes. It has the ability to view kernel events and is therefore a low-level form of debugging. This is what makes it so powerful as it can capture common system calls such as open, connect and read just to mention a few. For each of these system calls there is an corresponding tracepoint: sys_enter_open, sys_enter_connect etc.
It’s used by both system admins and developers and aims to give clues as to what the problem may be. For example, if it’s a disk related issue, tracing may show many i/o messages or if it’s a network issue, the clues may be in the socket messages.

After an overview of what tracing was, Damian showed a use case of how it may be used in conjunction with Node. He used the example of a production system going down, with no pointers to the problem in any log files or system outputs. He recommended LTTng 2.x as the best app when it comes to tracing as it has very low overhead, 10 years of active development and is easy to install. It’s described as a suite of tools to help with tracing and comes with all of the tools necessary to implement tracing into your Node application.
The demo given was an example of kernel tracing with LTTng and involved the following steps via the command line:

lttng create demo -o demo ./demo (create a tracing session)  
lttng enable-event --userspace --all (enable all user space events)  
lttng enable-event --kernel --all (enable all kernel events)  
lttng start (start tracing)  
curl http://localhost:1337 (hit local Hello World Node application)  
lttng stop (stop tracing)  
babeltrace ./demo | more  
babeltrace ./demo | grep http_server (show output from scheduler)  
lttng list -k | more (show kernel events)  
lttng list -u (show ust events which are tracepoints that are placed in your own Node app. Express supports http_server_request and http_server_response for example)  

The above steps show how your Node application is interacting with the system at a low level and also shows network activity (ports, urls, http methods etc).
To summarise, LTTng offers:

  • High performance tracing of kernel and userspace events.
  • Remote Tracing
  • Can be switched on or off with a process restart
  • Acts as a flight recorder and live tracing
  • Supports common trace format

To add to Damian’s presentation, Glen Keane demoed the nearForm web trace controller that he developed. The nfTrace web controller is a web based application that allows you to create and view sessions for LTTng. It lists all running processes and their tracepoints and also lists kernel specific tracepoints. What I like about this app was how you can select individual tracepoints and events to be captured. You can also start tracing sessions directly from the web interface and nfTrace is clever enough to show only the tracepoints and events that you selected. It shows output from the scheduler directly to the web interface which is obviously much nicer to use that the cli based LTTng. nfTrace is available to download here.

You can watch his presentation here:

And you can access the slides from this presentation here

Next up we had Sheena Bailey from Test Reach who offer an online application that provides both online exam supervision and testing capabilities over the web. Sheena talked about how online learning has grown by 33% in the last year which is much higher than the growth of traditional learning. Test Reach is now in full production and acts as two apps in one as it offers both test authoring and online invigilation. It does this by allowing examiners to access students’ webcam and microphone during an exam.

As a relatively new business, Sheena decided to go with Node.js from the start after some research and a look at requirements. Although she’s had some issues getting features up and running, she is now starting to see the benefits of Node. From a business point of view, it offers many plugins and libraries and has a large community of developers working on it.
She talked about some of the libraries that Test Reach use and have found to be invaluable throughout their development. These include:

Some of the tools they use to support development, testing and deployment include:

Overall Test Reach have found that Node has been a huge success for their business and would recommend it to any other businesses that are considering using it to build a new platform or rebuild an existing system.

You can watch Sheena's presentation here:

The final speaker of the night was Luis Faustino from OpenJaw Technologies. He talked about NSQ which is a real time distributed messaging platform designed to operate at scale.
It was a completely new idea to me so it was interesting to see that there was already an implementation out there capable of handling billions of messages per day.

NSQ is a messaging queue system. Luis explained some of the advantages to NSQ, including: guaranteed message delivery, agnostic to data types sent and received, easy to install and deploy and it’s scalable. It also supports several communication patterns: unicast which is from a producer to one consumer, broadcast which sends the message to all consumers and finally multicast which sends to specific groups of consumers.

The best way to show the use case of something is to demo it, so Luis gave a quick overview of it by running the following commands:

nsqlookupd  

-This sets up the nsq directory service running on port 4160 and allows consumers to lookup addresses of nsq instances.

nsqd --lookupd-tcp-address=127.0.0.1:4160  

-This starts a new nsq instance pointing to the directory service. nsq instances regularly send data to the directory service, so that the directory service knows what address to give to consumers.

It also comes with an admin interface which shows details on messages currently in the queue and the nodes that will receive the messages. This can be started by running

nsqadmin --lookupd-http-address=127.0.0.1:4161  

To publish a message to the queue, you can run:

curl -d 'hello world 1' 'http://127.0.0.1:4151/put?topic=test'  

Start the client:

nsq_to_file --topic=test --output-dir=/tmp --lookupd-http-address=127.0.0.1:4161  

The client retrieves the address information of the nsq instance from the directory service.

Publish more messages to the nsq instance:

curl -d 'hello world 2' 'http://127.0.0.1:4151/put?topic=test'  

Browse to the nsq admin interface at http://localhost:4171/ and you can see the number of messages that have been sent.

Luis also showed an example of a Node.js express app that makes use of the npm nsq plugin. This plugin makes it possible to run the above commands and write messages to the queue directly from a Node application. For more information on NSQ, please see nsq.io.

You can watch Luis's presentation here: