TOR Secure Messaging with Onion Services, a How-To

Secure Messaging with Onion Services, a How-To

meejah October 10, 2017


This post explores how Tor onion services can be integrated into existing web services, making them more secure. This integration will use the “publish / subscribe” pattern over Tor to trigger re-builds of the txtorcon documentation (which is hosted on an onion service). We will use Tor to transport the published messages so the network-location of the machine hosting the onion service remains hidden.

We will use a messaging system called “Web Application Messaging Protocol” or WAMP. This specifies “routed” Remote Procedure Calls (RPC) and Publish/Subscribe (PubSub) messaging over many transports. This means there is one central “router” to which any number of clients connect to communicate with each other (via RPC or PubSub).

We'll also use the Crossbar.io router for WAMP which recently added native support for Tor. We will connect to an onion service the router listens on for WebSocket connections.

Note: there is a companion blog post to this one which goes into much more technical detail about the specifics of the setup of Crossbar.io and includes a code repository; see my personal blog.

Why This Matters


The txtorcon documentation (clearnet) should be re-built whenever GitHub changes happen, and GitHub provides a WebHook service which can notify you whenever interesting changes happen. These notifications take the form of an HTTP POST.

Ordinarily, accepting such an HTTP request and rebuilding the documentation would be straightforward. However, an onion service is trying to hide its network location – giving GitHub some way to call this directly would reveal its location. Instead, we will run an agent on a public IP that listens for GitHub WebHook requests and forwards them as “publish” events over Tor to the Crossbar.io router. There will be another agent connected to the router that will subscribe to these forwarded GitHub events. This second agent rebuilds the documentation that is served by the txtorcon onion service.



There are three parts in the above diagram that we will deal with here:

• configure the Crossbar.io router to work over Tor
• write the GitHub Agent in Python
• write the Doc Builder in Python

The Crossbar Router


Crossbar.io is a Twisted application that provides a robust implementation of a “WAMP router”, routing RPC and PubSub messages between various connected clients. It supports many transports (stdin/out, Unix sockets, TCP sockets and WebSockets). For our purposes, we will use WebSockets over Tor, which Crossbar natively supports.

The complete details of Crossbar.io setup are in the `companion blog post`. The result of this is that the `Crossbar`_ router will use the ADD_ONION Tor Control Protocol command to add a new service to a running Tor instance. It will then be listening as a Tor onion service on a WebSocket URI that looks like ws://m6dazoly4sqnoqrm.onion:5000/.

The two clients we will write will then use this address to connect. Because onion URIs are self-certifying (see “step two” of Tor’s onion-services description) each client can be sure they’re talking to the correct Crossbar.io router.

The GitHub (WebHook) Agent


Although Crossbar.io has native support for turning WebHook-style requests directly into WAMP “publish” events, we don’t want the crossbar router itself on a public IP address. (One reason is because the private-key of our onion service would then be on a public machine). Therefore we have a simple service that listens on 443 for HTTP POSTs from GitHub and connects to Crossbar over Tor to the onion service (“GitHub Agent” in the diagrams).

This will do two things:

• listen on 443 for HTTPS connections from GitHub;
• and connect to Crossbar via Tor turning any WebHook calls into “publish” events

The documentation builder will subscribe to these topics and trigger builds when the source code changes. The setup will include a WebSocket connection via Tor onion service (speaking WAMP) and a web server listening for incoming GitHub WebHook POST requests.

Again, see the `companion blog post` for the full details; we use a Klein web server and txacme to get a (free!) Let’s Encrypt certificate for our server. So we accept TLS connections and verify the GitHub signatures before publishing any valid events as a WAMP “publish” message.

Building Documents


So now we have: a Crossbar.io router on a private machine and a WebHook agent on a public machine. The Crossbar.io machine is connected only via Tor, with no incoming connections allowed (all Tor connections are outbound).

The next step is to add the agent that actually re-builds the documentation on the machine serving the txtorcon documentation. This machine also is connected to the outside world only via Tor. This agent will connect to Crossbar.io via Tor and listen for “publish” events – that is, the GitHub WebHook announcements that are now being turned into WAMP “publish” messages.

For the full details, see the repository and `companion blog post`. What this agent does is to simply wait for “push” events to the “master” branch and then update the code and run the build commands.

Onions for All


We have set up a Crossbar.io router listening on a `Tor Project` onion service with a publicly-facing component that verifies and forwards GitHub “WebHook” requests. These requests are acted on by a second “builder” component that re-builds the txtorcon documentation. This keeps the network-location of the documentation host unknown.

Anyone can set up an onion service to integrate Tor into their software. Onion services are often considered “for anonymity only”, but they provide a number of other benefits:

• no incoming connections at all need be allowed on the onion service (in this case the Crossbar.io router machine)
• clients can be assured they’re talking to the correct host because onion domains are self-certifying (the address is a hash of the public-key of the service) without the need for a centralized Certificate Authority (CA)
• onion services are end-to-end encrypted, from the client to the service provider and traffic never leaves the Tor network; and
• onion services are difficult to block.

It’s well-known that onion services are more secure than services on the wider internet. This is why several large public web services like Facebook and DuckDuckGo provide consumers with the option to connect over Tor, even though they have no need of the anonymity feature. If you want to configure your own onion service, start here.

Continue reading...