TAME: Truly Asynchronous Messaging Environment

When we talk about asynchronous communication on the internet, we’re not really talking about pure asynchronicity.   When a JavaScript client makes a standard AJAX request, it is waiting for a response — a single response — and that completes that particular communication event.  The reason this is called “asynchronous” is because it happens after the page has loaded, and may re-occur at any given time.  Still, it remains a basic 1:1 relationship of request:response.  While this is effective for most Single Page Applications (SPA), it does not often lend itself to the more non-linear and/or time consuming tasks of Machine Learning, Predictive Analytics, Data Warehousing, Deep Learning, and so on.  These types of solutions require a non-linear communication methodology capable of carrying on an extended conversation with any given set of internal or external services, and communicate amongst themselves internally to share relevant discoveries that may be uncovered during a time consuming — or on-going — set of tasks.

TAME rises…

With this in mind, Capture Club developed the concept of a Truly Asynchronous Messaging Environment (TAME) several years ago when tasked with integrating a variety of disparate systems.  These systems had different data models, authentication schemes, end point access gateways, APIs, and so on.  We needed to be able to kick-off processes that may take some time, and be able to “peek” into that process at any given time, and provide status updates.  We needed to accommodate users of varying skill sets and with varying backgrounds.  In this case, we extended the TAME protocol and used the ZEN interface to provide periodic status updates to the users via a variety of communication methods.  The users could interact with ZEN by simply typing into a search box, or using Natural Language Processing.  On the back end, the TAME architecture enabled users to interact with remote web sites, creating or modifying active advertising campaigns from a single location.  From their, the system would push values out to sites like TradeDesk, etc, allowing campaign managers to rapidly modify the attributes of a given campaign.

TAME, untamed…

TAME relies on a communication model similar to, but distinct from, the way chat servers operate.  When an agent needs to run an extended task requiring more than 1 response, it opens a private, secure channel with the orchestrator.  The orchestrator is an elected leader, similar to the concept used in Apache’s Zookeeper.  It’s important to note here that this is not a hub->spoke relationship.  The orchestrator is a task like any other, but it asserts an ad hoc leadership for the duration of the event.  Generally speaking, the orchestrator would be the first task initiated.  This task would use data gathered from other running tasks to produce the final result.  The beauty of TAME is that it still very happily completes a single request:response event, but can also carry on the conversation for an indefinite period of time, all the while running a variety of supplementary tasks concurrently.