This is the second article in our four part series on measuring email. Last week we talked about the role of internal email in your organization.

If you're wondering why communicators are even measuring their email, or how you can present these ideas to someone else in your organization who understands less about this space -- start with last week's article!

This article is intended to give you the information you need to dive in and pick an email measurement tool. Both of the options we mention are ones that you try for free, right now -- Bananatag and Mailchimp. We've chosen these tools because they are the ones we've encountered being used and talked about the most by communicators. For those looking to implement a tool this year, these are two of the best all around options.

These services can give you the same metrics (opens, clicks etc.) although they are quite different in how they work and even their intended use, which will become clear by the end of this article.

Obligatory disclaimer about this article: Since we are writing about our own service here (Bananatag) we can't claim to be completely free of bias. Our entire product team is obsessively focused on making our service the most useful that it can be, and this probably influences us more than we can easily discern. That said, our marketing team uses Mailchimp on a weekly basis for our external emails (including possibly the email that brought you to this article). We genuinely like their products too, and are long time customers.
 

Internal Email Measurement Workflow

How do these email tools work into the daily life of an internal communicator? Which will make your team more productive? Let's tackle this area first.

We'll look at the user interface and workflow integration, email template features, and general customization options for each.

User interface

How each system is designed to be used is one of the primary differentiating factors between Bananatag and Mailchimp.

With Bananatag, emails are tracked right from Outlook, so you only need to use a system other than your email client when you want to view metrics.

Creating an email in Mailchimp happens entirely in their platform, accessible through your web browser -- there is no way to directly send to an existing distribution list with Mailchimp. How important this is to you will largely depend on your workflow and personal preferences, and we'll cover the main differences below.

Maintaining lists of employees

In Mailchimp, your list of employees needs to be up to date in their system before you click send. This can be done manually or by uploading an updated spreadsheet with all current employees before sending each email. This requires you to define a process that ensures that lists are up to date each time an email needs to be sent if it's critical that all current employees receive it. (The most common complaint from communicators relating to this issue has to do with new employees not receiving communications.)

One consideration to look at if you are considering Mailchimp is where your organization stores their master lists of employee emails and work groups, and how your internal comms team can get access to these lists on an ongoing basis.

Since Bananatag is an extension for your mail client, you can send to the distribution lists that are maintained by your IT or HR department. This means you only have one list across your organization and as an internal communicator, you may not even need to ever think about maintaining it.

Sending an internal email through MailChimp

Mailchimp is designed to guide you through the process of sending a mass email, and although their interface is intuitive, this is often where the learning curve steepens. For every email, you'll need to define many of the details that are automatically set or more familiarly laid out if you've become accustomed to sending internal emails through your mail client. It's a five step process every time.

Rather than just composing a new email, you'll need to create a campaign (or duplicate an existing one), define your recipients, the email address you're sending from, and a number of other options:

Bananatag vs Mailchimp for internal comms

This workflow makes sense for external marketing emails, but does feel disconnected from the inbox that most communicators feel at home in.

Sending an internal email through Bananatag

With Bananatag's plugin for Outlook, Gmail or Lotus Notes, communicators only need to click a 'track email' button in their mail client to start collecting data for that specific email:

Tracking an internal email with Bananatag in Outlook

This is truly one click email measurement that can be turned on and off in an intuitive and instant way. Not much else to say here, but if you'd like a demo, check out Bananatag here.

Using internal email templates

The template builder is one of the areas that Mailchimp does better than anyone else. Their interface is robust and, more importantly, very forgiving. Their template builder interface is a benchmark for the industry and we have to say it's a pleasure to use:

Bananatag vs Mailchimp for internal comms

Despite the built in templates being focused on external communications (think promotional salesy emails) it's easy to adapt to any messaging you want to convey into one of their pre-configured templates.

Bananatag's template builder is currently in beta and aims to offer similar functionality, but again tailored to how internal communicators use templates:

Bananatag internal email template builder

Although Bananatag will give finally give users the ability to access templates they've created right through Outlook, Mailchimp has the overall advantage at the time of this writing.

Customization of internal emails

Both template builders allow you to visually customize your email templates in any way that you can imagine. It's easy to add your organization's brand colours through templates and insert text blocks and images. (We'll have examples of templates and creative ideas for your emails later in this series, so stay tuned.)

One limitation with Mailchimp is that you can't remove the unsubscribe link in the footer of every email you're sending. This means that individual employees can unsubscribe from an internal email list. There are ways to deal with this issue, but the ability for employees to even appear to have access to remove themselves from internal email lists tends to stick out as a point of frustration from communicators we've spoken to.
 

Reporting and Internal Email Metrics

As a communicator you're probably interested in both how many employees are opening your emails, and if they're clicking the links through to your intranet or other internal resources.

How do these tools help you better understand your internal audience? We'll break these considerations down below, but the short version is that you can get very similar (and accurate) real-time data from either of these systems.

What metrics can we report on?

Bananatag and Mailchimp will report on how many recipients opened each email, as well as how many times each link was clicked. Additionally, both platforms can give you data on the location recipients were in when they opened their email as well as what type of device was used.

Both systems can also easily pass data to your Google Analytics account if you use that platform to measure your intranet.

The primary difference in this area has to do with tracking individuals, which we'll cover shortly.

Where do I see my email engagement data?

Although you don't need to log into Bananatag to send emails, your metrics are viewable from any web browser in your Bananatag account, and this is similar to how you would view reports in Mailchimp. You can log into either system at any time and look at all of your data and run individual or aggregate email reports.

Below is a sample dashboard from Bananatag:

Bananatag internal email metrics

And here is a sample internal email campaign report from Mailchimp:

Bananatag vs Mailchimp for internal comms

How close is it to real time?

Both platforms we're comparing today will show you engagement as it happens for any emails you've tracked and sent in your web browser. This means you can see data coming into your Bananatag or Mailchimp account as employees engage with emails. In practice both can be considered real-time measurement tools.

Viewing engagement with emails over time

Both Mailchimp and Bananatag offer a report which shows you how opens and clicks occurred on a time graph, allowing you to see how quickly emails were opened and messaging was received across your organization.

In Mailchimp this is called the 24-hour Performance Graph, and in Bananatag it's referred to as the Activity Distribution Graph.
 

Security and where your data goes

For many industries, security of internal data is both critical to stakeholders and a huge factor when deciding on what kind of tool can be used.

When it comes to measuring emails, it's sometimes a tradeoff between how much data can be collected to improve decision making, and how much the legal department will allow.

Let's look at how each system treats individual users' data, and where this data is stored.

Tracking individual employee behavior

With Mailchimp, the data is there, right in their system. You can see if individual email addresses in your organisation opened emails. You can also see an estimation of how engaged each recipient is based on the percentage of internal emails they open. This data is definitely useful for external emails and marketing, but many communicators we've spoken to have told us their organizations don't allow them to monitor individuals in this way.

Bananatag doesn't track individuals. You'll always get aggregate data on your list as a whole, never on the individual employees that opened or clicked lists in emails.

How secure is your data (and where is it stored)?

Both Mailchimp and Bananatag have extensive documentation regarding how your data is stored and we won't bore you with the details here. We do encourage you to compare the documentation or have a member of your IT team do the same.

Both services use enterprise level security measures to protect your data.

One difference to consider here is that Mailchimp does currently store more of your data since they store the internal email addresses that emails are sent to with their service and the body of your messages. Remember that Bananatag does not access your recipient list, and if desired, can be used without storing the body of your messages as well.
 

Pricing and implementation

Both Bananatag and Mailchimp offer plans based on the size of your email lists.

In the case of Bananatag, this is calculated based on the number of unique recipients on your internal distribution lists. With Mailchimp, this is calculated based on the number of contacts you maintain in their system.

We've also considered how long implementing measurement with each platform takes.

How much do they cost?

There are a few options for plan types when looking at Mailchimp, but the most relevant plan they offer for a communications applications would be their 'Growing Business' plan. You can actually use this plan for free if you have fewer than 2,000 recipients.

Bananatag is designed specifically for internal communicators, so the plans offered are more straightforward -- plans start at $250 per month. For full pricing details for Bananatag, please contact our team.

How quickly can we adopt a system?

As we mentioned earlier, Mailchimp requires you to maintain a list of all of the employees you want to send to in their system. This structure does mean that implementing it also requires you to create a process for keeping your list in their system up to date. Other than this, creating an account and getting started is straightforward.

Bananatag can be set up in literally five minutes. The only thing a user needs to do is install the free Bananatag plugin for Outlook (or another mail client) and register the email addresses you want to send from in our system. You can then send to any list you have set up in your mail client.