Why a Lot of Spam Isn't Actually Spam

This is one in an occasional series of posts on the trials of bulk emailing.

We're all told how spam is a big problem, and getting bigger. Lots of people say this. Strangely, not so many people tell us how junk mail coming through our letter box is a problem also. This is because the 'carrier' of junk mail gets paid to deliver the unsolicited junk to your door, while online carriers don't. So when someone tells 'us' that 'we' have a problem, it usually means that 'they' have a problem. But I digress.....

If we set aside malicious emails that are generated to spread viruses, then spam is really no different from all the other unwanted advertising that is blasted into our faces day in day out.

However, if you actually start sending out bulk email to legitimate subscribers, then you will find that what you call spam and what the mail carriers call spam are two completely different beasts.

There is a rough threshold for the number of mails you send out, above which you start to get noticed by the main mail carriers – Google, Hotmail, AOL, Yahoo etc, and when I was running email I think the threshold was around one million mails per mailshot. Below that things kind of ran ok, aside from having to manage the normal bounces and returns (more of which in a separate post). Above that, and a new set of problems creep out of the woodwork.

At this point, the main mail carriers would each be getting a few hundred thousand mails through their mailservers each time you ran a mailshot.

The main mail carriers have two metrics that they monitor – number of and percentage bounces, and percentage of reported spams.

Email addresses that were given to you legitimately will bounce for a number of reasons – there was a typo when the user entered the address, the mailbox has become full or has been discontinued, for example.

When a mail carrier sees bounced emails coming from a single IP address going over a certain limit, it will most likely block your IP address and bounce ALL your mail, so the remainder of your mailshot fails.

It will consider all these emails as being spam, which they are not – none of them.

Your IP address may remain blocked, in which case you have to try and find a contact within the carrier to talk to, and convince them, that you are a legitimate sender of email, and they will then suggest to you how to change the content of your emails to make them pass through their system more successfully. When this happens, it feels just like you can imagine if a member of the Post Office staff advised you how to change the layout of a printed flyer. It's done to suit their system, because they are the carrier. Some carriers have a whitelisting service, some don't.

I have known AOL to simply take in all the emails and 'drop' them. They don't get passed to the end user, and they don't get bounced back. Unless you embed some means of tracking when the mails are opened (usually a single pixel image embedded in the mail – which ironically also counts against you on spam filters), you cannot tell if any of them actually made it to your target.

The next problem relates to the mechanism that the carriers employ to allow their users to report mail as spam. If a user drags a mail to the 'spam bin', that counts as spam. No matter that they gave you their email address to subscribe, they often feel it's just easier to mark it as spam than to unsubscribe themselves.

When these reach a certain level (usually 10%) of your mailshot, then you get problems. You can be required to set up a special 'abuse' email address on your mailserver, and then you get sent all these mails by the carrier and you are obliged to unsubscribe them yourself. The mails get sent to you by the carrier, but are usually stripped of the original email address, due to reasons of privacy. As a result, you need to mark every outgoing email with an identifier that allows you to retrieve the original recipient – in other words to find the email address of the person who marked it as spam, which the carrier won't tell you but which you obviously need in order to unsubscribe them. If that sounds stupid then, well yeah.

Again, the carrier will call this spam, but again it isn't.

This can give you a few downstream problems. If you have been sending your mailshots from your office mailserver, then if it gets blocked you will lose all mail communication through that carrier to your actual paying customers until you get the carrier to unblock your IP. Even if your mailserver sends from a different domain, you can still find your back-office IP address blocked also, and if you have paying customers from that mail carrier domain, then this obviously poses bigger problems.

So, aside from some of the problems that occur when your in-house mailing system starts to grow, two things pop out of this.

The first is that what is counted as spam within the industry is often not, and is inflated by those with vested interest.

The second, and more important, is that as a business you have no real control over email delivery, so be careful when you promise your customers a service that relies on it.

Being Realistic About Website Numbers

This is short and simple, but I think sometimes overlooked when business forecasts are put together.

You have a subscription model website, where people sign up and pay you money on an ongoing basis – usually it's monthly.

You're delivering a service, maybe, or an online tool or whatever, that you feel people will pay for.

You make an estimate of the number of new customers you might sign up on a weekly basis, and then you create a kind of straight line graph that shows ever increasing revenue over time.

However, if you haven't taken customer retention into account your graph will be completely meaningless, because retention is as important a figure as new sales.

Let's say that your site is signing up 2,000 paying customers per week, and to keep the numbers easy, let's assume it does that from day one, and just keeps doing that every week for the first year.

By the end of the year, you will have signed up 2,000 x 52 = 104,000 customers. That's not bad for a year, if you're getting £10 per month off them, for example.

However, look what happens if your customers stay with you for six months on average, which is probably not a bad retention rate for a subscription model.

When the six month point has been reached, your cancellation rate will now be the same as your signup rate, at 2,000 per week.

Your customer base will now flatten out at 2,000 x 26 = 52,000 customers, and that is where it is likely to stay for some considerable time, with your marketing costs that were originally growing the business at a great rate now going towards stemming the flow of cancelling customers and leaving you with zero growth.

Oh, and if your conversion rate between site visitors and customers is 1%, which is a not untypical value, and the cost per thousand clicks is £250, then to get 2,000 customers per week you need 200,000 people to hit your site, which is 200 x £250 = £50,000 per week, or £2.6m per year just to keep your customer base flat at 52,000 customers.

That's not to say you can't run a business on those figures, but retention is often given little attention in business plans but yet can have a major impact on growth.

Choosing a Development Platform

This week I've started looking at Ruby on Rails, which is a development environment that I've had no previous exposure to, and it has coincided with a chat I had with Blaine Cook, the lead developer in Twitter, after his talk at Refresh Belfast.

I was asking Blaine about application scaling, as I know that Twitter has had problems in that area using Ruby on Rails and MySQL, and I was curious about his experience in using that platform because I am curious about the decision-making process in choosing a development platform generally, especially given the choice nowadays.

What was interesting for me was that I found my thoughts drifting from the technical to the non-technical as we spoke, and in the context of the rest of the evening, I came away with a slightly different perspective on the subject.

Previously, I would have considered business factors first and technical factors second in choosing a platform. The reason is that for most online businesses, pretty much any platform will actually deliver the goods technically, but the kind of business and its projected direction will often make some platforms more suitable than others.

Now, I think I would throw cultural factors into that mix.

When you Google around for opinions on different application languages, it seems to me that actually most of it boils down to two things – personal preference and fashion.

For example, I have read a lot of negative stuff about Coldfusion, but in Tescodiets.com we had a profitable multi-million turnover driven by Coldfusion 5, which was obsolete some years ago, but to which we were still predominately adding CF5 code.

It takes in user data, does stuff with it and dumps stuff back out on the screen. You can write data-driven applications very quickly – it was agile before Agile Development was born. You can create revenue-driving apps for low cost – what more would a business want?

So how come it can be difficult to get developers to be enthusiastic about something that does its job well?

The answer, I think, has a lot to do with fashion. It is a simple fact that development platforms, like most aspects of IT, are driven by fashion, and when young developers are extolling the virtues of one language over another, and filling their sentences with a lot of techie phrases, all they are often saying is that 'it's cool'.

And the bit I was left wondering about is that maybe if you want to do something new and exciting online, then all you need to consider in choosing a development platform is whether it is considered cool enough to attract the kinds of people you want to work with.

The eCommTech Blog

Welcome to the eCommTech blog.

I'm starting this blog as I make a transition from Tescodiets (www.tescodiets.com, formerly Ediets Europe), where I was Head of IT, to my new state, which is currently as a freelance Tech Consultant.

My expertise over the past nine years, if I've had any, was to have built the platform upon which Ediets Europe, and then Tescodiets, have delivered their business.

Ediets Europe started life in 2000 as a joint venture with Ediets.com in the US, and then Tesco took over the UK/IRL side of the business in 2004.

I was involved with Ediets Europe from the 'empty office' stage, setting up servers, plugging in cables and writing application code, and managed the technical side of the business all the way through to its enterprise level operation and the subsequent Tesco acquisition.

During that time, more so in the pre-Tesco days, I was closely involved with the business direction and strategy, and it is in that space that encompasses both business and tech that interests me most, especially in the startup phase which is by far the most interesting part of any business.....

I intend this blog to be a reflection of what I have learned over that time, mixed with whatever I am currently focused on, but above all for it to be of practical interest to anyone following along in the startup or ongoing development phases of online business.