ColdFusion Muse

Solving an Email Send Problem: a Simple, Innovative Approach

Mark Kruger October 22, 2008 6:40 PM Coldfusion Tips and Techniques Comments (7)

Getting email to go through for your customer or newsletter can be tricky these days. An email has to work pretty hard to make it through all the barriers in its way. The problem is compounded when you have an automated email system like a newsletter or marketing apparatus. Even if you follow all the rules you may still achieve a marginal success rate. One of our customers (Tom Long of digbro.com) has a unique system for marketing that is clearly not spam. It involves users sending individual emails out to customers with whom they already have a relationship. It's not bulk email and there are clear opt out instructions. The email goes out in a small trickle of a few hundred an hour. The system tracks the lead internally and an elaborate reporting system allows the user to figure out success rates, run referral programs and keep track of the customer information.

In spite of the benign nature of this system it has still been difficult getting emails out to some domains - particularly large email domains like Yahoo and Hotmail. We have done some innovative things like creating a "scoring" system that utilizes calls to spam assassin to try and "rate" the email before it is sent. The user submits his email text, the system dummies up an email, pipes it to Spam Assassin, retrieves it back and extracts the score - all before the email is sent. Pretty nifty - but not a panacea. In spite of our best efforts the success rate to known good email addresses on larger email domains improved only marginally. This customer is one of those visionary men that simply can't help thinking outside the box. Check out his solution.

Tom noted that naturally, emails sent from Outlook or Thunderbird tended to get through pretty well because the presence of "office like" or "client like" headers and other personal stuff tips the filters in favor of the email being from a person and not a system. Of course you can spoof headers but, besides being unethical, it is pretty easy to spot spoofed headers. The minute a filter flags your headers as spoofed it raises the spam score. So the trick is to get these emails to be created automatically, logged by the system and still get sent from the user's actual email client.

What Tom suggested was to allow the user to enter message information and user information as before, log the messages in our reporting system as before, but in the final step create a mailto: link that pops up in outlook with the requisite message and subject line (and tracking link). This seemed deceptively simple, but it worked really well. You probably already know that mailto links can contain subject and body attributes that end up in the subject and body of the message. Like so:

<a href="mailto:joe@thecrabshackaroo.com?subject=Craborama&body=Hello World">email joe</a>
The above link looks like this: email joe. If you click on it it should open up your client email with a subject of "Craborama" and a body of "hello world". So all Tom's users had to do was generate a page of links and tediously click on them one at a time and send. Hmmmm.... we needed a way to make it slightly more user friendly. Enter the "iframe".

<div id="hiddenDiv">
<Cfloop query="emailstosend">
    <iframe
        style="height:0px; width:0px;"
        src ="mailto:#email#?subject=#subject#&body=#urlencodedformat(body)#" >

    </iframe>
</CFLOOP>
        </div>

This rather handy piece of code is hidden on the page. When the page loads it pops up a "new message" window for the email client containing the message that is to be sent. Multiple windows pop up one on top of each other and the client can send each of them quickly using control+s or clicking "send" in the same spot on the screen. This is about as streamlined as we could make it while still utilizing the users email client.

The downside

Of course this doesn't solve Tom's issues completely. He has traded a fully automated system for one with more human intervention - which is exactly why it has a shot at working with anti-spam systems. Still that means he is dependent on the SMTP configuration of hundreds of email clients that are out of his control. He will have users with dozens of different clients and versions of clients. His users with web based email are out of luck and he cannot predict what restrictions will be placed on future browser behavior or email client behavior. Finally the "body" parameter can only carry plain text messages (no HTML). So in the end he has restricted his users to a fairly narrow set of programs and parameters. Still, for what's it's worth, it is not really much more difficult than the fully automated system, and recipients downstream are getting a bona fide personal email. I'm anxious to see how it improves his send rate.

Meanwhile, some spam jockey who is also an avid muse reader will no doubt set me straight on some of the alternatives. I look forward to those comments as they add to my burgeoning arsenal of tools - so post away (just be nice:).

  • Share:

Related Blog Entries

7 Comments

  • duncan's Gravatar
    Posted By
    duncan | 10/23/08 10:32 AM
    What about using an external email campaign site to do some of the donkey work? Chances are they've come across all these problems before. Even if just calling their service API to send the emails or whatever. e.g. check Campaign Monitor:
    http://www.campaignmonitor.com/api/samples/#coldfu...
  • duncan's Gravatar
    Posted By
    duncan | 10/23/08 10:34 AM
    What about using an external email campaign site to do some of the donkey work? Chances are they've come across all these problems before. Even if just calling their service API to send the emails or whatever. e.g. check Campaign Monitor:
    http://www.campaignmonitor.com/api/samples/#coldfu...
  • duncan's Gravatar
    Posted By
    duncan | 10/23/08 10:35 AM
    Weird, first time I tried to submit, I got the following error:
    String index out of range: 0

    The error occurred in C:\websites\cfwt_blogs\mkruger\tags\blocker.cfm: line 96
    Called from C:\websites\cfwt_blogs\mkruger\tags\blocker.cfm: line 116
    Called from C:\websites\cfwt_blogs\mkruger\Application.cfm: line 30

    94 :    if (NOT StructIsEmpty(COOKIE)) {
    95 :       for (testvar in COOKIE) {
    96 :          if (Application.injChecker.reset(lcase(COOKIE[testvar])).find()) {
    97 :             hackattempt="yes";
    98 :          }
  • mkruger's Gravatar
    Posted By
    mkruger | 10/23/08 10:54 AM
    I took care of that error... the cferror tag was being called "after" that instead of "before" that.

    Regarding your external service provider idea. There are a few things wrong with it. First, it cuts into revenue (both mine and Toms :), secondly their success rate is better than mine (probably) but only marginally. Even they cannot be as successful as personal emails sent from individuals.

    I would add that using a service provider means relying on them to be up, stay in business, support their system etc.
  • Ta'wanna's Gravatar
    Posted By
    Ta'wanna | 10/23/08 3:24 PM
    Could it not be done within Microsoft Word/Outlook and using a mail merge with a connection to the database? Then they would still be personalized, have the correct content, and should look as if they are coming from a real person verses a spammer. It's not to different than what you already have, but it would prevent having to go through multiple different email windows...
  • mkruger's Gravatar
    Posted By
    mkruger | 10/23/08 3:32 PM
    @Tawanna,

    This is an interesting idea - but it would not fit the application. Prior to the send operation these users do not exist in the database. So I would have to "prepare" the batch and then run it. I would also need to expose my MS SQL server, or provide a downloadable spreadsheet or access DB to do this. Remember, this is a public facing application.

    Finally, this application is not designed to send out 100 or 1000 emails - only a max of 10, its is a trickle by design. So a mail merge, being fairly complex to set up in it's own right, would be overkill.


    Still... you are definitely thinking outside the box and I can appreciate that :)
    -Mark
  • jc's Gravatar
    Posted By
    jc | 10/30/08 12:14 PM
    Users should be aware that while that solution probably works for [some versions of] outlook, it will not work for lots of other email clients. Most of them will pick up the subject part, fewer will pick up the body part, and of those that pick up either or both, some part of them will leave it urlencoded.