GENBlog

This is the Blog of the technical experts at GEN and its companies

Outlook Spam/Junk Filter Issues

 Microsoft Outlook

We recently became aware that some customers using Microsoft Outlook of various versions were experiencing missing email. Our technical team investigated and found the missing email's in the users Spam/Junk folders. The issue appeared to be localised to the last two months so we looked deeper and discovered that Microsoft had released an update to the Spam filter in Outlook in June. There are two issues that impact this, firstly some users due to their configuration cannot see the 'spam' and 'Junk email' folders without going into folder view, and secondly, even if you set the spam filter to 'None' it still in some circumstances takes action when it shouldn't. 

One of the most significant issues that our customers have experienced with this 'change' is that email's between users on the same domain are being flagged as spam, when of course they are not. Surely, if fred@ sends to tom@ then the spam filter should leave well alone? 

Further testing revealed that the updated spam filter was even more sensitive to spammy signatures (HTML Signatures that use external images etc) and that by removing the signature the email was passed. 

The Spam filter can be disabled within the outlook settings but we have found, and users have reported that Outlook continues to filter regardless. Therefore we have found registry settings that can be applied to all three versions of outlook to permanently disable the spam filter, and this is our recommended option if you don't have the time or enthusiasm to educate the user base on how to manage Outlook's crazy spam filter. A link to these registry files can be found in our FAQ here. If you have office 365 then you can also find an article on how to disable its spam filter at the same link. 

Continue reading
  1777 Hits
  0 Comments
1777 Hits

AntiSpam and AntiVirus Defence

GEN's development team is pleased to announce the general availability of our new Anti-Spam service for corporate email gateways and domains. Maxim extends our standard Anti-Spam and Anti-Virus gateways by providing process intensive enhanced spam and virus detection which greatly reduces the volume to Spam to virtually zero.

We asked 47 professional users of the GENZone platform to participate in the trial of this new service by subscribing an IMAP folder called 'Maxim' and moving any spam received into that folder. Using this feedback we were able to fine tune the system to maximise its effectiveness and gather valuable performance metrics. 

The fight against Spam

The detection of spam is a continuos battle between the spammers and companies like us who are dedicated to eliminating it. As we evolve so do the spammers and we have to invest ever more complex and expensive technologies to counter them. Some of the technologies are outlined below: 

Standards: The internet is governed by a set of standards known as RFC's and the email delivery protocol is specified by RFC822 and RFC5321. The standards exist so that email can be interoperable between all platforms and servers, but spammers using email bots don't care about being compliant. By enforcing the standards and rejecting violations we can eliminate a percentage of spam, and of course legitimate email from organisations who can't configure their email system correctly. 

The blacklist: A number of worthy organisations like Spamhaus, SpamCop, etc are dedicated to maintaining lists of domains, hosts and subnets which are used to originate spam. Using these blacklists is an expensive but effective tool to eliminate a good percentage of spam at the first gate. Blacklists however are not realtime, and there is always a delay between a spammer launching a mass mailing and the blacklists listing it. 

Authentication: Several technologies exist to verify sender domains and hosts such as SPF & DKIM and these can serve (where used by the receiving server) to block spoofed spam which constitutes the vast majority of scams. For example, the HMRC who are under constant attack from scammers specify in their SPF records two hosts that are allowed to send email for @hmrc.gov.uk and of course the spammers cannot originate email from those addresses so SPF wins the day and any email coming from, say This email address is being protected from spambots. You need JavaScript enabled to view it. that doesn't come from the two hosts listed in the SPF record are canned. This however all falls down then either the receiving server doesn't check, the sending organisation doesn't use it, or the sending organisation has been compromised.

DNS: The domain name system is that which coverts gen.net.uk to 212.140.242.10 and back again, and when you send email to someone @gen.net.uk DNS gives up the address of the mail server that is designated to receive that email, in this case farpoint.gen.net.uk. The RFC1124/1124 which form part of Internet Standard 1 specify clearly that every host on the internet should have forward and reverse DNS, that is gen.net.uk to 212.140.242.10 and 212.140.242.10 to gen.net.uk. So, when a host spamer.com connects from 212.140.242.50 to our mail server, we (a) check that 212.140.242.50 corresponds to spammer.com, that spammer.com has a valid MX record and that the host listed in the MX record actually exists on the internet. This is particularly hard for a spammer to forge and therefore this check eliminates a percentage of spam as well as a percentage of legitimate email from companies who don't know how to setup DNS correctly. 

Content Filtering: By far the most effective tool at eliminating spam which passes all the above tests is pattern matching. This involves looking and detecting elements in the body of an email and assigning a score to each detection. An example would be a HTML only email which scores 3 points, external links to pictures which scores 0.2 points each and so on. The more spammy the email the most points it will accumulate and once a threshold is reached the message is flagged as spam. Content filtering can make use of content lists which are maintained by third parties and provide known phrases and content to score. 

Bayesian Probability Filtering: A gross simplification of this would be that email which is known to be spam can be 'learned' and that data used to identify 'similar' spam. The area of mathematics is complex and the techniques even more so, but the result is the same in that spam that looks like spam based on learned data can be flagged as such, usually by giving it a score, such as +10

When you combine all these techniques together you wind up with a spam detection system that, in our tests has an effective performance of 99.67% which is exceptional in the market. Spammy email is passed through with subject modifications for your gateway to filter (or not) as you require, or for individual users to filter using IMAP or similar rules. Full Diagnostic information is provided in email headers to permit more complex filtering based on spam score or infection type should this be required by your IT Team. 

Customers with GENX and GENZone and those with gateways and dedicated services can have this added to their email feed for a nominal charge. 

For more information or to request a demo please contact us today.  

Continue reading
  2077 Hits
  0 Comments
2077 Hits