Providing Email client autoconfiguration information from moens.ch
The original blog entry is gone. I posted a archive snapshot heres here from web.archive.org
Original URLS:
- http://moens.ch/2012/05/31/providing-email-client-autoconfiguration-information/
- http://moens.ch/njpvg
Providing Email client autoconfiguration information
If you’ve ever configured a gmail or hotmail account in a mail client such as Thunderbird, Outlook or Apple Mail you will have noticed that all it asks of you is your username and password and automatically sets up the mail server hostnames, port numbers and connection settings. However, if you set up an email account hosted on your own mail server run on non-proprietary your mail client – at best – tries to make an educated guess as to what your SMTP and IMAP servers are. If you follow certain standards (e.g. calling your SMTP server smtp.maildomain.com) this might work out, however, if you do not (and got reasons to do so), you find yourself constantly having to manually enter information.
Auto configuration is most handy though if third parties are using your mail server too. It saves you the hassle of bringing up FAQ pages on your website explaining your users how to set up their account. With auto configuration they just enter their username and password – and everything will (should) work.
I found documentation on this for all things except Thunderbird is scarce on the internet and you only find individual bits and pieces. I took an afternoon to set up my mail server to support auto configuration for most well known mail clients.
Before you can start
Before you begin you have to gather the following pieces of information:
- Incoming mail server
- Hostname
- Protocol (IMAP or POP3)
- Whether SSL is available or not
- Port number (IMAP: 143, IMAPs: 993, POP3: 110, POP3s: 995)
- Username format (full email address or just the local part)
- Password mechanism (plain or encrypted)
- Outgoing mail server
- Hostname
- Port number (most likely 587. If your SMTP server still only listens on port 25 many people will have troubles connecting as ISPs around the world started blocking this port. Enable the submission port 587, which on postfix for example is up by default)
- Whether SSL or STARTTLS should be used
- Username format
- Password mechanism
In essence
All mail clients – in essence – perform auto configuration or auto discovery in a similar way. Mostly they request an XML file from a specific location which contains the information they need. It is up to you to put that XML file at exactly the location they expect and follow the format they expect it to be in. The only client which doesn’t seem to do that is Mail on iOS (iPhone, iPod and iPad). No worry though, I found a solution for that too.
Mozilla Thunderbird
The process for email server admins to provide auto configuration for Thunderbird is actually very well documented here. I summarize:
Thunderbird assumes that everything after the @-sign of the username you entered is your email domain. If the user enters fred@example.com, the email domain will be example.com. It then looks for a configuration file at http://autoconfig.example.com/mail/config-v1.1.xml to find the settings for this domain. You can find the authoritative description of this XML file here.
The format allows you to specify more than on server of each type (incoming and outcoming). For example my IMAP server is available on port 143 using STARTTLS and on 993 using IMAPs. The live example for my domain mylansite.org can be found here: http://autoconfig.mylansite.org/mail/config-v1.1.xml.
In my case, the username is the full email address. Hence you will find this in my configuration file:
<username>%EMAILADDRESS%</username>
If on your server only the local part of the email address (the string before the @-sign) is used for authentication, use the token %EMAILLOCALPART%.
Now all you have to do is set up your config file, create a vhost on your webserver for autoconfig.yourdomain.com, place the config-v1.1.xml file in the /mail subfolder and bob’s your uncle. If you are hosting multiple email domains, the Thunderbird Autoconfiguration documentation contains a brilliant example for how to set up an Apache rewrite rule to automatically match on all host names called autoconfig.*.
Microsoft Outlook
This is where things started to get tricky as documentation was scarce. Outlook works brilliantly when connecting to an Exchange server, tt figures out everything it needs to know, but how to make use of this mechanism you wonder? Well, wonder no longer. It isn’t as simple as the Thunderbird implementation, but the concept is very similar.
Outlook performs an HTTP POST request to https://autodiscover.emaildomain.com/autodiscover/autodiscover.xml with the following content:
<?xml version="1.0" encoding="utf-8"?>
<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/requestschema/2006">
<Request>
<AcceptableResponseSchema>http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a</AcceptableResponseSchema>
<EMailAddress>fred@emaildomain.com</EMailAddress>
</Request>
</Autodiscover>
There is some decent documentation in this MS TechNet article on how to format the response. To very quickly cut to the chase, you can find my example here with the minimum required to set up a simple email account. You will notice that the <LoginName> is blank. I’ll get to that in a minute: https://ns2.samhostuk.net/autodiscover/autodiscover.xml
Remember that this POST request will go via HTTPs and not HTTP. You don’t want to have different SSL certificates for each email domain you are hosting, each matching autodiscover.emaildomain.com (also considering that Apache cannot have name based virtual hosts with different SSL certs). There is a very elegant way to solve this:
Outlook first does a DNS lookup for a SRV record at _autodiscover._tcp.emaildomain.com to find the address of the autodiscover server. This record should have the following format:
0 0 443 ssl.mailprovider.com
This means that instead of looking for the XML file at https://autodiscover.emaildomain.com/autodiscover/autodiscover.xml, Outlook will now look at https://ssl.mailprovider.com/autodiscover/autodiscover.xml. If you are running bind, the entry in your zone file will look somewhat like this:
_autodiscover._tcp SRV 0 0 443 ns2.samhostuk.net.
If you want to test your record before going through the pain of firing up your Outlook client, run something like:
#> dig +short -t SRV _autodiscover._tcp.mydomain.com
This also means you do not have to set up an autodiscover subdomain for each email domain you host. You still however have to add this SRV record to each email domain’s DNS zone.
Getting the user’s email address in the response XML
I have found no documentation on valid tokens you can use in the response XML such as the %EMAILADDRESS% token honored by Thunderbird. This means it’s up to you to populate the <LoginName> element on the server side. Fortunately, the client just posted it to you, so all you have to do is extract it from the POST data and print it into the output XML. PHP is my language of choice, but you can implement a similar solution in your preferred server side scripting language.
First, you need to tell Apache that .xml files should be sent through the PHP interpreter. Locate the <VirtualHost> directive for your SSL host (on my Ubuntu install it’s in /etc/apache/sites-available/default-ssl) and add the following line:
AddType application/x-httpd-php .php .php3 .php4 .php5 .xml
Save your config and reload Apache.
Then, edit your autodiscover.xml file and add some magic. Here is what mine looks like (I am a believer that for this simple search, a preg_match is quicker than a full XML parse):
<?php
$raw = file_get_contents('php://input');
$matches = array();
preg_match('/<EMailAddress>(.*)<\/EMailAddress>/', $raw, $matches);
header('Content-Type: application/xml');
?>
<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
<Response xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a">
<User>
<DisplayName>myLANsite</DisplayName>
</User>
<Account>
<AccountType>email</AccountType>
<Action>settings</Action>
<Protocol>
<Type>IMAP</Type>
<Server>ns2.samhostuk.net</Server>
<Port>993</Port>
<DomainRequired>off</DomainRequired>
<SPA>off</SPA>
<SSL>on</SSL>
<AuthRequired>on</AuthRequired>
<LoginName><?php echo $matches[1]; ?></LoginName>
</Protocol>
<Protocol>
<Type>SMTP</Type>
<Server>ns2.samhostuk.net</Server>
<Port>465</Port>
<DomainRequired>off</DomainRequired>
<SPA>off</SPA>
<SSL>on</SSL>
<AuthRequired>on</AuthRequired>
<LoginName><?php echo $matches[1]; ?></LoginName>
</Protocol>
</Account>
</Response>
</Autodiscover>
To test your script, save a copy of the Autodiscover XML Request above into a local file called req.xml and then execute the below command to execute the request and validate the response:
cat req.xml | lwp-request -m POST https://autodiscover.mymaildomain.com/autodiscover/autodiscover.xml
If your _autodiscover._tcp SRV record and the autodiscover.xml file are correctly in place, Outlook users will be able to set up their email accounts absolutely hassle free.
Apple Mail on iOS (iPhone, iPod, iPad)
The Mail App on iOS works differently to Outlook at Thunderbird. It does not perform any lookups to discover settings. It’s got the built-in presets for providers such as Hotmail and Gmail, but makes not attempt to look up settings for any other mail domains. However, there is a way – through profiles. If you’ve got a Mac you’re in luck. If you have not, ask a friend with a Mac to lend it to you for five minutes 
First, get your hands on the free iPhone Configuration Utility from Apple. Install it and run. Select Configuration Profiles on the left, click New and give it a name.
Select Email from the list of categories and click Configure. Here an example for the same config we’ve been using all throughout this article for my email domain. The important thing to notice is that I enter MAILADDR as value for the email address and user name. This is because I use a similar replace mechanism as I did on the Outlook autodiscover.xml file above. If you wish a simpler solution, you can just leave it blank and the user will be prompted for their email address.
Hit the Export button, select None as security (unless you’ve got a fancy signed cert for signing) and finish the export process. You will end up with a file called something.mobileconfig. Upload it to your web server at any desired location. Now all you have to do is direct users to the location of your .mobileconfig file and the iP(hone|od|ad) will do its magic!
If you want some real magic, and if you used the MAILADDR “token” when creating the Configuration Profile, take a look at the iphone.php script below. It captures the email parameter either from GET or POST, opens up a local .mobileconfig file, replaces the MAILADDR token and outputs the file. The header() lines are vital – the download filename must have the extension .mobileconfig or the iDevice will not automatically detect it as a profile.
<?php if (isset($_REQUEST['email'])): ?>
<?php
header('Content-Type: text/plain');
header("Content-Disposition: attachment; filename=\"mylansite.mobileconfig\"");
$conf = file_get_contents('iphone.mobileconfig');
$conf = str_replace('MAILADDR', $_REQUEST['email'], $conf);
print $conf;
?>
<?php else: ?>
<html>
<head>
<title>iPhone Email Auto Configuration</title>
<meta name="viewport" content="width=device-width; initial-scale=1; user-scalable=no" />
</head>
<body style="font: 100% Verdana;">
<form method="post" action="iphone.php">
<p style="text-align: center">
Enter your email address to obtain an automatic
configuration profile for your %ISPNAME% Email Account.<br/>
<br/><br/>
<input type="text" name="email" style="height: 30px; width: 250px;"/><br/>
<br/>
<input type="submit" value="Submit" style="width: 100px; height: 30px;"/>
</p>
</form>
</body>
</html>
<?php endif ?>
I am using a control panel back end for my mail host which is automatically sending a welcome email to the user and also provides a web mail interface. I extended the code which sends the welcome email to include a link to http://myhost.com/iphone.php?email=youraddress@myhost.com. Actually, I shorten it with bit.ly first using David Walsh’s neat little PHP example on how to access the bit.ly API, as the user will most likely have to manually enter the URL into their phone. So the user enters bit.ly/LODdtw into their iPhone browser, the profile gets downloaded, and all they have to provide is a Display Name, email address and password and the rest is automatically configured.