Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

Secure Password Authentication: NTLM over SMTP

By gblues in Internet
Sun Apr 28, 2002 at 09:23:58 AM EST
Tags: Help! (Ask Kuro5hin) (all tags)
Help! (Ask Kuro5hin)

Secure Password Authentication (SPA) is a feature provided by Microsoft's e-mail servers (and subsequently implemented in its e-mail clients) as a secure means of authenticating e-mail traffic (POP3, SMTP, and IMAP). This document describes the Microsoft-proprietary protocol that operates via the SMTP AUTH interface defined by RFC 2554.

I am still in the process of reverse-engineering the protocol--however most of it is completed. The information I have, and the information I need, is described below.


Some of you may be wondering, "Why bother?"

The simple answer is that I am stuck with MSN as an ISP. While I can handle using Outlook Express, my mother dislikes it and would prefer to use a different e-mail client. However, MSN's servers only accept authentication via SPA. Unless you want to cough up money for an e-mail client, that means you use either Microsoft Outlook or Microsoft Outlook Express.

So, seeking alternatives, I did some hunting with Google and discovered this security advisory which states that SPA is really a warmed over version of NTLM. Armed with this information, I found a reference for NTLM over HTTP by Ronald Tschalär. This document, combined with the ENCRYPTION.txt from Samba, gave me most of the information needed to decipher the SPA handshake.

The SPA handshake is broken into three sections: the NTLM authentication request, the NTLM challenge, and the NTLM response.

The NTLM Authentication Request

Below is a C-style structure describing the NTLM authentication request. Portions that are currently unknown are marked with a '?':

// char = 8-bit datatype
// long = 32-bit datatype

struct ntlm_request {
char protocol[8]; // 'N' 'T' 'L' 'M' 'S' 'S' 'P' '\0'
long msgType;     // 0x01000000
long flags;       // 0x02820000
char seed[4];     // 0x50bc0f00 32-bit seed value. (?)
long other_flags; // 0x02000000(?)
long unknown1;    // 0x00000000(?)
long unknown2;    // 0xd4d59101(?)
NTLM Challenge

Once the authentication request has been sent, the SMTP server sends back an NTLM challenge. The challenge uses the following format:

struct ntlm_challenge {
char protocol[8]; // 'N' 'T' 'L' 'M' 'S' 'S' 'P' '\0'
long msgType;          // 0x02000000

short svr_host_maxlen; // 0x0c00
short svr_host_len;    // 0x0c00
short svr_host_offset; // 0x2000
short svr_host_pad;    // 0x0000
char challenge[16];    // NTLM challenge
char server_hostname[<svr_host_maxlen>];
NTLM Response

The client generates an NTLM hash of the SMTP password and uses this hash to encrypt the challenge. The encrypted form is then sent back as the response.

The NTLM response has the following format:

struct ntlm_response {
char protocol[8];         // 'N' 'T' 'L' 'M' 'S' 'S' 'P' '\0'
long msgType;             // 0x03000000
short ntlm_resp_maxlen;   // 0x1800
short ntlm_resp_len;      // 0x1800
short ntlm_resp_offset;   // 0x3400
short ntlm_pad;           // 0x0000
short md4_resp_maxlen;    // 0x0000
short md4_resp_len;       // 0x0000
short md4_resp_offset;    // 0x4c00
short md4_pad;            // 0x0000
short svr_host_maxlen;    // 0x0000
short svr_host_len;       // 0x0000
short svr_host_offset;    // 0x4c00
short svr_host_pad;       // 0x0000
short user_maxlen;        // For a 5-character username: 0x0500
short user_len;           // 0x0500
short user_offset;        // 0x4c00
short user_pad;           // 0x0000
short host_maxlen;        // For a 5-character hostname: 0x0500
short host_len;           // 0x0500
short host_offset;        // 0x6300
short host_pad;           // 0x0000
char ntlm_response[24];
char server_hostname[<svr_host_len>]; // not null-terminated
char smtp_username[<user_len>];       // not null-terminated
char hostname[<host_len>];            // not null-terminated
How Authentication Works

The NTLM authentication process uses the following algorithm:

  1. Client sends NTLM Authentication Request to the server.
  2. Server sends a 64-bit NTLM challenge to the client.
  3. Client uses the challenge and an NTLM password hash to create an NTLM response.
  4. The client sends the NTLM response to the server.
  5. The server creates an NTLM response (since it also has your NTLM password hash) and compares its version with the one received by the client.
  6. If the responses match, the authentication is successful.
Creating the NTLM password hash

The NTLM password hash is created using the following algorithm:

#define PASSWORD "gblues"

char pw_ntlm[14];
char hashpw1[8], hashpw2[8];
char hashpw[21];
char des1[8], des2[8];
char magic[8] = { 0x4b, 0x47, 0x53, 0x21, 0x40, 0x23, 0x24, 0x25 };

memcpy(hashpw1, magic, 8);
memcpy(hashpw2, magic, 8);
strcpy(pw_ntlm, PASSWORD);
format_passwd(pw_ntlm); // pads and performs tr/a-z/A-Z/;

read_key(pw_ntlm, des1, 0); // pw_ntlm is split into
read_key(pw_ntlm, des2, 7); // two 56-bit DES keys.

encrypt_block(des1, hashpw1, 8); // encrypt the magic values
encrypt_block(des2, hashpw2, 8); // using each DES key

mempcy(hashpw, hashpw1, 8);   // concatenate the two encrypted
memcpy(hashpw+8, hashpw2, 8); // blocks together, and pad the
memset(hashpw+16, 0, 5);      // remaining five bytes with NUL ('\0')
Creating the NTLM Response

In Phase 2 of the NTLM authentication process, the server sends a 64-bit challenge. Did you notice that the final array in the above algorithm was 21 bytes long? This is because the NTLM password hash is itself split into 3 DES keys and used to encrypt the challenge. The algorithm looks similar to the following:

char challenge[8] = { 0x1a, 0x9d, 0xb9, 0xd9, 0xc9, 0xb4, 0x3b, 0x2a };
char des1[8], des2[8], des3[8];
char ch_hash1[8], ch_hash2[8], ch_hash3[8];
char hashpw[21]; // contains results of above algorithm
char response[24];

memcpy(ch_hash1, challenge, 8); // make three copies of the challenge
memcpy(ch_hash2, challenge, 8);
memcpy(ch_hash3, challenge, 8);

read_key(hashpw, des1, 0); // split into 3 DES keys
read_key(hashpw, des2, 7);
read_key(hashpw, des3, 14);

encrypt_block(des1, ch_hash1, 8); // encryption magic!
encrypt_block(des2, ch_hash2, 8);
encrypt_block(des3, ch_hash3, 8);

memcpy(response, ch_hash1, 8); // concatenate the results together.
memcpy(response+8, ch_hash2, 8);
memcpy(response+16, ch_hash3, 8);
What's Next?

The NTLM Authentication Request format is unclear. It is clearly different from the request described in the Samba ENCRYPTION.txt file. If anyone has information they can contribute to this, please feel free to add it to the comments.

As the security advisory linked above notes, this protocol is wide open to a Man in the Middle (MitM) attack. Well, "attack" may be too strong of a word. My goal is to roll this into an SMTP proxy that will allow others trapped behind MSN's SMTP servers to use any e-mail client they desire.

All help is greatly appreciated!


Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure


Related Links
o Google
o RFC 2554.
o this security advisory
o NTLM over HTTP
o Also by gblues

Display: Sort:
Secure Password Authentication: NTLM over SMTP | 7 comments (6 topical, 1 editorial, 0 hidden)
Excellent! (3.66 / 3) (#2)
by vadim on Sun Apr 28, 2002 at 08:38:48 AM EST

We need more proxies like this!

I have a project like this too. I'm thinking of writing a program to download Yahoo Mail. I've already got a really nice system set up on my mail server that gets mail from all my accounts, scans for spam, removes HTML and sends alerts. Adding this to it would make it really neat.
<@chani> I *cannot* remember names. but I did memorize 214 digits of pi once.

pay for pop (none / 0) (#6)
by ironfroggy on Thu May 02, 2002 at 01:37:20 AM EST

Would be so much easier if they kept POP access free!
-- Question
[ Parent ]
That's not the point (none / 0) (#7)
by vadim on Thu May 02, 2002 at 12:26:11 PM EST

I never paid yet for any online services besides ADSL. I'm not planning to do that any time soon, I already set up a web server in my room. Besides, I don't really have such a great need for it, it's just a challenge! And a nice module on CPAN too, maybe.
<@chani> I *cannot* remember names. but I did memorize 214 digits of pi once.
[ Parent ]
LibESMTP (5.00 / 2) (#3)
by chbm on Sun Apr 28, 2002 at 10:17:21 AM EST

Brian Stanford is also working on NTLM auth on libESMTP. You might want to drop him a line or two :)

"While I can handle using Outlook Express, my mother dislikes it and would prefer to use a different e-mail client." -- this is priceless.

-- if you don't agree reply don't moderate --
Oh... the lengths we'll go for mom! (3.50 / 4) (#4)
by MicroBerto on Sun Apr 28, 2002 at 12:54:43 PM EST

This goes to show,
"If mama ain't happy, ain't nobody happy!"

- GAIM: MicroBerto
Bertoline - My comic strip
NTLM Authorization Proxy Server (5.00 / 3) (#5)
by c960657 on Mon Apr 29, 2002 at 08:01:55 AM EST

'NTLM Authorization Proxy Server' (APS) is a proxy software that allows you to authenticate via an MS Proxy Server using the proprietary NTLM protocol. [...] It is written in Python v1.5.2 language.

Secure Password Authentication: NTLM over SMTP | 7 comments (6 topical, 1 editorial, 0 hidden)
Display: Sort:


All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!