Categories
Dovecot Linux Mail Postfix Ubuntu

OAuth2 for dovecot and postfix?

Currently I’m using Nextcloud with snappymail as mail client. While this works nicely, I do have a “comfort problem” since I switched my Nextcloud authentication to SAML/SSO: I can no longer use the Nextcloud credentials to log into my mail account (as Nextcloud does not know about my password when using SSO).

There’s two things with my current solution:

  1. My password needs to be stored in cleartext somewhere
  2. I have to enter it again and again

So once more I started to look for other solutions. And it looks like postfix and dovecot both do support OpenID/OAuth … didn’t know that till now.

And there’s an option in the snappymail admin interface (hidden under “Additional settings” -> “SnappyMail Webmail“) named “Attempt to automagically login with OIDC when active“.

So with a bit of luck this could be my way to go …

Check current dovecot/postfix config

First I wanted to check what version of postfix/dovecot I’m running (to make sure OAuth is really supported):

linux # dpkg-query -s postfix | grep ^Version
Version: 3.8.6-1build2
linux # dpkg-query -s dovecot-core | grep ^Version
Version: 1:2.3.21+dfsg1-2ubuntu6

So I’m running postfix 3.8.6 and dovecot 2.3.21.

After that I’d check the basic authentication settings for my current dovecot/postfix config:

linux # openssl s_client -quiet -verify_quiet -connect imap.mydomain.de:143 -starttls imap
. OK Pre-login capabilities listed, post-login capabilities have more.
Step1 CAPABILITY
* CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN AUTH=LOGIN AUTH=GSSAPI
Step1 OK Pre-login capabilities listed, post-login capabilities have more.
Step2 LOGOUT
* BYE Logging out
Step2 OK Logout completed.

For postfix this:

linux # openssl s_client -quiet -verify_quiet -connect mail.mydomain.de:587 -starttls smtp
250 CHUNKING
EHLO client.mydomain.de
250-mail.mydomain.de
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-AUTH PLAIN LOGIN GSSAPI
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250-SMTPUTF8
250 CHUNKING
QUIT
221 2.0.0 Bye

So both dovecot and postfix currently support these auth methods:

  • PLAIN
  • LOGIN
  • GSSAPI

As my postfix setup is using dovecot for SASL authentication, it may be enough to only change the authentication configuration for dovecot.

Here’s how to check the used SASL backend for postfix:

linux # postconf smtpd_sasl_type
smtpd_sasl_type = dovecot

According to the dovecot documentation I’m missing the authorization mechanisms OAUTHBEARER or XOAUTH2.

Getting basic IdP (keycloak) settings

In order to get some global settings from our keycloak OpenID provider, we can use the .well-known URL and jq:

linux # curl -s -o - https://keycloak.mydomain.de/realms/MYDOMAIN/.well-known/openid-configuration | jq
<...>

Or more specific (e.g. to obtain the introspection endpoint):

linux # curl -s -o - https://keycloak.mydomain.de/realms/MYDOMAIN/.well-known/openid-configuration | jq .introspection_endpoint
"https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/token/introspect"

Update: Here’s a nice jq command to list all available endpoints:

linux # curl -s https://keycloak.mydomain.de/realms/MYDOMAIN/.well-known/openid-configuration | jq 'to_entries | map(select(.key | endswith("_endpoint"))) | .[] | "\(.key): \(.value)"'
"authorization_endpoint: https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/auth"
"token_endpoint: https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/token"
"introspection_endpoint: https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/token/introspect"
"userinfo_endpoint: https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/userinfo"
"end_session_endpoint: https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/logout"
"registration_endpoint: https://keycloak.mydomain.de/realms/MYDOMAIN/clients-registrations/openid-connect"
"revocation_endpoint: https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/revoke"
"device_authorization_endpoint: https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/auth/device"
"backchannel_authentication_endpoint: https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/ext/ciba/auth"
"pushed_authorization_request_endpoint: https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/ext/par/request"

We also need to create an OpenID client within keycloak and save the oidc_clientid and the (generated) oidc_secret.

Adding OAUTH authentication

Following the examples mentioned in the dovecot docs, I extended my existing dovecot config like this:

linux # vi /etc/dovecot/conf.d/11-auth-oath.conf
auth_mechanisms = $auth_mechanisms oauthbearer xoauth2

passdb {
  driver = oauth2
  mechanisms = xoauth2 oauthbearer
  args = /etc/dovecot/conf.d/auth-oauth2.conf.ext
}
linux # vi /etc/dovecot/conf.d/auth-oauth2.conf.ext
introspection_mode = post
introspection_url = https://oidc_clientid:oidc_secret@keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/token/introspect
username_attribute = email
pass_attrs = pass=%{oauth2:access_token}
# debug = yes
linux # systemctl restart dovecot

After doing so my dovecot announced OAUTH2 support:

linux # openssl s_client -quiet -verify_quiet -connect imap.mydomain.de:143 -starttls imap
. OK Pre-login capabilities listed, post-login capabilities have more.
Step1 CAPABILITY
* CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN AUTH=LOGIN AUTH=GSSAPI AUTH=OAUTHBEARER AUTH=XOAUTH2
Step1 OK Pre-login capabilities listed, post-login capabilities have more.
Step2 LOGOUT
* BYE Logging out
Step2 OK Logout completed.

And as my postfix config is using the dovecot authentication this now offers OAUTH2 support, too. Even without a restart:

linux # openssl s_client -quiet -verify_quiet -connect mail.mydomain.de:587 -starttls smtp
250 CHUNKING
EHLO client.mydomain.de
250-mail.mydomain.de
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-AUTH PLAIN LOGIN GSSAPI OAUTHBEARER XOAUTH2
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250-SMTPUTF8
250 CHUNKING
QUIT
221 2.0.0 Bye

Ok, but how the h… do I test this?

Testing OAUTH2 authentication

Client #1: Thunderbird

Even though I found links about thunderbird supporting OAUTH2 the option was not available while configuring a new mail account (this is thunderbird version 140.3.1esr (64-bit)).

Oddly enough it does show up when adding my Gmail account or using a office365.com account.

Enforce auth type (thunderbird)

So I tried to enforce the missing auth type for preconfigured IMAP/SMTP server. In order to do so you’ll first need to determine their configuration index:

So first you’ll have to search for “mail.smtpserver.smtp*.hostname” to get the right index number for the SMTP server settings, and “mail.server.server*.hostname” for the IMAP server.

You can then change the authMethod to “10” (=OAUTH2, see table below). If this setting is missing for your configuration you may have chosen the default auth setting. In that case you need to create that value first.

IDType
2Password, original method (insecure)
3Normal password
4Encrypted password
5Kerberos/GSSAPI
6NTLM
7TLS Certificate
8Any secure method (deprecated)
9Any method (insecure)
10OAuth2

In my case the IMAP server index was 4 and the SMTP server index 2, so here’s the settings I changed/added:

mail.server.server4.authMethod=10

mail.smtpserver.smtp2.authMethod=10

This really changes the authentication type to OAUTH2 in the configuration dialog, however it didn’t have any effect besides that.

Enable thunderbird autoconfiguration

So let’s go for the next idea: Gmail and Microsoft both use a pre-configuration provided by thunderbird, so maybe we’ll need the same thing for using OAUTH2.

Initially I was hoping for thunderbird to probe the SMTP and IMAP server for supported authentication types during setup (as we did manually at the start of this article), however I couldn’t see any related connections to the mail servers during setup. So I looks like we need to provide that information in a different way.

Autoconfiguration could be the way to go: a config file example is given in the Mozilla Wiki. A global collection of configurations can be found here.

The create config-v1.1.xml file needs to be available at either

https://mydomain.de/.well-known/autoconfig/mail/config-v1.1.xml

or you need to create an autoconfig and/or autodiscovery DNS entry (A or CNAME record) that specifies the webserver serving this file (s. below). It then should be located at

https://autodiscover.mydomain.de/mail/config-v1.1.xml

or

https://autodiscover.mydomain.de/autodiscover/autodiscover.xml

(the last one is based on my web server logs, and wasn’t mentioned in the related docs).

There’s also a SRV record _autodiscover._tcp mentioned:

KeyTypeValue
autoconfigA / CNAMEautodiscover.mydomain.de
autodiscoverA / CNAMEautodiscover.mydomain.de
_autodiscover._tcpSRV0 0 443 www.mydomain.de

While basic settings were taken from that config, OAUTH2 still did not appear as an option in thunderbird

Looking for further options

Some more research and I found this:

OAuth2. Works only on specific hardcoded servers, please see below. Should be added only as second alternative.

And below:

Unfortunately, this […] makes it impossible to support arbitrary OAuth2 servers. That’s why Thunderbird is forced to hardcode the servers that it supports and the respective client keys. That means that you cannot use OAuth2 for your own server. Only the servers listed on OAuth2Providers.jsm will work.

And a little further down:

Implementation note: While Thunderbird supports <authentication>OAuth2</authentication>, it does not support the <oAuth2> contents (server URL etc).

So the required code to support OAUTH2 seems to be in place, but someone decided that they’ll be only available for some of the “big players”…

The official reason mentioned is the requirement to get valid tokens from the mail providers (which in turn often require some kind of agreement). So there’s effort involved with every (commercial) provider, and that’s why people running their own services cannot use it … what a shame.

Some time later I also found this presentation (German only) from the University of applied sciences Deggendorf which contains a nice wrap-up of OAUTH2. They seem to be running their roundcube installation with OAUTH2 authentication.

They also mention (a 6 year old!) bug report #1602166 (RFE) complaining about this missing feature.

But maybe there’s hope: In this bug report thread there’s also a proposed solution (December 2024) … so maybe thunderbird will get OAUTH2 support sometime (soon?)…

Client #2: mutt

Mutt also supports OAUTH2 using some plugin (part of contrib/):

linux # sudo apt install mutt
linux # dpkg -L mutt  | grep oauth
/usr/share/doc/mutt/examples/mutt_oauth2.py
/usr/share/doc/mutt/examples/mutt_oauth2.py.README

Getting the settings right is however a bit challenging, so I tried to get the right settings for my keycloak installation by comparing the pre-defined configuration (e.g. from Microsoft):

linux # curl https://keycloak.mydomain.de/realms/MYDOMAIN/.well-known/openid-configuration | jq
<...>
linux # curl https://login.microsoftonline.com/common/v2.0/.well-known/openid-configuration | jq
<...>

As a first step we need to modify mutt_oauth2.py::

  1. Change ‘YOUR_GPG_IDENTITY’ to our newly created gpg identity string (make sure to run “gpg --gen-key” as described in the README).
  2. Add a new OAUTH2 setup to the existing “registrations”

Here’s how it could look like:

linux # vi mutt_oauth2.py
<...>
ENCRYPTION_PIPE = ['gpg', '--encrypt', '--recipient', 'My mutt_oauth2 token store']
<...>
registrations = {
    'MYDOMAIN': {
        'authorize_endpoint': 'https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/auth',
        'devicecode_endpoint': 'https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/auth/device',
        'token_endpoint': 'https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/token',
         'redirect_uri': 'https://keycloak.mydomain.de/realms/MYDOMAIN/',
        'imap_endpoint': 'imap.mydomain.de',
        'smtp_endpoint': 'mail.mydomain.de',
        'sasl_method': 'XOAUTH2',
        'scope': 'openid',
        'client_id': 'oidc_clientid',
        'client_secret': 'oidc_secret',
    },
<...>

Now we need to obtain an access token (will be stored in the file “marcel@mydomain.de.tokens”):

linux # ./mutt_oauth2.py --verbose --authorize marcel@mydomain.de.tokens
Available app and endpoint registrations: google microsoft mydomain
OAuth2 registration: mydomain
Preferred OAuth2 flow ("authcode" or "localhostauthcode" or "devicecode"): authcode
Account e-mail address: marcel@mydomain.de
https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/auth?client_id=oidc_cliendid&scope=openid&login_hint=marcel%40mydomain.de&response_type=code&redirect_uri=https%3A%2F%2Fkeycloak.mydomain.de%2Frealms%2FMYDOMAIN%2F&code_challenge=BnYc<...>I4_2M&code_challenge_method=S256
Visit displayed URL to retrieve authorization code. Enter code from server (might be in browser address bar):

Copy the URL to your favorite browser, log into keycloak and copy the trailing code from the browser address bar into the shell running the above command. If everything worked fine you’ll get something like:

<...>
Exchanging the authorization code for an access token
NOTICE: Obtained new access token, expires 2025-10-12T17:51:43.733427.
Access Token: eyJhbG<... long token here ...>

Now let’s run a first login test:

linux # ./mutt_oauth2.py marcel@mydomain.de.tokens --verbose --test
NOTICE: Invalid or expired access token; using refresh token to obtain new access token.
NOTICE: Obtained new access token, expires 2025-10-12T20:42:03.831683.
Access Token: eyJhb<... long token here ...>
IMAP authentication succeeded
SMTP authentication succeeded

(Normally the test script would also try POP3 access, but as I don’t use it I just commented this test in mutt_oauth2.py to obtain the above output).

So now all that’s left is to create a basic mutt configuration, to access my mail systems:

linux # vi ~/.muttrc
set imap_user = 'marcel@mydomain.de'
set imap_pass = ''
set spoolfile = imaps://imap.mydomain.de/Inbox
set folder = imaps://imap.mydomain.de/
set copy = no
set postponed="imaps://imap.mydomain.de/Drafts"
set mbox="$HOME/Mail"
# OAUTH2:
set imap_authenticators="oauthbearer:xoauth2"
set imap_oauth_refresh_command="/home/marcel/mutt/mutt_oauth2.py //home/marcel/mutt/${imap_user}.tokens"
set smtp_authenticators=${imap_authenticators}
set smtp_oauth_refresh_command=${imap_oauth_refresh_command}

Now simply calling mutt is enough to read the first mails (after successful OAUTH2 authentication)!

Client #3: Roundcube

As I’m running quite a few services in docker containers I also decided to test roundcube by using its docker image (I went for the all-in-one image roundcube/roundcubemail:latest-apache).

I’m using an external reverse proxy to handle SSL termination, so keep that in mind.

I was using the default config (only adding the imap and smtp server settings). After that all I had to do was add some OpenID config and restart.

linux # vi /var/www/html/config/config.inc.php
<...>
    $config['use_https'] = true;

    $config['oauth_provider'] = 'generic';
    $config['oauth_provider_name'] = 'MyDomain OIDC';
    $config['oauth_client_id'] = 'oidc_clientid';
    $config['oauth_client_secret'] = 'oidc_secret';
    $config['oauth_auth_uri'] = 'https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/auth?';
    $config['oauth_token_uri'] = 'https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/token';
    $config['oauth_identity_uri'] = 'https://keycloak.mydomain.de/realms/MYDOMAIN/protocol/openid-connect/userinfo';
    $config['oauth_verify_peer'] = true;
    $config['oauth_identity_fields'] = null;
    $config['oauth_login_redirect'] = false;
    $config['oauth_auth_parameters'] = [];
    $config['oauth_scope'] = 'email openid profile';
    // Optionally, skip Roundcube's login page
    // $config['oauth_login_redirect'] = true;
<...>

On first attempt I got an error from keycloak when trying to login: “Invalid redirect_uri”, so make sure to add the redirect_uri value from the URL to your OpenID clients “Valid redirect URIs” (in my case “https://roundcube.mydomain.de/index.php/login/auth“).

Nothing more … just works.

Client #4: Android FairEmail

According to its FAQ, OAuth should be supported for arbitrary setups. However configuration needs to be provided in a special XML format.

Maybe also tomorrow … or the day after …

Client #5: Evolution

Looks like Evolution also only supports OAUTH2 for Microsoft.

Client #6: Snappymail (Nextcloud)

I’ll have to either switch from my current SAML authentication for OIDC or I’ll have to first set up a new demo system using Nextcloud with OIDC … but not today 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *