Welcome to the DjaoDjin Blog!

A place to share experiences in building Software-as-a-Service.

Single sign-on through OpenLDAP on Fedora 21

by Sebastien Mirolo on Tue, 31 Mar 2015

Whether you work with a lot of contractors that come and go or you are dealing with the pains of a growing business, there is a point where copying ssh keys around on your EC2 instances does not cut it anymore. Time for a centralized login solution.

SSSD is the newest official daemon on Fedora 21 for delegating account management. SSSD integrates well with a variety of services, including LDAP and PAM.

Unfortunately because of restrictions in PAM itself, SSHD bypasses PAM when it comes to key-pair authentication. Since we still want to use SSH keys to login into our instances, we rely on the new AuthorizedKeysCommand introduced in OpenSSH 6.2.

Technically we will connect to our OpenLDAP server twice, once through /usr/libexec/openssh/ssh-ldap-wrapper (script run by SSHD) to retrieve the public key for a login attempt and once through SSSD via PAM to retrieve the POSIX user account information.

Getting ready to debug

If there is one thing dealing with system installation that is guaranteed is that you will have to go through a lot of logs to figure out what is going on. You will also have to go through a lot of documentation and history to figure out how to enable logging in each parts of the system.

Fedora 21 disabled syslog and relies on journalctl by default. I haven't looked much into why yet. I just love text files and having spent some real quality time with syslog-ng, I will just install it first here.

$ sudo yum install syslog-ng syslog-ng-libdbi
$ sudo systemctl enable syslog-ng
Failed to execute operation: File exists

There is already a symlink for /etc/systemd/system/syslog.service so we just remove it and try again.

$ sudo rm /etc/systemd/system/syslog.service
$ sudo systemctl start syslog-ng

Then of course, there are always SELinux to deal with before any tool starts to work correctly on a system these days.

$ sudo systemctl start syslog-ng
$ sudo sh -c 'grep syslog-ng /var/log/audit/audit.log | audit2why -M syslog-ng'
$ sudo sh -c 'grep syslog-ng /var/log/audit/audit.log | audit2allow -M syslog-ng'
$ semodule -i syslog-ng.pp
$ sudo service syslog-ng start

Note you might still be in for a rough ride trying to write output to /var/log/secure. Reading Journald in conjunction with syslog, it seems systemd default works well with syslog-ng 3.6.

$ systemctl --version
systemd 216
$ syslog-ng --version
syslog-ng 3.5.6
$ diff -u prev /etc/systemd/journald.conf
$ sudo systemctl restart systemd-journald

For SSSD, we add debug_level under the LDAP section and will read the logs in /var/log/sssd/sssd_LDAP.log.

+debug_level = 9

Configuring LDAP

This post explains in detail how to add schemas for sshPublicKey in the OpenLDAP directory so that public keys can be stored alongside a user information.

$ cat openssh-ldap.ldif
dn: cn=openssh-openldap,cn=schema,cn=config
objectClass: olcSchemaConfig
cn: openssh-openldap
olcAttributeTypes: {0}( NAME 'sshPublicKey' DES
 C 'MANDATORY: OpenSSH Public key' EQUALITY octetStringMatch SYNTAX
 1.1466. )
olcObjectClasses: {0}( NAME 'ldapPublicKey' DESC
  'MANDATORY: OpenSSH LPK objectclass' SUP top AUXILIARY MUST ( sshPublicKey $
  uid ) )

$ ldapadd -x -H ldap:/// -f ./openssh-ldap.ldif -D "cn=config" -W
adding new entry "cn=openssh-openldap,cn=schema,cn=config"

Since I am a bit lazy to install a web or GUI tool to modify a LDAP entry, I will do it through the command line. The syntax is a little arcane and the error messages not very informative.

ldap_add: Object class violation (65) additional info: no structural object class provided

Biggest time spent was to realize you actually do need '-' characters in your ldif file.

$ cat johnd-openssh-ldap.ldif
dn: uid=johnd,ou=people,dc=mydomain,dc=com
changetype: modify
add: objectClass
objectClass: ldapPublicKey
add: sshPublicKey
sshPublicKey: john-doe-public-key

$ sudo ldapmodify -x -H ldap:/// -f johnd-openssh-ldap-2.ldif \
    -D "cn=Manager,dc=mydomain,dc=com" -W

Now that the required information is in the LDAP directory, it is time to check we can access it on the remote machine. We install our own version of the openldap clients on the remote machine to test connectivity to the LDAP server.

Our OpenLDAP server is solely listening on the private IP, none-the-less we still want to encrypt connections and use certificate authentication.

$ scp ldap.crt remoteHost:/etc/pki/tls/certs

$ diff -u prev /etc/openldap/ldap.conf
-#TLS_CACERTDIR /etc/openldap/cacerts
TLS_CACERT      /etc/pki/tls/certs/ldap.crt
TLS_REQCERT     demand

$ ldapsearch -d -1 -xLLL  -H ldaps://LDAPHostPrivateIP/ -b "dc=mydomain,dc=com" uid=johnd
ldap_connect_to_host: TCP LDAPHostPrivateIP:636
ldap_new_socket: 3
ldap_prepare_socket: 3
ldap_connect_to_host: Trying LDAPHostPrivateIP:636

Configuring SSSD

Authentication through LDAP

$ sudo yum update
$ sudo yum install authconfig sssd
$ sudo vi /etc/sssd/sssd.conf
config_file_version = 2
reconnection_retries = 3
services = nss, pam, sudo
# SSSD will not start if you do not configure any domains.
# Add new domain configurations as [domain/] sections, and
# then add the list of domains (in the order you want them to be
# queried) to the "domains" attribute below and uncomment it.
domains = LDAP

filter_users = root,ldap,named,avahi,haldaemon,dbus,radiusd,news,nscd
reconnection_retries = 3

reconnection_retries = 3


# Debugging:
debug_level = 9

ldap_tls_reqcert = demand
# Note that enabling enumeration will have a moderate performance impact.
# Consequently, the default value for enumeration is FALSE.
# Refer to the sssd.conf man page for full details.
enumerate = true
auth_provider = ldap
# ldap_schema can be set to "rfc2307", which stores group member names in the
# "memberuid" attribute, or to "rfc2307bis", which stores group member DNs in
# the "member" attribute. If you do not know this value, ask your LDAP
# administrator.
#ldap_schema = rfc2307bis
ldap_schema = rfc2307
ldap_search_base = dc=mydomain,dc=com
ldap_group_member = uniquemember
id_provider = ldap
ldap_id_use_start_tls = False
chpass_provider = ldap
ldap_uri = ldaps://LDAPHostPrivateIP/
ldap_chpass_uri = ldaps://LDAPHostPrivateIP/
# Allow offline logins by locally storing password hashes (default: false).
cache_credentials = True
ldap_tls_cacert = /etc/pki/tls/certs/ldap.crt
entry_cache_timeout = 600
ldap_network_timeout = 3
sudo_provider = ldap
ldap_sudo_search_base = ou=sudoers,dc=mydomain,dc=com
# Enable group mapping otherwise only the user's primary group will map correctly. Without this
# defined group membership won't work
ldap_group_object_class = posixGroup
ldap_group_search_base = ou=group,dc=mydomain,dc=com
ldap_group_name = cn
ldap_group_member = memberUid

$ sudo chmod 600 /etc/sssd/sssd.conf
$ sudo authconfig --update --enablesssd --enablesssdauth
$ sudo systemctl enable sssd
$ sudo service sssd start

Followed by a little bit of magic for SELinux:

$ setsebool -P authlogin_nsswitch_use_ldap 1

Configuring SSHD

We are now able to retrieve entries from the LDAP server so it is time to check we can remotely login into the machine. In order to do that, we will install ssh-ldap-helper

$ sudo yum install openssh-ldap

$ diff -u /etc/ssh/sshd_config
-#AuthorizedKeysCommand none
-#AuthorizedKeysCommandUser nobody
+AuthorizedKeysCommand      /usr/libexec/openssh/ssh-ldap-wrapper
+AuthorizedKeysCommandUser  nobody

$ diff -u prev /etc/ssh/ldap.conf
URI             ldaps://LDAPHostPrivateIP/
BASE            ou=people,dc=mydomain,dc=com
TLS_CACERT      /etc/pki/tls/certs/ldap.crt
TLS_REQCERT     demand
TIMELIMIT       15
TIMEOUT         20

$ sudo service sshd restart

If ssh-ldap-wrapper times out trying to connect to the OpenLDAP server, sshd will try to use an authorized_keys on the local file system. Unfortunately the ssh client connection can time out before ssh-ldap-wrapper times out and you will not be able to connect to the machine when there are some connectivity issues between the machine and the LDAP server. Since the EC2 instances all start with a key to connect as the fedora user, we modify slightly the ssh-ldap-wrapper script to bypass connection to the LDAP server in that case.

# avoid time-out
$ diff -u prev /usr/libexec/openssh/ssh-ldap-wrapper
+if [ "$1" == "fedora" ] ; then
+    exit 1
exec /usr/libexec/openssh/ssh-ldap-helper -s "$1"

First attempt to remotely login into the machine will trigger SELinux denials. We allow ssh-ldap-helper to connect to the LDAP port.

$ ssh -v johnd@ec2-xx-xx-xx-xx.compute.amazonaws.com

$ sudo sh -c 'grep ssh-ldap-helper /var/log/audit/audit.log | audit2why -M ssh-ldap-helper'
$ sudo sh -c 'grep ssh-ldap-helper /var/log/audit/audit.log | audit2allow -M ssh-ldap-helper'

That is it! We can now rely on our OpenLDAP directory to remotely connect to a machine. Time to read Secure Secure Shell to make the ssh connection really secure.