Configuring IBM Spectrum Scale Object with SSL enabled External Keystone server

Prerequisite :  

  • IBM Spectrum Scale 4.2.1.x
  • SSL enabled Keystone server.
  • A swift user with valid password having ‘admin‘ role in ‘service‘ tenant of ‘default‘ domain in a external Keystone server

Endpoints on External Keystone server will look like

[root@externalKeystone ~]# openstack endpoint list
+———————————-+———–+————–+————–+———+———–+—————————————–+
| ID | Region | Service Name | Service Type | Enabled | Interface | URL |
+———————————-+———–+————–+————–+———+———–+—————————————–+
| 78cd570651f34b848890687c4f1578a9 | None | keystone | identity | True | public | https://cesip:5000/ |
| ff82aa833eec42b7a345a1c029b74959 | None | keystone | identity | True | internal | https://cesip:35357/ |
| 692c4bb6c4a14ece9f810ba4fc1944f1 | None | keystone | identity | True | admin | https://cesip:35357/ |
| 52b559474c03454eaef67af14a3c4afe | RegionOne | swift | object-store | True | public | http://c1ces:8080/v1/AUTH_%(tenant_id)s |
| ed14c1c50f3242a0aed61b9ccdfc8c4d | RegionOne | swift | object-store | True | internal | http://c1ces:8080/v1/AUTH_%(tenant_id)s |
| f26a242b71b34128abf87cbc1e8937aa | RegionOne | swift | object-store | True | admin | http://c1ces:8080 |
+———————————-+———–+————–+————–+———+———–+—————————————–+

 

On IBM Spectrum scale, Object will be already configured.
Remove the Current Object Authentication and Id-mapping using following command.

[root@c1n3 ~]# mmuserauth service remove –data-access-method object 
mmuserauth service remove: Command successfully completed

[root@c1n3 ~]# mmuserauth service remove –data-access-method object –idmapdelete
mmuserauth service remove: Command successfully completed

Copy CA-Certificates used on External Keystone server on Protocol node of IBM Spectrum Scale at following location.

[root@c1n3 ~]# ls /var/mmfs/tmp/ks_ext_cacert.pem
/var/mmfs/tmp/ks_ext_cacert.pem

Run mmuserauth command on protocol node where CA-Certificate is copied

[root@c1n3 ~]# mmuserauth service create –data-access-method object –type userdefined –ks-swift-user swift –ks-swift-pwd passw0rd –ks-ext-endpoint https://cesip:5000/v3 –enable-ks-ssl
mmcesobjcrbase: Validating execution environment.
mmcesobjcrbase: Validating Keystone environment.
mmcesobjcrbase: Configuration complete.
Object authentication configuration completed successfully.

Note : External keystone server must be reachable from protocol nodes using the hostname used for creating SSL Certificates for external Keystone. Hostname in Keystone Endpoints should also match the hostname used for creating SSL certificates

IBM Spectrum scale Object is successfully configured with SSL enabled external Keystone server.

~/openrc file on protocol node will be updated with details of External Keystone.

[root@c1n3 ~]# cat openrc
export OS_AUTH_URL=”https://cesip:5000/v3
export OS_CACERT=”/etc/swift/ks_ext_cacert.pem”
export OS_IDENTITY_API_VERSION=3
export OS_AUTH_VERSION=3
export OS_USERNAME=”admin”
export OS_PASSWORD=””
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_NAME=admin
export OS_PROJECT_DOMAIN_NAME=Default

Update the ~/openrc with valid username and password from external Keystone server.

[root@c1n3 ~]# source ~/openrc

Verify IBM Spectrum Scale Object is working with external Keystone server.

[root@c1n3 ~]# swift stat
Account: AUTH_0557d5eb51294e48b1c5041c684b4f66
Containers: 0
Objects: 0
Bytes: 0
Content-Type: text/plain; charset=utf-8
Keep-Alive: timeout=5, max=100

“These are my personal views and do not necessarily reflect that of my employer”

 

Setting up Openstack Keystone with Kerberos..

Prerequisite :
– Kerberos server(MIT Kerberos)
– Keystone server with Apache (Redhat 7.*)

1. On Kerberos Server

Add/Create new principle for Keystone service with Kerberos server Hostname.

[root@krbserver ~]# kadmin.local 
Authenticating as principal root/admin@PUNE-KDC.COM with password.
kadmin.local: addprinc -randkey HTTP/spectrumscale1
kadmin.local: ktadd -k /tmp/http.keytab HTTP/spectrumscale1

Copy /tmp/http.keytab on Keystone server :/etc/httpd/conf/httpd.keytab

2. On Keystone Server

  1. Install Apache kerberos module
    [root@KeystoneServer ~]# yum install mod_auth_kerb -y

  2. Update wsgi-keystone.conf with Kerberos configuration
    [root@KeystoneServer ~]# cat /etc/httpd/conf.d/wsgi-keystone.conf
    LoadModule auth_kerb_module modules/mod_auth_kerb.so
    WSGISocketPrefix /var/run/wsgi
    SetEnv APACHE_RUN_USER keystone
    SetEnv APACHE_RUN_GROUP keystone
    <VirtualHost *:5000>
    WSGIDaemonProcess keystone-public user=keystone group=keystone display-name=keystone-public
    WSGIProcessGroup keystone-public
    WSGIImportScript /var/www/cgi-bin/keystone/main process-group=keystone-public application-group=%{GLOBAL}
    WSGIScriptAlias /krb “/var/www/cgi-bin/keystone/main”
    WSGIScriptAlias / “/var/www/cgi-bin/keystone/main”
    <Location “/krb/v3/auth/tokens”>
    LogLevel debug
    AuthType Kerberos
    AuthName “Kerberos Login”
    KrbMethodNegotiate on
    KrbMethodK5Passwd off
    KrbServiceName HTTP/spectrumscale1
    KrbAuthRealms PUNE-KDC.COM
    Krb5KeyTab /etc/httpd/conf/httpd.keytab
    KrbLocalUserMapping on
    require valid-user
    </Location>
    ErrorLog /var/log/keystone/httpd-error.log
    LogLevel info
    CustomLog /var/log/keystone/httpd-access.log combined
    LimitRequestBody 16384
    </VirtualHost>

    Listen 35357

    <VirtualHost *:35357>
    WSGIDaemonProcess keystone-admin user=keystone group=keystone display-name=keystone-admin
    WSGIProcessGroup keystone-admin
    WSGIImportScript /var/www/cgi-bin/keystone/main process-group=keystone-admin application-group=%{GLOBAL}
    WSGIScriptAlias /krb “/var/www/cgi-bin/keystone/admin”
    WSGIScriptAlias / “/var/www/cgi-bin/keystone/admin”
    <Location “/krb/v3/auth/tokens”>
    LogLevel debug
    AuthType Kerberos
    AuthName “Kerberos Login”
    KrbMethodNegotiate on
    KrbMethodK5Passwd off
    KrbServiceName HTTP/spectrumscale1
    KrbAuthRealms PUNE-KDC.COM
    Krb5KeyTab /etc/httpd/conf/httpd.keytab
    KrbLocalUserMapping on
    require valid-user
    </Location>
    ErrorLog /var/log/keystone/httpd-error.log
    LogLevel info
    CustomLog /var/log/keystone/httpd-access.log combined
    LimitRequestBody 16384
    </VirtualHost>

  3. Install kerberos client package
    yum install krb5-workstation -y

  4. Configure the Kerberos client(as per your environment)
    [root@KeystoneServer ~]# cat /etc/krb5.conf
    [logging]

    [libdefaults]

    default_realm = PUNE-KDC.COM
    [realms]
    PUNE-KDC.COM = {
    kdc = krbserver.in.ibm.com:88
    admin_server = krbserver.in.ibm.com:749

    }
    [domain_realm]
    in.ibm.com = PUNE-KDC.COM
    .in.ibm.com = PUNE-KDC.COM

  5. Get kerberos token for user
    [root@KeystoneServer ~]# kinit userrw
    Password for userrw@PUNE-KDC.COM:
    [root@KeystoneServer ~]# klist
    Ticket cache: KEYRING:persistent:0:0
    Default principal: userrw@PUNE-KDC.COM
    Valid starting Expires Service principal
    2016-02-26T12:17:28 2016-02-27T11:45:15 krbtgt/PUNE-KDC.COM@PUNE-KDC.COM
    renew until 2016-02-26T12:17:28

  6. Curl request for keystone token without username and password
    [root@KeystoneServer ~]# curl -i -H “Content-Type:application/json” –negotiate -u : -d ‘{ “auth”: { “identity”: { “methods”: []}, “scope”: { “project”: { “domain”: { “name”: “Default” }, “name”: “admin” } } } }’ -X POST http://SpectrumScale1:5000/krb/v3/auth/tokens
    HTTP/1.1 401 Unauthorized
    Date: Tue, 01 Mar 2016 07:35:52 GMT
    Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_auth_kerb/5.4 PHP/5.4.16 mod_wsgi/3.4 Python/2.7.5
    WWW-Authenticate: Negotiate
    Content-Length: 381
    Content-Type: text/html; charset=iso-8859-1
    HTTP/1.1 201 Created
    Date: Tue, 01 Mar 2016 07:35:52 GMT
    Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips mod_auth_kerb/5.4 PHP/5.4.16 mod_wsgi/3.4 Python/2.7.5
    X-Subject-Token: MIIHuQYJKoZIhvcNAQcCoIIHVqXmOQZM5KBfGnHhW3FMJW7nXDAELZ0X2s2WO9e6w==
    Vary: X-Auth-Token
    x-openstack-request-id: req-d6738e8b-b9b6-42e7-a615-ae03c8eb563e
    WWW-Authenticate: Negotiate YIGZBgkqhkiG9xIBAgICAG+BiTCBhqADAgEFoQMCAQ+iejB4oAMCARKicQRvwFfLUoOfh04
    …………
    …………
    3Tg5Ts8goQatVJ5JEnCYqkXIo8Yk5vYB7BWto2FRhDKzcAp75Qqciv6DgT8gnc6
    Content-Length: 1634
    Content-Type: application/json
    {“token”: {“methods”: [], “roles”: [{“id”: “e23776346ab747f4bddf7f056b8d62c9”, “name”: “admin”}], “expires_at”: “2016-03-31T07:35:52.528404Z”, “project”: {“domain”: {“id”: “default”, “name”: “Default”}, “id”: “bb86db4204e84b4b857241a0b23a62fb”, “name”: “admin”}, “catalog”: [{“endpoints”: [{“region_id”: null, “url”: “http://SpectrumScale1:35357/&#8221;, “region”: null, “interface”: “admin”, “id”: “1bd1b57b2cb14e74b5864261588a1e57”}, {“region_id”: null, “url”: “http://SpectrumScale1:35357/&#8221;, “region”: null, “interface”: “internal”, “id”: “43472eb0474d469c90d85a4dc7c6058d”}, {“region_id”: null, “url”: “http://SpectrumScale1:5000/&#8221;, “region”: null, “interface”: “public”, “id”: “6a9c87ec04f3480eb254568b60b8db07”}], “type”: “identity”, “id”: “7a22d7f78b224843811ef0708431eb46”, “name”: “keystone”} “extras”: {}, “user”: {“domain”: {“id”: “default”, “name”: “Default”}, “id”: “userrw”, “name”: “userrw”}, “audit_ids”: [“rcLqPQp5T6y9hKu12Tle2g”], “issued_at”: “2016-03-01T07:35:52.528436Z”}}

    “These are my personal views and do not necessarily reflect that of my employer

Configuring IBM Spectrum Scale Object With External Keystone

IBM Spectrum Scale provide an option to configure Object with External Keystone.

There are two method to achieve the same First is using Installer to Configure the Object with External Keystone and Other is to use mmuserauth cli for setting up Object Authentication. 

Prerequisite :
    Following entities must be present on keystone server which is hosted outside of Spectrum Scale cluster. 

  • User ‘swift’ and ‘admin’ with valid password
  • Project ‘service’ and ‘admin’
  • Role ‘admin’
  • ‘swift’ user should have ‘admin’ role in ‘service’ tenant
  • ‘admin’ user should have ‘admin’ role in ‘admin’ tenant
  • keystone service of type ‘identity’
  • Keystone endpoint
  • Swift service of type ‘object-store’
  • Swift endpoint.

Refer Configure Openstack-Keystone for IBM Spectrum Scale Object Storage for configuring external keystone server which fulfill all above requirement.

On Spectrum Scale cluster. 

# Remove the existing authentication if any using following command

[root@swiftnode ~]# mmuserauth service remove --data-access-method object
[root@swiftnode ~]# mmuserauth service remove --data-access-method object --idmapdelete

# Configure IBM Spectrum Scale Object with external keystone using following command

[root@swiftnode ~]# mmuserauth service create --data-access-method object --type userdefined --ks-ext-endpoint http://mykeystone:35357/v3 --ks-swift-user swift --ks-swift-pwd password

Spectrum Scale Object is configured with External Keystone and Waiting for you to upload lots of Object. Hurry up … 🙂

Disclaimer: The content of this post is not approved nor endorsed by IBM.

Configure Openstack-Keystone for IBM Spectrum Scale Object Storage

Steps for configuring External keystone server for IBM Spectrum Scale Object Storage

Prerequisite :
               RHEL 7 or 7.1 host with Enabled Redhat and Openstack Kilo repository

# Install openstack-keystone rpm and other required rpms from repository

 $ yum install openstack-keystone openstack-utils openldap-clients python-openstackclient -y 

# Add required firewall rule on node or stop the firewalld

$ service firewalld stop

# Update the keystone.conf. Update admin_token for administration/configuration

$ openstack-config --set /etc/keystone/keystone.conf DEFAULT admin_token ADMIN 

# Update the database connection. Assumption: Mysql(MariaDB) will be used and same node is used as database node.

$ openstack-config --set /etc/keystone/keystone.conf database connection 'mysql://keystone:Passw0rd@localhost/keystone'

# In this setup PKI is used for token. One can choose to use UUID for token, In that case skip following steps.

$ openstack-config --set /etc/keystone/keystone.conf token provider 'keystone.token.providers.pki.Provider'
$ keystone-manage pki_setup --keystone-user keystone --keystone-group keystone

# Install mariadb and initialize the keystone database

$ /usr/bin/openstack-db --service keystone --init --password password --rootpw password

$ service openstack-keystone start 

# At this stage Openstack-keystone service will be running

$ export OS_TOKEN="ADMIN"
$ export OS_IDENTITY_API_VERSION=3
$ export OS_URL=http://localhost:35357/v3 

#  Create required User,Project,Role entries in Keystone

$ openstack project create --domain default service
$ openstack project create --domain default admin

$ openstack user create --password password admin --domain default
$ openstack user create --password password swift --domain default

$ openstack role create admin

$ openstack role add --user admin --domain default admin
$ openstack role add --user admin --project admin admin
$ openstack role add --user swift --domain default admin
$ openstack role add --user swift --project service admin

# Create Keystone endpoints

$ openstack service create --name keystone identity
$ keystoneservice=`openstack service show keystone -f value -c id`
$ keystoneendpoint='mykeystone' #---> Change this as per hostname/dnsname of keystone
$ openstack endpoint create $keystoneservice public http://${keystoneendpoint}:5000/v3
$ openstack endpoint create $keystoneservice internal http://${keystoneendpoint}:5000/v3
$ openstack endpoint create $keystoneservice admin http://${keystoneendpoint}:35357/v3 

# Create Swift endpoints

$ openstack service create --name swift object-store
$ swiftservice=`openstack service show swift -f value -c id`
$ swiftendpoint='swiftnode'  #---> Change this as per hostname/dnsname of swift
$ openstack endpoint create $swiftservice public  "http://${swiftendpoint}:8080/v1/AUTH_%(tenant_id)s"
$ openstack endpoint create $swiftservice internal  "http://${swiftendpoint}:8080/v1/AUTH_%(tenant_id)s"
$ openstack endpoint create $swiftservice admin  "http://${swiftendpoint}:8080"

# Remove admin_token from keystone.conf

$ openstack-config --del /etc/keystone/keystone.conf DEFAULT admin_token

# Restart the openstack-keystone service to pickup the admin_token deletion.

$ service openstack-keystone restart


Disclaimer: The content of this post is not approved nor endorsed by IBM.

Now the keystone server is ready to be configured with IBM Spectrum Scale Object Store in USERDEFINED configuration.

Changing Token expiration in Openstack-Keystone

Sometime there is requirement of changing the Token expiration time in Openstack-keystone so that Token remain valid for the longer/shorter time based on requirement.

[root@keystone ~]# openstack-config –set /etc/keystone/keystone.conf token expiration 86400

[root@keystone ~]# service openstack-keystone restart
Redirecting to /bin/systemctl restart openstack-keystone.service

One can confirm the expiration time is actually changed by getting new token

[root@keystone ~]# date
Tue Oct 6 10:01:37 EDT 2015          #–> Current time 

[root@keystone ~]# source openrc
[root@keystone ~]# openstack token issue -c expires
+———+—————————–+
| Field | Value |
+———+—————————–+
| expires | 2015-10-07T14:01:39.753765Z |         #—-> Token Expiration time 24Hrs from current time
+———+—————————–+