IBM Spectrum Scale with Standalone Docker

One might want to try running some containerized application with persistent volume using standalone docker host. Persistent storage can be carved from IBM Spectrum Scale. Following are the step to use storage from IBM Spectrum Scale for Container in case of standalone docker host provided docker host is having Spectrum Scale client installed and Filesystem is mounted.

    1. Create Fileset on IBM Spectrum Scale
      [root@dgnode4 ~]# mmcrfileset gpfs0 fileset1
      Fileset fileset1 created with id 1 root inode 24932.
      
      [root@dgnode4 ~]# mmlsfileset gpfs0
      Filesets in file system 'gpfs0':
      Name Status Path
      root Linked /ibm/gpfs0
      fileset1 Unlinked --
    2. Link Fileset

      [root@dgnode4 ~]# mmlinkfileset gpfs0 fileset1 -J /ibm/gpfs0/fileset1
      Fileset fileset1 linked at /ibm/gpfs0/fileset1
    3. Run container with fileset as a persistent storage
      [root@dgnode4 ~]# docker run -v /ibm/gpfs0/fileset1:/gpfs --rm -it alpine /bin/sh
      Unable to find image 'alpine:latest' locally
      latest: Pulling from library/alpine
      Digest: sha256:e1871801d30885a610511c867de0d6baca7ed4e6a2573d506bbec7fd3b03873f
      Status: Downloaded newer image for alpine:latest
      / #
    4. Create file inside the /gpfs inside container and confirm same is available in fileset
      #Inside container
      / # cd /gpfs/
      /gpfs # touch file_from_container
      #On host
      [root@dgnode4 ~]# ls /ibm/gpfs0/fileset1
      file_from_container

“These are my personal views and do not necessarily reflect that of my employer”

Advertisements

Creating TLS Certificates

Developer/Tester need TLS certificates now and then for testing and development of product supporting secure communucation. Following is the docker-image based procedure to quickly create the TLS certificates.

Steps :

  1. clone the git repo (https://github.com/deepak-ghuge/create-tls-certificates.git)
  2. docker build -t createtlscert
  3. docker run -it –rm -v <hostdir>:/crt createtlscert <ip address>
    eg : docker run -it –rm -v /tmp/crt:/crt createtlscert 192.168.122.1

TLS certificates will be created for CN=<ip address> and copied in the specified <hostdir> directory.

Note – Creating TLS certificate using this procedure is only meant for Test purpose only.

“These are my personal views and do not necessarily reflect that of my employer”

IBM Spectrum Scale Object Authentication – Basic problem determination

The first step towards start using Spectrum Scale Object store is to properly plan and configure the Authentication mechanism for Spectrum Scale Object store to make sure Object store contents are being accessed by Authorized entities only. The famous and majorly used Authentication schemes are Lightweight Directory Access Protocol (LDAP) and Microsoft Active Directory. Organizations customize the Authentication server per their need because of which most authentication configuration tends to be unique. Spectrum Scale object authentication configuration supports most such authentication configurations. Common authentication issues, error and resolution for Spectrum Scale object authentication are captured in following presentation.

https://www.slideshare.net/SmitaRaut/ibm-spectrum-scale-authentication-for-object-deep-dive

CloudBerry Explorer with IBM Spectrum Scale

Steps to enable CloudBerry explorer to use Object Storage from IBM Spectrum Scale

Prerequisite:
IBM Spectrum Scale 4.2.2 with Object service enabled.
CloudBerry Explorer for Openstack Storage – Build-1.6.2.63

  1. Verify IBM Spectrum Scale Object storage is enabled by running following command[root@c1n3 ~]# mmces service list
    Enabled services: OBJ
    OBJ is running

    [root@c1n3 ~]# mmuserauth service list –data-access-method object
    OBJECT access configuration : LOCAL
    PARAMETERS               VALUES
    ————————————————-
    ENABLE_KS_SSL            false
    ENABLE_KS_CASIGNING      false
    KS_ADMIN_USER            none

  2. Configure the CloudBerry Explorer with IBM Spectrum Scale Object Storage12344
    56789
    10
  3. List/Create/Upload objects/Containers using Swift Client[root@c1n3 ~]# swift list
    myFirstContainerFromCloudBerry

    [root@c1n3 ~]# swift post containerFromWwiftClient

    [root@c1n3 ~]# swift list
    containerFromWwiftClient
    myFirstContainerFromCloudBerry

    [root@c1n3 ~]# swift upload myFirstContainerFromCloudBerry anaconda-ks.cfg
    anaconda-ks.cfg

  4.  View Object/Containers from CloudBerry Explorer
    1211

 

“These are my personal views and do not necessarily reflect that of my employer”

 

 

Decoding IBM Spectrum Scale Object Authentication configuration with Active Directory(AD)

            One can use Installer toolkit during deploy to configure Spectrum Scale object with Active Directory or use mmuserauth CLI to configure Spectrum Scale object with Active Directory after enabling Spectrum Scale object

In this blog entry I am making use of mmuserauth CLI to configure Spectrum Scale object with Active Directory

Example of mmuserauth CLI for configuring Spectrum Scale object with Active Directory.

mmuserauth service create

–data-access-method object
–type ad
–servers 10.0.0.27
–user-dn dc=sonas,dc=com
–base-dn dc=sonas,dc=com
–user-name administrator@sonas.com
–password Passw0rd
–ks-admin-user keystoneadminuser
–ks-swift-user swiftserviceuser
–ks-swift-pwd swiftPassw0rd
[–user-objectclass organizationalPerson]
[–user_name_attrib sAMAccountName]
[–user-id-attrib CN ]
Note : [option] denotes the optional parameter with default values

There are more option to this cli, Above listed are important.
Refer mmuserauth man page for details

Things to check before configuring the Spectrum Scale object with Active Directory

  1. Is Active Directory Server is correct, Reachable and Bind details are working ?
It simply means validating following parameter from mmuserauth example given at the start
–servers 10.0.0.27
–user-dn cn=users,dc=sonas,dc=com
–user-name administrator@sonas.com
–password Passw0rd
  Ldapsearch sample command to validate AD details 

/usr/bin/ldapsearch -x -H ldap://<–server> -b <–user-dn> -s sub -o nettimeout=10 -LLL -A -D < –username > -w <–password>

The command for my environment
/usr/bin/ldapsearch -x -H ldap://10.0.0.27 -b cn=users,dc=sonas,dc=com -s sub -o nettimeout=10 -LLL -A -D administrator@sonas.com -w Passw0rd

Above command print all the user details on command line so you can use following to limit the details on screen

/usr/bin/ldapsearch -x -H ldap://10.0.0.27 -b cn=users,dc=sonas,dc=com -s sub -o nettimeout=10 -LLL -A -D administrator@sonas.com -w Passw0rd dn

OR

/usr/bin/ldapsearch -x -H ldap://10.0.0.27 -b cn=users,dc=sonas,dc=com -s sub -o nettimeout=10 -LLL -A -D administrator@sonas.com -w Passw0rd cn=administrator

Note : Provide Bind user ( ie –user-name ) in
DN format (example : CN=Administrator,CN=Users,DC=SONAS,DC=COM)
or
user logon name format (example : administrator@sonas.com)

If this step is successful then your Active Directory is correct, reachable and provided bind details are Valid.

  1. Is –ks-admin-user keystoneadminuser is present in Active Directory ?

Note : Keystoneadminuser should be present in Active Directory under provide –user-dn and It should have Valid Password

To validate –ks-admin-user following options need careful attention
[–user-objectclass organizationalPerson]
[–user_name_attrib sAMAccountName]
[–user-id-attrib CN ]

Ldapsearch sample command to validate the –ks-admin-user
/usr/bin/ldapsearch -x -H ldap://10.0.0.27 -b cn=users,dc=sonas,dc=com -s sub -o nettimeout=10 -LLL -A -D administrator@sonas.com -w Passw0rd ‘(&(objectClass=organizationalPerson)(|(sAMAccountName=keystoneadminuser)(CN=keystoneadminuser)))’ -z1 dn

This command should return something like
dn: CN=keystoneadminuser,CN=Users,DC=SONAS,DC=COM

This steps confirm that you –ks-admin-user is good to go with.

Note : Use shortname for –ks-admin-user. Do not user DN or user logon name
For example I am using keystoneadminuser not the
CN=keystoneadminuser,CN=Users,DC=SONAS,DC=COM
or
keystoneadminuser@sonas.com

  1. is–ks-swift-user and –ks-swift-pwd is correct ?

Validating –ks-swift-user exist in Active Directory
–ks-swift-user swiftserviceuser
–ks-swift-pwd Passw0rd

Ldapsearch sample command to check if –ks-swift-user exist using admin credentials
/usr/bin/ldapsearch -x -H ldap://10.0.0.27 -b cn=users,dc=sonas,dc=com -s sub -o nettimeout=10 -LLL -A -D administrator@sonas.com -w Passw0rd ‘(&(objectClass=organizationalPerson)(|(sAMAccountName=swiftserviceuser)(CN=swiftserviceuser)))’ dn

This command will return something like dn: CN=swiftserviceuser,CN=Users,DC=SONAS,DC=COM use this DN to validate the provided –ks-swift-pwd

Ldapsearch sample command to check if –ks-swift-pwd exist is correct

/usr/bin/ldapsearch -x -H ldap://10.0.0.27 -b cn=users,dc=sonas,dc=com -s sub -o nettimeout=10 -LLL -A -D CN=swiftserviceuser,CN=Users,DC=SONAS,DC=COM -w swiftPassw0rd ‘(objectClass=*)’ -z1 dn

If this is successful then you are ready to configure Spectrum Scale with Active Directory

Note : Use shortname for –ks-swift-user. Do not user DN or user logon name
For example I am using swiftserviceuser not the
CN=swiftserviceuser,CN=Users,DC=SONAS,DC=COM or
swiftserviceuser@sonas.com

  1. Importance of –user-objectclass, –user-name-attrib and –user-id-attrib ?
All these are used as initial filter for Keystone users. Only user who has these attribute will be listed in Keystone users.
Default for Active Directory, You can change it as per your environment.
[–user-objectclass organizationalPerson]
[–user-name-attrib sAMAccountName]
[–user-id-attrib CN ]
Note : Choosing –user-id-attrib, It should be the attribute used in user DN. For example in all DN of users (CN=swiftserviceuser,CN=Users,DC=SONAS,DC=COM) in my environment it is CN. Hence I am using –user-id-attrib CN

“These are my personal views and do not necessarily reflect that of my employer”

Configuring IBM Spectrum Scale Object with SSL enabled External Keystone server

Prerequisite :  

  • IBM Spectrum Scale 4.2.1.x
  • SSL enabled Keystone server.
  • A swift user with valid password having ‘admin‘ role in ‘service‘ tenant of ‘default‘ domain in a external Keystone server

Endpoints on External Keystone server will look like

[root@externalKeystone ~]# openstack endpoint list
+———————————-+———–+————–+————–+———+———–+—————————————–+
| ID | Region | Service Name | Service Type | Enabled | Interface | URL |
+———————————-+———–+————–+————–+———+———–+—————————————–+
| 78cd570651f34b848890687c4f1578a9 | None | keystone | identity | True | public | https://cesip:5000/ |
| ff82aa833eec42b7a345a1c029b74959 | None | keystone | identity | True | internal | https://cesip:35357/ |
| 692c4bb6c4a14ece9f810ba4fc1944f1 | None | keystone | identity | True | admin | https://cesip:35357/ |
| 52b559474c03454eaef67af14a3c4afe | RegionOne | swift | object-store | True | public | http://c1ces:8080/v1/AUTH_%(tenant_id)s |
| ed14c1c50f3242a0aed61b9ccdfc8c4d | RegionOne | swift | object-store | True | internal | http://c1ces:8080/v1/AUTH_%(tenant_id)s |
| f26a242b71b34128abf87cbc1e8937aa | RegionOne | swift | object-store | True | admin | http://c1ces:8080 |
+———————————-+———–+————–+————–+———+———–+—————————————–+

 

On IBM Spectrum scale, Object will be already configured.
Remove the Current Object Authentication and Id-mapping using following command.

[root@c1n3 ~]# mmuserauth service remove –data-access-method object 
mmuserauth service remove: Command successfully completed

[root@c1n3 ~]# mmuserauth service remove –data-access-method object –idmapdelete
mmuserauth service remove: Command successfully completed

Copy CA-Certificates used on External Keystone server on Protocol node of IBM Spectrum Scale at following location.

[root@c1n3 ~]# ls /var/mmfs/tmp/ks_ext_cacert.pem
/var/mmfs/tmp/ks_ext_cacert.pem

Run mmuserauth command on protocol node where CA-Certificate is copied

[root@c1n3 ~]# mmuserauth service create –data-access-method object –type userdefined –ks-swift-user swift –ks-swift-pwd passw0rd –ks-ext-endpoint https://cesip:5000/v3 –enable-ks-ssl
mmcesobjcrbase: Validating execution environment.
mmcesobjcrbase: Validating Keystone environment.
mmcesobjcrbase: Configuration complete.
Object authentication configuration completed successfully.

Note : External keystone server must be reachable from protocol nodes using the hostname used for creating SSL Certificates for external Keystone. Hostname in Keystone Endpoints should also match the hostname used for creating SSL certificates

IBM Spectrum scale Object is successfully configured with SSL enabled external Keystone server.

~/openrc file on protocol node will be updated with details of External Keystone.

[root@c1n3 ~]# cat openrc
export OS_AUTH_URL=”https://cesip:5000/v3
export OS_CACERT=”/etc/swift/ks_ext_cacert.pem”
export OS_IDENTITY_API_VERSION=3
export OS_AUTH_VERSION=3
export OS_USERNAME=”admin”
export OS_PASSWORD=””
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_NAME=admin
export OS_PROJECT_DOMAIN_NAME=Default

Update the ~/openrc with valid username and password from external Keystone server.

[root@c1n3 ~]# source ~/openrc

Verify IBM Spectrum Scale Object is working with external Keystone server.

[root@c1n3 ~]# swift stat
Account: AUTH_0557d5eb51294e48b1c5041c684b4f66
Containers: 0
Objects: 0
Bytes: 0
Content-Type: text/plain; charset=utf-8
Keep-Alive: timeout=5, max=100

“These are my personal views and do not necessarily reflect that of my employer”