close

 

We will use the minIO tool to transfer data from IBM Cloud Object Storage (ICOS) to AIX Server on IBM Cloud

Scenario:
– You store data in IBM Cloud Object Storage and now you want to transfer these data to AIX server
Solution:
– We will build a Linux as intermediary role to transfer data from Cloud Object Storage – Linux – AIX server
Requirement:
– IBM Cloud account, you can create a trial account: https://cloud.ibm.com
– AIX server is running

1/ Create IBM Cloud Object Storage

– Login to and search for Object Storage, input requirement info and Create.

  • Click on Object Storage name –> Create a Bucket
  • Select option for Custom Bucket, input bucket info and ensure your Resiliency, Location and Storage Class options are appropriate. Then scroll down and click the Create Bucket button
  • Now click on Service credentials on the left side to create a key to access the bucket.
  • Provide your credential name and the click Add
  • Take note some below info to establish connection to bucket
    – access_key_id
    – secret_access_key
    – Endpoints: Use to connect
2/ Create Linux Instance
  • Login to IBM Cloud, from IBM Cloud Dashboard –> Create resource on the right side –> Select Virtual Server
  • Input requirement info
  • Scroll down and select some below options base on your requirement:
    – Profile: Select View all profile if you want to change instance profile (CPU, Memory)
    – SSH key: Add private key to access SSH from your laptop (How to generate sSH public And private key – Tech2Fun)
    – Image: Select OS type and OS version
    – Storage disk: Select boot disk size and Add new if you want to add more disk to your instance
    – Network interface: Select network type and bandwidth
  • In this case, we will create one more disk (100Gb) to store data. So click Add new
  • Double check all information again on the right side and then click Create. It will take about 3mins to create a Linux instance
3/ Configure the MinIO on Linux instance
  • Login to Linux instance, list disk info and look for the disk around 100Gb. Then format that disk

[root@virtualserver01 ~]# fdisk -l
Disk /dev/xvdb: 2 GiB, 2147483648 bytes, 4194304 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00025cdb

Device Boot Start End Sectors Size Id Type
/dev/xvdb1 63 4192964 4192902 2G 82 Linux swap / Solaris

Disk /dev/xvda: 25 GiB, 26843701248 bytes, 52429104 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xcf99ddef

Device Boot Start End Sectors Size Id Type
/dev/xvda1 * 2048 2099199 2097152 1G 83 Linux
/dev/xvda2 2099200 52428799 50329600 24G 83 Linux

Disk /dev/xvdc: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

[root@virtualserver01 ~]# mkfs.xfs /dev/xvdc
meta-data=/dev/xvdc             isize=512    agcount=4, agsize=6553600 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0, sparse=0
data     =                       bsize=4096   blocks=26214400, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=12800, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

  • Create the /stage directory, mount that disk to /stage permanent with /etc/fstab as below

[root@virtualserver01 ~]# mkdir /stage
[root@virtualserver01 ~]# vi /etc/fstab
………………………………
………………………………
# Filesystem for mksysb staging
/dev/xvdc    /stage     xfs    defaults    1 2

  • Download and configure the MinIO tool for Linux instance

[root@virtualserver01 ~]# cd /stage
[root@virtualserver01 ~]# wget https://dl.min.io/client/mc/release/linux-amd64/mc
[root@virtualserver01 ~]# chmod +x mc

  • Add IBM Cloud Object Storage to MinIO configure

[root@virtualserver01 ~]# mc alias set ibms3 https://s3.direct.tok.ap.cloud-object-storage.appdomain.cloud <access_key_id> <secret_key_access>

  • Test your connection and copy data from IBM Cloud Object Storage to /stage on Linux instance

Tags : AIXAutomationBeautifulSoupCloud Object StorageDevOpsLinux-UnixPythonSeleniumTestingtransfer data from cloud object storage to AIX
Tech2FunAdmin

The author Tech2FunAdmin

Leave a Response

error: Content is protected !!