getting a new centos server ready to be your minion with ... · copying our ssh keys to relevant...

Sysadmin Tasks : Ansible-Ready Centos 7

Getting a new Centos server ready to be your minion with Ansible Scenario

● Platforms ○ Server: Centos 7 ○ Admin laptop: Centos 7

● Sometimes you setup your server and then remember a bunch of other tasks that you could have automated; in this case we wanted the server to be an ansible minion so we could continue practicing / testing various automation activities. Doing this really only requires that the user “ansible” is created and given sudo rights, which is easy enough if the server is already stood up, but if we invest the time up front to create the custom image we can save time on several areas of setting up the server if we ever have to do a complete rebuild or if we expand more servers and want the same setup.

● Some areas to automate ○ Copying our SSH keys to relevant user .ssh/authorized_keys file ○ Generating ssh-keys for users ○ Any directory structures we want to create. ○ Additional user creation

● Requirements ○ A Centos ISO (i chose to use a minimal iso) ○ A remote server or lab setup with ssh activity.

■ Ansible depends on SSH functionality ○ Pykickstart

■ So we can validate our kickstart file’s syntax ○ Ansible

■ To control all the things. ○ Python

■ We’ll use a simple script to generate hashed passwords ○ Genisoimage

■ Allows us to use the mkisofs utility so we can create an iso

Steps ● Let’s get what we need; RHEL official documentation recommends we install pykickstart so we can use ksvalidator on our

kickstart file. If you don’t already have ansible on your admin pc, you’ll have to install that as well

$ yum install pykickstart $ yum install ansible $ um install genisoimage

● Before we get rolling with ansible we need to point it at our remote machine… so what we’ll do is make a backup of the

current ansible hosts file and then just create our own simple ansible hosts file

BAD X BINARIES


$ mv /etc/ansible/hosts /etc/ansible/hosts.bak

● Now our NEW ansible hosts file can just be as simple as a remote IP listing: below is an example

● If you want to specify groups of ansible machines you just label the groups in square brackets

● If your minion machine is running SSH on a non-standard port, you can just specify the port number in the hosts file ● For this document i also added an entry for localhost to test

○ [local] localhost

● When you’re done save the new /etc/ansible/hosts file and then we can run an adhoc command to test

● Now that we know ansible is working we can leave it be and work on creating our custom Centos image. ● The first thing we’ll do is make a mount point directory for the iso and a directory to copy our iso data to, since the ISO

will be mounted as read only

$ mkdir /mnt/iso /mnt/custom

● Then mount the iso to /mnt/iso

$ mount centos-image.iso /mnt/iso

BAD X BINARIES


● Noww we’ll copy the iso files to our /mnt/custom directory recursively ( -r)

$ cp -r /mnt/iso/* /mnt/custom/

● Once this is done we essentially just have to add our kickstart file into /mnt/custom/isolinux ○ We have a number of ways of doing this… we can build one from scratch or if we still have it, we can work off of

the anaconda-ks.cfg that was created in our root directory post install. ○ We can also attempt to create a pretty generic one ○ Below is an example of the one generated on my server

■ I PXE booted this machine so it retrieved an ISO from the web and then went through a text install process, which is why we see the url -- option there

#version=DEVEL # System authorization information auth --enableshadow --passalgo=sha512 # Use network installation url --url="http://mirror.centos.org/centos/7.5.1804/os/x86_64" # Use text mode install text # Run the Setup Agent on first boot firstboot --enable ignoredisk --only-use=sda # Keyboard layouts keyboard --vckeymap=us --xlayouts='' # System language lang en_US.UTF-8 # Network information network --bootproto=dhcp --device=enp2s0f0 --ipv6=auto --activate network --bootproto=dhcp --device=enp2s0f1 --onboot=off --ipv6=auto network --bootproto=dhcp --device=enp3s0f0 --onboot=off --ipv6=auto network --bootproto=dhcp --device=enp3s0f1 --onboot=off --ipv6=auto network --hostname=localhost.localdomain # Root password rootpw --iscrypted $6$SOME/HASH/HEREalkdsjf;alskdjfa;lsdjf # System services services --enabled="chronyd" # Do not configure the X Window System skipx # System timezone timezone US/Arizona --isUtc

BAD X BINARIES


user --name=user --password=$6$SOME/HASH/HERE/ --iscrypted # System bootloader configuration bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=sda autopart --type=lvm # Partition clearing information clearpart --all --initlabel --drives=sda %packages @core chrony kexec-tools %end %addon com_redhat_kdump --enable --reserve-mb='auto' %end %anaconda pwpolicy root --minlen=6 --minquality=1 --notstrict --nochanges --notempty pwpolicy user --minlen=6 --minquality=1 --notstrict --nochanges --emptyok pwpolicy luks --minlen=6 --minquality=1 --notstrict --nochanges --notempty %end

○ So working off the one above i’ll make some slight changes.

● For our new kickstart let’s do the following ○ Generate a root password hash and two user hashes; one for ansible, and one for default “user” ○ Drop into a python interpreter shell and execute the following commands (we’ve omitted our password hash

below)

○ We can do this for each user that we want to create a password for, that way the passwords will already be

encrypted in the new .iso we create, instead of hardcoded. ○ Once we have our user password hashes we can paste them into our kickstart file

BAD X BINARIES


# System Auth Info auth --enableshadow --hashalgo=sha512 # Install with text mode install text # run setup agent on first boot firstboot --enable # keyboard layout and system language keyboard --vckeymap=us --xlayouts=’’ lang en_US.UTF-8 # Setup root user rootpw --iscrypted $INSERTHASH-HERE$ # Setup other users user --name=user --password=$INSERT HASH HERE$ --iscrypted user --name=ansible --password=$INSERT HASH HERE --iscrypted # Simple network config network --bootproto=dhcp --activate # timezone timezone US/Arizona --isUtc # system bootloader config bootloader --append=” crashkernel=auto” --location=mbr --boot-drive=sda autopart --type=lvm # partition clearing clearpart --all --initlabel --drives=sda %packages @core chrony kexec-tools %end

● Once we think we’ve finished the file we can run ksvalidator on it; if we don’t get any output, we should be good to go

● Next we’ll copy our kickstart file into the isolinux directory

BAD X BINARIES


$ cp test-ks.cfg /mnt/custom/isolinux/

● And the next thing we need to do is add kickstart to the boot options in the isolinux.cfg file ; this can be done with sed for speed, but if you want to view the file as you edit it you can just add a new section that will show up on the boot menu.

$ vim /mnt/custom/isolinux/isolinux.cfg

● This should point our install properly at our kickstart file. ● Let’s edit it again and add some light scripting to the %post section of the kickstart to handle some remaining tasks

○ We want to be able to ssh into the target machine after install. To do this we’ll need to make .ssh directories for the users we want, and we’ll have to make authorized_keys files with our public keys for reach of the users as well.

○ We also have to make sure that .ssh and .ssh/authorized_keys have the appropriate permissions set, otherwise ssh won’t work and will throw errors

■ .ssh dir : 700 permissions ■ .ssh/authorized_keys : 600 permissions

○ Back in our test-ks.cfg file, we need to add a post section to the end of the file that looks similar to this

%post mkdir ~/.ssh echo “insert your id_rsa.pub key here” >> ~/.ssh/authorized_keys chmod 700 ~/.ssh chmod 600 ~/.ssh/authorized keys

BAD X BINARIES


mkdir /home/ansible/.ssh echo “insert your id_rsa.pub key here” >> /home/ansible/.ssh/authorized_keys chown ansible:ansible /home/ansible/.ssh chown ansible:ansible /home/ansible/.ssh/authorized_keys chmod 700 /home/ansible/.ssh chmod 600 /home/ansible/.ssh/authorized_keys %end

● After we ad these bits we can run ksvalidator, and then we’ll be ready to create our iso and give it a test run. You may

want to create copy of the file structure prior to creating to the iso. Change directories to the root directory of our copied iso . when you run ls it should look like ths

● Then we can run mkisofs to create the iso using the command below; note the space between the two dots at the end,

which are important.

$ mkisofs -o /tmp/boot.iso -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -V "CentOS 7 x86_64" -R -J -v -T isolinux/. .

○ -o : specifies the output file we want, an aribitrary name/location ○ -b : eltorito boot image; specifies the boot image path, ○ -c : boot catalog specifies the path and filename of the boot catolig required for an etorito bootable cd; must be

relative to the source path specified by genisoimage ○ -no-emul-boot : the boot image used to create el torito bootable CDs is a “no emulation” image; system will

load and execute this image without performing disk emulation ○ -boot-load-size : specifies the number of VIRTUAL 512-byte sectors to load in no-emulation moad; default is to

load the entire boot file; some bioses have problems if this number isn’t a multiple of 4 ○ -boot-info-table : specifies that a 56-byte table with info of the CD-ROM layout wil be patched in an offser 8 in

the boot file; the boot file is modified in the source filesystem, so make a copy of this file if it cannot be easily regenerated!

○ -V : volume id : specifies the volume ID or label to be written into the master block; space for 32 characters ○ -R : generate SSUSP and RR records using the Rock Ridge protocol to further describe the files on the ISO9660

filesystem ○ -J : make Joliet directory records in addition to regular ISO9660 filenames. Useful when disk will be used on

windows machines. Joliet filenames are specified in Unicode and each path component can be up to 64 unicode characters long. Only Microsoft and linux systems can read joliet extensions

BAD X BINARIES


○ -v : standard verbose option ○ -T : translation tabl -- generate a file TRANS.TBL in each directory on the CD-ROM which can be used on

non-Orck Ridge-capable systems to establish correct names. Also gives information on major and minor numbers for block and character devices, and each symlink has the name of the link file given.

● Afterward we should have an iso in tmp

● Now we can boot, either on a machine or VM to test ( we used virt-manager to test)

● When we boot we should be greated with the option we specified earler… fingers crossed the kickstart works

● And fail… there’s an issue on line 31 of our file which has to do with sda

○ This is probably because our VM uses a virtual drive, which gets labeled “VDA” but the install works fine without this line

○ The sda issue also appears in “--bootdrive=sda”, which we could test further

● Wee can comment out the line and try to generate the iso again ● Another test run and… womp womp… our keyboard layout was invalid

BAD X BINARIES


● In our test-ks.cfg file, the offending line appears to be

● So let’s see if it survives without the “--xlayouts=’’ “portion ● This seems more promising; it gets us to the text-based install menu; looks like we included some directives for

nonexistent packages

● Fortunately the installation continues without the erroneous package but we should remove it from the kickstart ffile so

the packages section looks like that below

● As the installation continues, things look promising.

BAD X BINARIES


● Look’s like we’re trying to install to a nonexistent disk though, we can choose one by selecting option 5 above then

pressing enter ● After stepping through these options, we discover the appropriate disk is VDA so we can now continue our install ● All appears to go well… the real test will be whether or not our post-install scripts work

● Well… it didn’t seem like any errors were thrown.. So it must have worked

● Testing our login as root works… ● And it looks like the directory was created for /home/ansible/.ssh with the appropriate permissions

BAD X BINARIES


● But root has no corresponding .ssh/authorized_keys directory… ● Testing ssh as ansible from host to our guest vm fails… this is initially because we set the wrong permissions on our

authorized_keys file; this should be 644 and NOT 600, so we should modify that in the test-ks.cfg before we forget. ● Once we manually configure the permissions on the vm we confirm successful login; so our networking was automatically

configured with dhcp as specified in the kickstart file; another plus!

● I’m also willing to bet that the issue with the root ssh directories is that it was specified using the “~/” notation, which

was probably not interpreted correctly by the shell we were in, so we can just specify the full path in our kickstart file. ● The new %post section should look like this

%post mkdir /root/.ssh echo “insert your id_rsa.pub key here” >> /root/.ssh/authorized_keys chmod 700 /root/.ssh chmod 644 /root/.ssh/authorized keys mkdir /home/ansible/.ssh

BAD X BINARIES


echo “insert your id_rsa.pub key here” >> /home/ansible/.ssh/authorized_keys chown ansible:ansible /home/ansible/.ssh chown ansible:ansible /home/ansible/.ssh/authorized_keys chmod 700 /home/ansible/.ssh chmod 644 /home/ansible/.ssh/authorized_keys usermod -aG wheel ansible %end

● With our corrections let’s kill our vm and recreate the iso; using the same mkisofs command above. Note that we added the “usermod” command also to make the ansible user a sudoer by adding him to the wheel group.

● Don’t forget to delete the “kexec” portion to avoid some error messages ● Run the install from kickstart one more time to test

● Yay, so far so good as root! Now to test the login from host → guest VM

● Another win… Now to test ssh as ansible to ansible and as root to ansible

BAD X BINARIES


● Winning again…. Now our final test; to see if the ansible user is a sudoer… and that fails… we double check that the user

was added to the wheel group, and we double check the sudo config with the visudo command, to no avail. There must have been an issue with copying over the password hash…

● we’ll have to regenerate this hash with python again and give it another go. ○ Double check the hash after you paste it into the kickstart file ○ Be sure that you copy the correct hash too if you execute crypt.crypt(pw) multiple times as this will return a

different value... ○ Below is an example of this behavior

● One more test run and we’re all set

○ Confirm root / ansible login with expected behavior - check ○ Confirm ansible user is a sudoer - check ○ Confirm we can ssh into the new machine from different accounts on our local machine - check ○ Confirm we can execute ad-hoc ansible command on our ansible minion

BAD X BINARIES


● Easy enough, right?

Notes ● We could probably do more digging into the python documentation to learn more about the crypt() method to see if we’re

making the best use of it here, but based off of our brief overview of it, without supplying a salt in addition to a password as arguments to the method, then it used the strongest salting option by default, so we should be fine.

● We could also create custom ISO’s to include various packages that we want, but this is probably not desirable as maintaining multiple isos could become a storage concern if you’re doing this on a home lab with limited space.

● We probably want to sign the image we create, so looking into gpg and signing iso’s is probably a must if we want this in production of any sort; at least it’s something to be aware of.

● We probably also want to consider adding a modicum of security either through ansible playbook or kickstart by adding an install of fail2ban to protect ssh and installing OSSEC for HIDS.

○ The latter is best if you configure an email relay that way you can get alerts direct to your inbox; we may cover this in the future.

Sources 1. Kickstart %post section --

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/installation_guide/s1-kickstart2-postinstallconfig

2. Python documentation on the crypt method -- https://docs.python.org/3/library/crypt.html

BAD X BINARIES



https://docs.python.org/3/library/crypt.html


3. Updates repo https://unix.stackexchange.com/questions/392731/how-can-i-include-all-recent-updates-during-kickstart-install-on-fedora-centos-w

BAD X BINARIES

https://unix.stackexchange.com/questions/392731/how-can-i-include-all-recent-updates-during-kickstart-install-on-fedora-centos-w

https://unix.stackexchange.com/questions/392731/how-can-i-include-all-recent-updates-during-kickstart-install-on-fedora-centos-w

getting a new centos server ready to be your minion with ... · copying our ssh keys to relevant...

Documents