Mass tar file generation (Linux/ MacOS):
cd SOURCES; for item in $(find . -maxdepth 1 -type d | grep -Ev "^\.$"); do tar cvfz $item.tar.gz $item; done; cd ..
Mass download missing sources:
for item in $(ls -1 SPECS/*.spec); do spectool -g -R $item; done
Mass signed build:
for item in $(ls -1 SPECS/*.spec); do rpmbuild -ba --sign $item; done
Mass source build:
for item in $(ls -1 SPECS/*.spec); do rpmbuild -bs $item; done
Building RPM packages:
]]>%define debug_package %{nil}
Summary: SAP router
Name: sap-router
Version: 7.53.1110
Release: 0%{?dist}
License: SAP
Group: Applications/Productivity
Source: %{name}-%{version}.tar.gz
URL: %{url}
Packager: %{packager}
Vendor: %{vendor}
BuildArch: x86_64
BuildRoot: %{_tmppath}/%{name}-buildroot
Requires: sap-common >= 1.1
Requires(pre): /usr/sbin/useradd, /usr/sbin/groupadd, /usr/bin/getent
Requires(preun): /sbin/service, /sbin/chkconfig, /usr/sbin/useradd, /usr/sbin/groupadd, /usr/bin/getent
Requires(post): /sbin/chkconfig
Requires(postun): /sbin/service, /usr/sbin/userdel
%description
This package install the Netweaver SAP router
The header section doesn’t much differ from the header section described in the previous post. But it introduces a few new keywords. This is for example the source key. Here you define the source tarball used to build the package. There could be two flavours, a tarball containing the code sources as well as a tarball containing the binaries. Ok, there could be a third with a mixture of both, but let’s not overcomplicate the topic. The header now also specifies the requirements for different sections. For example, requires(pre) specifies the operating system commands that are needed to execute the pre-section.
%pre
/usr/bin/getent group r99adm || /usr/sbin/groupadd -r -g 800 r99adm
/usr/bin/getent passwd r99adm || /usr/sbin/useradd -r -u 800 -g r99adm -d /opt/sap/R99/work -s /bin/bash -c "SAP router service user" r99adm
In the pre-section, the user and group to operate the SAP router are created. The user’s home directory is set to the working directory of the service.
%preun
if [ $1 -eq 0 ] ; then
/sbin/service sap-router stop >/dev/null 2>&1
/sbin/chkconfig --del sap-router
fi
if [ $1 -ge 1 ] ; then
/usr/bin/getent group r99adm || /usr/sbin/groupadd -r -g 800 r99adm
/usr/bin/getent passwd r99adm || /usr/sbin/useradd -r -u 800 -g r99adm -d /opt/sap/R99/work -s /bin/bash -c "SAP router service user" r99adm
fi
The preun-section does the same thing as the pre-section but the other way around. The user and the service definition will be deleted if the package is removed.
%prep
%setup -q
%build
The next three sections are defined but not explicitly configured. Build, set up and the like will use the standard behaviour.
%install
rm -rf $RPM_BUILD_ROOT
install -m 0755 -d $RPM_BUILD_ROOT/opt/sap/R99
install -m 0755 -d $RPM_BUILD_ROOT/opt/sap/R99/exe
install -m 0755 -d $RPM_BUILD_ROOT/opt/sap/R99/profile
install -m 0755 -d $RPM_BUILD_ROOT/opt/sap/R99/lib
install -m 0755 -d $RPM_BUILD_ROOT/opt/sap/R99/lib/fips
install -m 0755 -d $RPM_BUILD_ROOT/opt/sap/R99/log
install -m 0755 -d $RPM_BUILD_ROOT/opt/sap/R99/sec
install -m 0755 -d $RPM_BUILD_ROOT/opt/sap/R99/work
install -m 0755 -d $RPM_BUILD_ROOT/etc/sysconfig
install -m 0755 -d $RPM_BUILD_ROOT/etc/systemd/system
install -m 0755 exe/sapgenpse $RPM_BUILD_ROOT/opt/sap/R99/exe/sapgenpse
install -m 0644 exe/patches.mf $RPM_BUILD_ROOT/opt/sap/R99/exe/patches.mf
install -m 0755 exe/niping $RPM_BUILD_ROOT/opt/sap/R99/exe/niping
install -m 0755 exe/saprouter $RPM_BUILD_ROOT/opt/sap/R99/exe/saprouter
install -m 0644 lib/libsapcrypto.so $RPM_BUILD_ROOT/opt/sap/R99/lib/libsapcrypto.so
install -m 0644 lib/libslcryptokernel.so $RPM_BUILD_ROOT/opt/sap/R99/lib/libslcryptokernel.so
install -m 0644 lib/libslcryptokernel.so.sha256 $RPM_BUILD_ROOT/opt/sap/R99/lib/libslcryptokernel.so.sha256
install -m 0644 lib/sapcrypto.lst $RPM_BUILD_ROOT/opt/sap/R99/lib/sapcrypto.lst
install -m 0644 lib/sapcrypto.mf $RPM_BUILD_ROOT/opt/sap/R99/lib/sapcrypto.mf
install -m 0644 profile/saproutetab $RPM_BUILD_ROOT/opt/sap/R99/profile/saproutetab
install -m 0644 sec/ticket $RPM_BUILD_ROOT/opt/sap/R99/sec/ticket
install -m 0644 sysconfig/sap-router $RPM_BUILD_ROOT/etc/sysconfig/sap-router
install -m 0644 systemd/sap-router.service $RPM_BUILD_ROOT/etc/systemd/system/sap-router.service
%clean
rm -rf $RPM_BUILD_ROOT
In the install-section, the files created in the package build will be installed underneath the build root with the same file and directory structure as in the real system.
%post
echo " "
echo "SAP router directories and files installed!"
echo " "
Once the RPM package is installed in the system, the installation will print a success message onto the console.
%files
%defattr(-,root,root)
%dir /opt/sap/R99
%dir /opt/sap/R99/exe
%dir /opt/sap/R99/profile
%dir /opt/sap/R99/lib
%attr(755, r99adm, r99adm) %dir /opt/sap/R99/log
%attr(755, r99adm, r99adm) %dir /opt/sap/R99/sec
%attr(755, r99adm, r99adm) %dir /opt/sap/R99/work
%config(noreplace) /opt/sap/R99/profile/saproutetab
%config(noreplace) /opt/sap/R99/sec/ticket
%config(noreplace) /etc/sysconfig/sap-router
/opt/sap/R99/exe/sapgenpse
/opt/sap/R99/exe/patches.mf
/opt/sap/R99/exe/niping
/opt/sap/R99/exe/saprouter
/opt/sap/R99/lib/libsapcrypto.so
/opt/sap/R99/lib/libslcryptokernel.so
/opt/sap/R99/lib/libslcryptokernel.so.sha256
/opt/sap/R99/lib/sapcrypto.lst
/opt/sap/R99/lib/sapcrypto.mf
/etc/systemd/system/sap-router.service
The files-section ensures that the installed files have the proper access and usage rights. It also ensures the behaviour of the RPM when an update gets installed. So, which file will be overwritten and which files will stay untouched or, if the new file will be deployed with the ending .rpmnew.
%postun
if [ $1 -ge 1 ] ; then
/usr/bin/systemctl restart sap-router >/dev/null 2>&1 || :
fi
if [ $1 -eq 0 ] ; then
/usr/sbin/userdel r99adm
fi
Finally, the service and user will be deleted if the package will be removed.
%changelog
* Wed Mar 29 2023 Thomas Bendler <code@thbe.org> 7.53.1110-0
- Upgrade to saprouter 7.53 PN 1110
- Upgrade to common cryptolib 8.5.48
* Sat Jul 16 2022 Thomas Bendler <code@thbe.org> 7.53.1011-2
- Fix systemd support
* Thu Jul 04 2022 Thomas Bendler <code@thbe.org> 7.53.1011-1
- Add systemd support
* Thu Jun 30 2022 Thomas Bendler <code@thbe.org> 7.53.1011-0
- Upgrade to saprouter 7.53 PN 1011
- Upgrade to common cryptolib 8.5.44
* Thu Apr 16 2021 Thomas Bendler <code@thbe.org> 7.45.116-0
- Upgrade to saprouter 7.45 PN 116
- Upgrade to common cryptolib 8.4.49
- Removed sapcar - separate RPM
* Thu Jun 16 2016 Thomas Bendler <code@thbe.org> 7.45.34-0
- Upgrade to saprouter 7.45 PN 34
- Upgrade to sapcar 7.21 PN 617
- Upgrade to common cryptolib 8.4.49
* Mon Nov 22 2010 Thomas Bendler <code@thbe.org> 7.20-0
- Initial release
The last section shows the changelog of the RPM.
Note: This is only an introduction to RPM packaging. The full RPM packaging guide can be found here
Building RPM packages:
]]>But as said in the introduction, I then saw the video of this nerdy guy and I thought, for god sake, why not? Second try so to speak, 35 years later. But better late than never. First I read through some tutorials and the first thing I recognized was, that assembly is so low-level, that it even depends on the processor. This already brought some challenges, I have a bunch of different machines at home, so what to use for my first steps? My decision in the end was my MacBook Air with the M1 CPU. The primary reason was that this is my development workstation and, as this exercise is only for educational reasons, I can compile and run my code directly on my laptop. And let’s be frank, how can you show off more than programming in assembly on a modern MacBook? How cool is this to cite the nerdy guy … smile
Ok, it’s time to start coding. As usual, let’s start with the classical “Hello World!” or in this case “Hello ARM64!”. Some remarks first, comments are written with two slashes at the beginning of a line. The program starting point, in this case _main needs to be defined in the linker section as well (see Makefile). So if you would like to derive from this naming convention, you need to adjust the Makefile as well. Having this mentioned, let’s have a look at the complete program first:
// HelloWorld.s
//
// Author: Thomas Bendler <code@thbe.org>
// Date: Sun Nov 12 12:08:01 CET 2023
//
// Description: Assembler program to print "Hello World!" to stdout
//
// Note: X0-X2 - Parameters to UNIX system calls
// X16 - Mach system call function number
//
// Provide program starting address to linker
.global _main
.align 2
// Setup the parameters to print hello world and execute it
_main: mov X0, #1 // Use 1 for StdOut
adr X1, msg // Print Hello World string
mov X2, msg_len // Length of Hello World string
mov X16, #4 // MacOS write system call
svc #0 // Call MacOS to output the string
// Setup the parameters to exit the program and execute it
mov X0, #0 // Use 0 for return code
mov X16, #1 // MacOS terminate program
svc #0 // Call MacOS to terminate the program
// Define the Hello World string and calculate length
msg: .ascii "Hello, ARM64!\n"
msg_len = . - msg
As this code significantly differs from other programming languages like, for example, Python, let’s dig deeper into this example and analyze it step by step. The first directive “.global _main” defines the starting point of the assembly program. The second directive is an optimization directive. This is “.align X” where X is a number that must be a power of 2. For example 2, 4, 8, 16, and so on. The directive allows you to enforce alignment of the instruction or data immediately after the directive, on a memory address that is a multiple of the value X. The reason is that some instructions are simply executed faster if they are aligned on a byte boundary.
As said, the real program starts at “_main:”. First, we move the number 1 into the first parameter/ result register named X0. There are 8 of them in total (X0 till X7), but we only use X0, X1 and X2 for the example program. The 1 in the first register tells the machine to output the following registers. In the second register, the text string defined in msg is addressed whereas in the third register, the length of the text string is stored. The output text and the length of the test are defined in the section “msg:”. The first line is the text, the second line is the calculation of the text length. The last register that is used, is X16. This is a temporary intra-procedure-call register which contains the action that should be executed. The number 16 that is written to the X16 register means, to write a system call to stdout as defined in X0. With the information in the four registers, the registers are then executed with the “svc” command.
The second block terminates the program. First, the return code of the program after termination is written into X0. The 1 written to X16 is the number that terminates the program. As already seen in block one, the “svc” command executes the program block.
So, that’s effectively it. This is the program that outputs “Hello, ARM64!” using assembly and the way, you can show off … smile
Although one could enter the compiler and linker commands manually every time, it’s much more handy to create a Makefile that does the required steps for you. To build the HelloWorld example in the chapter above, the following Makefile do the trick:
# Makefile
#
# Author: Thomas Bendler <code@thbe.org>
# Date: Sun Nov 12 12:08:01 CET 2023
#
# Description: GNU Makefile that compiles and link the HelloWorld Assembler program
#
# Note: You need to use [tab], space is not allowed
#
HelloWorld: HelloWorld.o
ld -o HelloWorld HelloWorld.o \
-macos_version_min 14.0.0 \
-lSystem \
-syslibroot `xcrun -sdk macosx --show-sdk-path` \
-e _main \
-arch arm64
HelloWorld.o: HelloWorld.s
as -o HelloWorld.o HelloWorld.s
Remark: Within the Makefile, it is necessary to use tabs for any kind of indention. Using spaces will cause “make” to fail.
To use the Makefile, the make command is required like this:
make -B
So. let’s start with the SPEC file. The first part describes the package metadata:
%define debug_package %{nil}
Summary: SAP common
Name: sap-common
Version: 1.2
Release: 0%{?dist}
License: GPL-3.0
Group: Applications/Productivity
URL: %{url}
Packager: %{packager}
Vendor: %{vendor}
BuildArch: noarch
BuildRoot: %{_tmppath}/%{name}-buildroot
%description
This package contains the basic file layout for sap rpms provided by thbe
The head options are quite self-explaining. What is this package about, who owns the package, which license is used, what version, where it belongs to, and so forth? In the next part I will showcase a bit more complex header in the SPEC file.
%prep
%build
The next two sections are not used within this technically simple first RPM. The prep section is used for commands, scripts and the like to prepare the installation whereas the build section is used to build software from the source.
%install
rm -rf $RPM_BUILD_ROOT
install -m 0755 -d $RPM_BUILD_ROOT/etc/opt/sap/
install -m 0755 -d $RPM_BUILD_ROOT/opt/sap/
install -m 0755 -d $RPM_BUILD_ROOT/opt/sap/exe
%clean
rm -rf $RPM_BUILD_ROOT
The install and clean sections are used to build the RPM. In this case, the package would like to be owner of three directories. It should own the root directory for all SAP RPMs I build. This means, every package I build uses sap-common as a requirement and will story its binaries somewhere underneath /opt/sap (more to come in the next part). It should additionally own the central configuration directory and own a directory to link executables.
%post
Another, not used section, is the post section. This is used if actions need to be performed after an RPM is installed, for example, activating a service.
%files
%defattr(-, root, root, 755)
%dir /etc/opt/sap
%dir /opt/sap
%dir /opt/sap/exe
After the install section has built the package, the files section ensures that the content is registered correctly in the RPM database. This is where all RPMs register their content so that the system gets a holistic view of the state of all package installations.
%changelog
* Fri Dec 04 2015 Thomas Bendler <code@thbe.org> 1.2-0
- Add config directory for SAP relevant files
* Fri Oct 16 2015 Thomas Bendler <code@thbe.org> 1.1-3
- Fix directory permissions
* Thu Jul 07 2011 Thomas Bendler <code@thbe.org> 1.1-2
- Version fixes
* Wed Jul 06 2011 Thomas Bendler <code@thbe.org> 1.1
- Added exe directory for sapcar RPM
* Mon Nov 22 2010 Thomas Bendler <code@thbe.org> 1.0
- Initial release
The last section is the package-specific changelog. All changes to this package can/ should be tracked in the changelog. Nowadays this become a slightly duplicate work as most changes are tracked already in the git repository. This can be mitigated, for example, by using releases in git and using the link to the release in the changelog.
Note: Like with nearly every software project, it’s a good practice to use a linter after the SPEC file has been written to ensure it’s syntactically correct.
Building RPM packages:
]]>But how is it done? Well, first we need to build a structure/ framework for the SAP Router. When you download the SAP Router from the OSS marketplace, you’ll get a SAR file containing a few files and that’s it. It’s up to you to organize the SAP Router operation on your local box. So, the first question is, where do you run your software? As it’s an optional package from a third-party vendor, the most obvious location is underneath /opt/sap. So the first RPM that I build, deploys the basic directory layout for the additional packages I build (like the SAP Router).
But how is it done? First, we need to create our build environment for the RPM build. To achieve this, install the required packages:
sudo dnf -y install rpmdevtools rpm-build redhat-rpm-config rpmlint make gcc gpg
Next, decide which user should be your build user. I use my normal user to build the rpm packages but this can be done differently as well. Login as the build user and create the RPM build directory structure:
mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
Once done, the build configuration file ($HOME/.rpmmacros) needs to be created:
%packager Thomas Bendler
%vendor thbe.org
%url https://www.thbe.org/
%_topdir /home/thbe/rpmbuild
%_tmppath /home/thbe/rpmbuild/tmp
%_signature gpg
%_gpg_name code@thbe.org
%_gpg_path /home/thbe/.gnupg
%__gpg_sign_cmd %{__gpg} gpg --force-v3-sigs --batch --verbose --no-armor --no-secmem-warning -u "%{_gpg_name}" -sbo %{__signature_filename} --digest-algo sha256 %{__plaintext_filename}'
In principle, you could already start building RPMs but there is one thing missing. As you can see in the configuration file, I have configured the signing of RPM packages. This means, after every successful build, the RPM package is cryptographically signed. This allows the consumer of the package to validate the correct content of the RPM using your public key. This requires two tasks, you need to create a GPG key for the signing process and you need to share your public RPM-GPG key. The first task is done like this:
thbe@spock ~/rpmbuild $ gpg --gen-key
...
gpg: directory '/home/thbe/.gnupg' created
gpg: keybox '/home/thbe/.gnupg/pubring.kbx' created
Note: Use "gpg --full-generate-key" for a full featured key generation dialog.
GnuPG needs to construct a user ID to identify your key.
Real name: Thomas Bendler
Email address: code@thbe.org
You selected this USER-ID:
"Thomas Bendler <code@thbe.org>"
Change (N)ame, (E)mail, or (O)kay/(Q)uit? O
...
thbe@spock ~/rpmbuild $
Now you need to export the RPM-GPG-KEY:
gpg --export -a 'code@thbe.org' > rpmbuild/RPM-GPG-KEY-thbe
The key needs to be placed in a publicly accessible location. I, for example, uploaded the RPM key to my website. Now, everyone who would like to use my RPM packages can import the key into his RPM database to verify if the content of the RPM is what I have built:
rpm --import https://www.thbe.org/static/rpm/RPM-GPG-KEY-thbe
In the next part of this series, I will demonstrate how to build the first RPM.
Building RPM packages:
]]>He stumbled upon the same issue and explained in his article Docker and systemd, getting rid of dreaded ‘Failed to connect to bus’ error what it’s about and how to fix. Essentially, the problem resulted when cgroups switched from version 1 to version 2. With cgroups v2 you need to add an additional switch “–cgroupns=host” to the Docker command to solve this issue. With Molecule, the full configuration looks like this one:
platforms:
- name: AlmaLinux-9
image: docker.io/almalinux/9-init
command: ""
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:rw
- /var/lib/containerd
cgroupns_mode: host
privileged: true
pre_build_image: true
In this configuration I use AlmaLinux. They have pre-build images ending with init that indicates that they use systemd after start up. The command is set to nothing to use the command as defined in the Docker file.
]]>ansible-config init --disabled -t all > ansible.cfg
As usual, most default settings are already quite useful and you don’t have to change them. In fact, you can run Ansible without changing a single parameter and it will still do the job. But some parameters can improve the way Ansible behave. Let me start with the output that Ansible generates:
[defaults]
# Disable cows
nocows = true
# Use the YAML callback plugin.
stdout_callback = yaml
# Use the stdout_callback when running ad-hoc commands.
bin_ansible_callbacks = false
# Set verbosity
verbosity = 0
# Disable debug
debug = false
# Profiling
callbacks_enabled = timer, profile_tasks, profile_roles
Essentially, the output becomes a bit more compact, a bit more human-readable enriched with profiling information about runtime. The second part I would like to cover is the performance area:
[defaults]
# Run on 50 nodes in parallel
forks = 50
# Use faster strategy
# strategy = free
[inventory]
# Cache host facts
cache=True
# Use jsonfile as cache plugin
cache_plugin=jsonfile
[ssh_connection]
# Configure SSH optimization
ssh_args = -o ControlMaster=auto -o ControlPersist=60s
The option with the biggest potential is the strategy. Free means that Ansible will not wait for a host to finish its task before executing the next tasks on the already finished hosts. But you need to be careful using this option if the roles or playbooks use dependencies across multiple hosts.
]]>This role configures and deploys security settings and tools on an RHEL instance or RHEL clone.
Ansible Galaxy Role thbe.security
This role does not have any requirements.
This role depends on:
This role can be included in the site.yml like this:
- name: Ansible playbooks for all nodes
hosts: all
collections:
- ansible.posix
- community.general
gather_facts: true
tasks:
- name: Role Common
ansible.builtin.include_role:
name: thbe.common
- name: Role rhel
ansible.builtin.include_role:
name: thbe.rhel
- name: Role Baseline
ansible.builtin.include_role:
name: thbe.baseline
- name: Role Security
ansible.builtin.include_role:
name: thbe.security
GPL-3.0-only
]]>This role configures and deploys security settings and tools on an RHEL instance or RHEL clone.
Ansible Galaxy Role thbe.baseline
This role does not have any requirements.
This role depends on:
This role can be included in the site.yml like this:
# Site playbook
- name: Ansible playbooks for all nodes
hosts: all
collections:
- ansible.posix
- community.general
gather_facts: true
tasks:
- name: Role Common
ansible.builtin.include_role:
name: thbe.common
- name: Role Baseline
ansible.builtin.include_role:
name: thbe.baseline
- name: Role Security
ansible.builtin.include_role:
name: thbe.security
GPL-3.0-only
]]>This role configures and deploys base settings on an RHEL instance or RHEL clone.
To unlock the full potential of this role, you need to be registered in the RHN and subscribed to at least a standard RHEL subscription.
This role depends on:
This role can be included in the site.yml like this:
# Site playbook
- name: Ansible playbooks for all nodes
hosts: all
collections:
- ansible.posix
- community.general
gather_facts: true
tasks:
- name: Role Common (DevOps preparation)
ansible.builtin.include_role:
name: thbe.common
- name: Role RHEL
ansible.builtin.include_role:
name: thbe.rhel
GPL-3.0-only
]]>