aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorSergey Nazaryev <sergey@nazaryev.ru>2020-08-08 16:28:09 +0300
committerSergey Nazaryev <sergey@nazaryev.ru>2020-08-08 16:28:09 +0300
commit279ba2e71c1f6a89c59e78f5a8451a5eccbbb76f (patch)
tree4c9ba2ca202a7f11aba9d4c6af4e8a555f73d33a
parent025fb970c997d1efcedfbf19c014e0ac421f5109 (diff)
downloadportfolio-279ba2e71c1f6a89c59e78f5a8451a5eccbbb76f.zip
portfolio-279ba2e71c1f6a89c59e78f5a8451a5eccbbb76f.tar.gz
portfolio-279ba2e71c1f6a89c59e78f5a8451a5eccbbb76f.tar.bz2
Add two new English posts
-rw-r--r--_posts/2016-11-21-jjb-folders.md22
-rw-r--r--_posts/2020-08-07-arm-chroot.md163
2 files changed, 173 insertions, 12 deletions
diff --git a/_posts/2016-11-21-jjb-folders.md b/_posts/2016-11-21-jjb-folders.md
index 221721e..93bf267 100644
--- a/_posts/2016-11-21-jjb-folders.md
+++ b/_posts/2016-11-21-jjb-folders.md
@@ -1,23 +1,21 @@
---
layout: post
-title: "Поддержка папок в Jenkins Job Builder"
-description: "Как привязать job к определённой папке?"
+title: "Folders support in Jenkins Job Builder"
+description: "How to put the Jenkins job to the folder?"
---
-Для того, чтобы в [Jenkins Job Builder (jjb)](http://docs.openstack.org/infra/jenkins-job-builder/)
-создать и обновлять job в определённой папке, достаточно прописать в
-названии job'а полный путь а-ля:
+If you want to create and update your Jenkins Job in some folder
+using [Jenkins Job Builder (jjb)][jjb], you need to put the name of the
+folder into the name of the job:
```yaml
- job:
name: "developer/zar/buildroot"
```
-Таким образом job с названием `buildroot` попадёт в директорию
-`developer/zar`.
+**Note**: at this moment Jenkins Job Builder isn't able to create
+folder on demand by itself. This feature is not supported yet, but it
+will be sometimes, I hope. You can track the progress on this feature
+in Gerrit [here](https://review.openstack.org/#/c/134307/).
-**Важно**: на данный момент `jjb` не умеет
-самостоятельно создавать директории: поддержку этой фичи внедряют уже
-несколько лет, а проследить за прогрессом можно в официальном
-Gerrit'е создателей `jjb` [вот
-тут](https://review.openstack.org/#/c/134307/).
+[jjb]: http://docs.openstack.org/infra/jenkins-job-builder/
diff --git a/_posts/2020-08-07-arm-chroot.md b/_posts/2020-08-07-arm-chroot.md
new file mode 100644
index 0000000..984795f
--- /dev/null
+++ b/_posts/2020-08-07-arm-chroot.md
@@ -0,0 +1,163 @@
+---
+layout: post
+title: "Build software for your ARM device using virtualization"
+---
+
+Yesterday we've discussed one guy's problem in the chat. He had a
+device based on old ARM SoC with installed Linux on it. His target
+was to launch his own Qt application on it, however the Qt Framework
+on his device was too old.
+
+He asked the device vendor for cross-toolchain, kernel sources or at
+least instruction for building rootfs but his request was not
+satisified. Hopefully, there is at least `gcc` compiler on the
+device. In this case the option of building newer Qt Framework
+"in-place" on the device is not so bad.
+
+Of course, cross-toolchain is the best solution for this problem. But
+sometimes you don't have anything else but the working device with
+the operating system on SD card. It's almost miraculous if you also
+have a compiler on this system.
+
+However, you can't build *fat* software on the low-end ARM system, it
+will take days for builiding something like Qt Framework. Therefore,
+in this case you have only two options:
+
+ - build your own cross-toolchain using the appropriate `libc` and
+ `Linux kernel` version in accordance with the device's rootfs;
+
+ - dump the rootfs of the device and launch the building process
+ inside it on your host system.
+
+I want to discuss only the second option. So, in this case you can
+make virtual machine with full virtualization using `qemu-system-arm`
+and put the dumped rootfs and kernel into it. For full virtualization
+you need to emulate ARM processor and all needed hardware and run
+kernel code on it along with the code of all drivers, file systems
+and so on. It requires too many resources of your host system and
+still it will work very slow.
+
+However, there is another way to do it: you can emulate only ARM
+instructions for syscalls and "convert" it to the x86 syscalls. For
+this purpose there are several `qemu` utilties for it for all popular
+CPU architectures. For testing purpose, you can download any static
+compiled ARM binary for Linux (e.g. [busybox][busybox]) and launch
+it on your host system (`qemu-arm busybox`).
+
+Moreover, you can chroot into your ARM rootfs, if you put
+`qemu-arm-static` into this rootfs. **Note**: `qemu-arm-static` should
+be static compiled because it can't access in runtime to your host
+libraries while being in chroot. Usually, there is `qemu-arm-static`
+package in your Linux repository (at least there are in Arch Linux,
+Alpine Linux and all Debian derivatives).
+
+So, if you try to chroot into it, the `/bin/bash` compiled for ARM
+systems will be launched via `qemu-arm-static` on your host x86
+system. Just a miracle!
+
+In addition, in chroot you will have access to the all cores of your
+processor, all available RAM and disk space.
+
+I compared full virtualization performance with performance of
+syscalls-only emulating: I run Qt 5.15.0 `configure` script (in the
+process of configuring it also builds `qmake`).
+
+Full virtualization:
+
+```
+time ./configure -skip qt3d -no-warnings-are-errors -release \
+ -recheck-all -opensource \
+ -confirm-license -nomake examples -nomake tests \
+ -c++std c++17 -I /usr/include/xcb/ -xcb-xlib -xcb \
+ -feature-thread -feature-xkbcommon -qt-libpng -qt-libjpeg \
+ -qt-zlib -I /usr/include/xcb/ --recheck-all -skip wayland \
+ -skip qtwebengine -skip qtwayland
+...
+real 91m49.471s
+user 78m43.608s
+sys 7m22.934s
+```
+
+Syscalls-only emulating:
+
+```
+time ./configure -skip qt3d -no-warnings-are-errors -release \
+ -recheck-all -opensource \
+ -confirm-license -nomake examples -nomake tests \
+ -c++std c++17 -I /usr/include/xcb/ -xcb-xlib -xcb \
+ -feature-thread -feature-xkbcommon -qt-libpng -qt-libjpeg \
+ -qt-zlib -I /usr/include/xcb/ --recheck-all -skip wayland \
+ -skip qtwebengine -skip qtwayland
+...
+real 17m22.799s
+user 16m59.336s
+sys 0m23.783s
+```
+
+As you see, the second variant was faster by the factor of 5 (17
+minutes vs 91 minutes) in comparison with the first one. Both of
+variants used only one CPU core. Of course, it can be highly improved
+if we make `./configure` use all of cores while building `qmake`.
+Despite this improvement, 17 minutes is still very slow, if we
+compare it with native build: the same test on x86-based system
+took only 2 minutes.
+
+If you are interested, you can try this trick yourself. This is a
+small instruction for getting chrooted into ARM rootfs on the example
+of `Raspberry Pi OS`:
+
+1. Download `Raspberry Pi OS` lite image and unzip it:
+
+ ```bash
+ wget -O image.zip http://downloads.raspberrypi.org/raspios_lite_armhf_latest
+ unzip image.zip
+ rm -f image.zip
+ ```
+
+2. Extract rootfs from OS image:
+
+ ```bash
+ sudo su
+ modprobe loop
+ losetup -fP --show 2020-05-27-raspios-buster-lite-armhf.img
+ mount /dev/loop0p2 /mnt
+ cd /mnt
+ cp -r * /home/xxx/rootfs
+ ```
+
+3. Install additional packages on your host system (it is for Debian;
+ however, for Arch Linux there is `qemu-user-static` package in
+ AUR):
+
+ ```bash
+ sudo apt-get install qemu-user-static qemu-user-static
+ ```
+
+4. Copy `qemu-arm-static` into target rootfs:
+
+ ```bash
+ cp "$(which qemu-arm-static)" /home/xxx/rootfs/usr/bin
+ ```
+
+5. Mount dev, sys, proc, run, tmp into target rootfs (don't forget to
+ unmount it afterwards):
+
+ ```bash
+ mount --bind /dev /home/xxx/rootfs/dev
+ mount --bind /sys /home/xxx/rootfs/sys
+ mount --bind /proc /home/xxx/rootfs/proc
+ mount -t tmpfs tmp /home/xxx/rootfs/tmp
+ mount -t tmpfs run /home/xxx/rootfs/run
+ ```
+
+6. Chroot into target rootfs:
+
+ ```bash
+ chroot /home/xxx/rootfs/
+ export PATH=/bin:/sbin:/usr/bin:/usr/sbin
+ echo "nameserver 8.8.8.8" > /etc/resolv.conf
+ ```
+
+7. Success! Now you are chroot'ed in ARM-based rootfs. Good luck!
+
+[busybox]: https://busybox.net/downloads/binaries/1.31.0-defconfig-multiarch-musl/busybox-armv7l